Skip to content

pr-review engine: COPILOT_API_MODEL unbound var silently disables the cross-engine rubber-duck tier (engine.sh:431) #881

Closed as duplicate

Description

@don-petry

Summary

On every claude-primary PR review (the default engine), the cross-engine rubber-duck (tier-2) is silently disabled by an unbound variable error. The cascade falls back to deep-review-only, so the o4-mini rubber-duck cross-check never runs.

Evidence (live v1.8.0 / ring-0 run)

Observed in pr-review-trigger run 27904795931 (job 82571409998), 2026-06-21, which ran the agent pinned at pr-review/next (ded84ce = v1.8.0). The run still concluded success, masking the defect:

scripts/engine.sh: line 431: COPILOT_API_MODEL: unbound variable
[tier2] rubber duck did not produce valid JSON — continuing with deep review only

Root cause

  • set_engine_config's claude) branch sets the cross-engine duck to copilot: DUCK_ENGINE="copilot" / DUCK_MODEL="o4-mini" (scripts/engine.sh:87-89).
  • But COPILOT_API_MODEL's default is assigned only inside the copilot) branch: COPILOT_API_MODEL="${COPILOT_API_MODEL:-openai/o4-mini}" (scripts/engine.sh:147).
  • The rubber-duck path invokes _copilot_invoke, which references $COPILOT_API_MODEL at scripts/engine.sh:431 (and :436).
  • With REVIEW_ENGINE=claude (default), the copilot) branch never runs, so COPILOT_API_MODEL is unset. Under set -euo pipefail (set -u), the reference aborts the duck subprocess → invalid JSON → tier-2 skipped.

Note: DUCK_MODEL="o4-mini" is set, but _copilot_invoke keys off COPILOT_API_MODEL (the GitHub Models API id, e.g. openai/o4-mini), not DUCK_MODEL — so setting the duck model alone doesn't cover this path.

Impact

  • Cross-engine rubber-duck (independent o4-mini second opinion) is off on all claude-primary reviews — reduced review coverage, no error surfaced to the run conclusion (it stays green).
  • Latent for any non-copilot primary that uses the copilot duck.

Suggested fix

Give COPILOT_API_MODEL an unconditional default so it's defined whenever the copilot engine is invoked (primary or rubber-duck), e.g. hoist the default out of the copilot) branch (set it alongside the other unconditional exports near scripts/engine.sh:167-170), or default at point-of-use in _copilot_invoke (${COPILOT_API_MODEL:-openai/o4-mini}). Add a regression test asserting that a claude-primary config leaves COPILOT_API_MODEL bound (and that the duck path doesn't trip set -u).

Found while verifying v1.8.0 ring-0 run health.

Activity

  1. github-actions commented on Jun 21, 2026

    @github-actions
    Contributor

    Thanks for the exceptionally thorough report — the run ID, line references, and root-cause trace make this very easy to follow! Since both proposed approaches (hoisting the COPILOT_API_MODEL default unconditionally vs. defaulting at point-of-use in _copilot_invoke) would fix the unbound variable, could you share which you'd recommend and whether either has been validated against the existing shell test suite?

  2. don-petry commented on Jun 21, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #881 — pr-review engine: COPILOT_API_MODEL unbound var silently disables the cross-engine rubber-duck tier (engine.sh:431)

    Scope

    Give COPILOT_API_MODEL an unconditional default in set_engine_config so the copilot-backed rubber-duck path stays bound (and doesn't trip set -u) when a non-copilot engine (e.g. claude) is primary.

    Implementation Checklist

    • scripts/engine.sh — hoist the COPILOT_API_MODEL="${COPILOT_API_MODEL:-openai/o4-mini}" default out of the copilot) branch so it is set unconditionally (alongside the other unconditional exports near the end of set_engine_config)

    Tests to Write

    • New tests/dev-lead/unit/test_engine_duck_model.bats:
      • claude-primary config leaves COPILOT_API_MODEL bound to openai/o4-mini
      • sourcing engine.sh as claude under set -u and referencing $COPILOT_API_MODEL does not abort (regression for engine.sh:431)
      • explicit COPILOT_API_MODEL override is still honored under claude
      • copilot-primary still binds COPILOT_API_MODEL (no regression)

    Test Command

    bats tests/dev-lead/unit/

  3. don-petry commented on Jun 21, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Implementation Complete

    Plan Execution

    • scripts/engine.sh — hoisted the COPILOT_API_MODEL="${COPILOT_API_MODEL:-openai/o4-mini}" default out of the copilot) branch so it is set unconditionally after the case, before the export block. The existing export ... COPILOT_API_MODEL line now always exports a bound value.
    • tests/dev-lead/unit/test_engine_duck_model.bats — new regression suite (5 tests).

    Root cause / fix

    The claude) branch wires the cross-engine duck to copilot (DUCK_ENGINE=copilot), but COPILOT_API_MODEL's default was assigned only inside the copilot) branch. Under a claude-primary review (the default) that branch never runs, leaving COPILOT_API_MODEL unset; copilot_chat's bare $COPILOT_API_MODEL reference (engine.sh:431/436) then aborted under set -u, silently skipping tier-2. The default is now unconditional, so the duck path stays bound for any non-copilot primary.

    Test Results

    New regression suite (red before fix on tests 1/2/5, green after):

    1..5
    ok 1 duck-model: claude-primary config leaves COPILOT_API_MODEL bound to default
    ok 2 duck-model: sourcing as claude under set -u does not trip on COPILOT_API_MODEL (engine.sh:431 regression)
    ok 3 duck-model: explicit COPILOT_API_MODEL override is honored under claude
    ok 4 duck-model: copilot-primary still binds COPILOT_API_MODEL to default (no regression)
    ok 5 duck-model: gemini-primary also leaves COPILOT_API_MODEL bound (latent non-copilot duck path)
    

    Full dev-lead unit suite: 577 passed, 0 failed (bats tests/dev-lead/unit/).

    Files Changed

    • scripts/engine.sh: moved the COPILOT_API_MODEL default + explanatory comment out of the copilot) branch to an unconditional assignment in set_engine_config (before the exports). No model values changed; copilot-primary behavior is byte-identical.
    • tests/dev-lead/unit/test_engine_duck_model.bats: new — asserts claude/gemini/copilot primaries all leave COPILOT_API_MODEL bound to openai/o4-mini, that an explicit override is honored, and that sourcing as claude under set -u + a bare $COPILOT_API_MODEL reference does not abort.

    Verification

    • bash .dev-lead/scripts/dev-lead-lint.sh → all checks passed (shellcheck on 72 files + agent frontmatter + CODEOWNERS).
    • shellcheck scripts/engine.sh → exit 0 (only the two pre-existing SC2015/SC1091 info notes at line 276, unrelated to this change).

    Notes

  4. added a commit that references this issue on Jun 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBug reportsdev-leadFor dev-lead agent pickupneeds-human-reviewFlagged by automated PR review agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions