Enhance dev-lead and pr-review engines for Gemini 3.5 models

Summary

Upgrade the dev-lead and pr-review agent engines to support the new Gemini 3.5 model family (starting with gemini-3.5-flash) with safe rollout controls, capability-aware routing, and observability.

Why

Current implementation appears optimized around older Gemini variants. Gemini 3.5 introduces improved function-calling/tool-use behavior and updated SDK/API patterns that are directly relevant to both dev-lead (planning/orchestration) and pr-review (code reasoning/review quality).

This enhancement should reduce drift from current Google model best practices and allow us to tune for latency/quality/cost across both agents.

Research notes (official docs)

Gemini docs now include current Gemini 3.5 pages and examples using model IDs like gemini-3.5-flash.
Function-calling docs show Gemini 3.5 usage and emphasize explicit function declarations + structured call handling.
Migration docs emphasize moving to Google GenAI SDK (google-genai / @google/genai) and centralized client usage.
Model listings can lag or vary by page/locale, so we should avoid hard-coding assumptions and rely on capability/config checks at runtime.

Proposed approach

Configurable model registry
- Add per-agent model config (default + fallback + override), e.g.:
  - DEV_LEAD_MODEL=gemini-3.5-flash
  - PR_REVIEW_MODEL=gemini-3.5-flash
- Keep safe fallback (e.g., existing 2.5 model) behind feature flags.
Capability-aware selection
- Route by task type and required capabilities (tool-calling, long-context, structured output).
- Fail closed with explicit errors when requested model/capability is unavailable (no silent downgrade unless fallback explicitly enabled).
Provider client modernization
- If still on legacy SDKs, migrate provider adapter to Google GenAI SDK patterns.
- Normalize request/response schema for function calls across both agents.
Prompt/runtime compatibility updates
- Validate tool-call schema strictness for Gemini 3.5.
- Ensure pr-review output contracts (JSON/markdown sections) remain deterministic.
Observability + rollout
- Add per-agent metrics/log tags: model ID, token usage, latency, tool-call count, error class.
- Ship behind staged rollout flag:
  - Phase 1: opt-in (internal)
  - Phase 2: default-on for one agent
  - Phase 3: default-on for both agents

Deliverables

Engine config support for Gemini 3.5 model IDs per agent.
Provider adapter updates for SDK/API compatibility.
Routing + fallback logic with explicit error handling.
Regression tests for:
- dev-lead planning/tool orchestration
- pr-review multi-file review quality and output schema
Rollout docs + env var examples.

Acceptance criteria

Both agents can be configured to run on gemini-3.5-flash in non-prod and prod.
No schema regressions in pr-review output contract.
Tool-calling reliability is at least baseline (or better) vs current model.
Latency/cost deltas are measured and documented.
Fallback behavior is explicit, tested, and controllable.

Risks / open questions

Exact availability of specific Gemini 3.5 variants across environments/regions.
Potential function-calling behavior differences that may require stricter tool schema validation.
Whether to split routing defaults by agent objective (quality-first pr-review vs speed-first dev-lead).

References

Dev-Lead Implementation Plan

Issue: #382 — Enhance dev-lead and pr-review engines for Gemini 3.5 models

Scope

Add per-tier Gemini model configurability and a safe-rollout fallback chain to scripts/engine.sh, mirroring the existing Claude chain pattern, so dev-lead and pr-review workflows can opt into Gemini 3.5 model IDs (e.g. gemini-3.5-flash) via env vars without code changes — with fail-closed behavior when every model in the chain is rate-limited.

Implementation Checklist

scripts/engine.sh: honor GEMINI_{TRIAGE,DEEP,AUDIT,ACTION,SINGLE}_MODEL env-var overrides in the gemini branch of set_engine_config
scripts/engine.sh: introduce GEMINI_{TRIAGE,DEEP,AUDIT,ACTION,SINGLE}_MODEL_CHAIN env vars (default: empty) for staged rollout
scripts/engine.sh: add _gemini_chain_invoke helper parallel to _claude_chain_invoke — walks comma-separated chain, rate-limited attempts fall through, non-rate-limit failures propagate immediately, all-rate-limited returns 2
scripts/engine.sh: wire _gemini_chain_invoke into the gemini branches of run_triage, run_agentic, and run_writer only when a chain is configured; otherwise current single-model behavior is preserved
scripts/engine.sh: preserve explicit model argument pinning (chain expansion only when caller used the tier default — same contract as Claude)
docs/dev-lead/spec.md or docs/pr-review-agent/setup.md: short rollout-vars section with GEMINI_*_MODEL / GEMINI_*_MODEL_CHAIN examples

Tests to Write

tests/dev-lead/unit/test_engine_chain.bats: gemini chain — first model succeeds (no fallback attempted)
gemini chain — first model rate-limited → falls through to second
gemini chain — all rate-limited → returns exit 2
gemini chain — non-rate-limit failure → propagates immediately
gemini chain — GEMINI_*_MODEL override is honored when no chain is set
gemini chain — explicit caller-pinned model bypasses chain expansion (regression guard)
gemini chain — writer uses CLAUDE_ACTION_MODEL_CHAIN/GEMINI_ACTION_MODEL_CHAIN symmetrically per engine

Test Command

bats tests/dev-lead/unit/test_engine_chain.bats for the focused unit tests; bats tests/dev-lead/unit/ tests/*.bats for the full bats suite, plus shellcheck --severity=warning -x scripts/engine.sh for lint.

Dev-Lead: Implementation Complete

Plan Execution

scripts/engine.sh: per-tier GEMINI_{TRIAGE,DEEP,AUDIT,ACTION,SINGLE}_MODEL env overrides honored in set_engine_config
scripts/engine.sh: GEMINI_{TRIAGE,DEEP,AUDIT,ACTION,SINGLE}_MODEL_CHAIN env vars (empty default → preserves single-model behavior)
scripts/engine.sh: _gemini_chain_invoke helper mirrors _claude_chain_invoke (rate-limit → walk chain; non-rate-limit → propagate; all RL → exit 2; empty chain → config error rc=1)
scripts/engine.sh: chain wired into run_triage, run_agentic, run_writer
scripts/engine.sh: explicit caller-pinned model arg bypasses chain expansion (same contract as Claude)
scripts/engine.sh: token-record logging records the model that actually produced output (chain-aware)
tests/dev-lead/fixtures/engines/stub-gemini: extended to support STUB_ENGINE_EXIT_BY_MODEL / STUB_ENGINE_RESPONSE_BY_MODEL / STUB_ENGINE_RECORD_MODELS (parity with stub-claude)
docs/engine-model-rollout.md: phased rollout doc with phase-1/2/3 env var examples

Test Results

New tests (tests/dev-lead/unit/test_engine_gemini_chain.bats):

ok 1 gemini env: GEMINI_DEEP_MODEL override sets ENGINE_DEEP_MODEL
ok 2 gemini env: GEMINI_TRIAGE_MODEL override sets ENGINE_TRIAGE_MODEL
ok 3 gemini env: GEMINI_ACTION_MODEL override sets ENGINE_ACTION_MODEL
ok 4 gemini env: GEMINI_AUDIT_MODEL override sets ENGINE_AUDIT_MODEL
ok 5 gemini env: GEMINI_SINGLE_MODEL override sets ENGINE_SINGLE_MODEL
ok 6 gemini env: defaults unchanged when no overrides set
ok 7 gemini chain: first model succeeds → no fallback attempted
ok 8 gemini chain: first model rate-limited → falls through to second
ok 9 gemini chain: all models rate-limited → returns exit 2
ok 10 gemini chain: non-rate-limit failure → propagates immediately without trying next
ok 11 gemini chain: whitespace and empty entries are tolerated
ok 12 gemini chain: empty/whitespace-only chain → config error (rc=1), not rate-limited (rc=2)
ok 13 gemini writer: flash rate-limited → pro tried via GEMINI_ACTION_MODEL_CHAIN
ok 14 gemini writer: both models rate-limited → exit 2 (cross-provider fallback signal)
ok 15 gemini writer: no chain set → single-model behavior unchanged
ok 16 gemini agentic: deep tier flash rate-limited → pro tried (GEMINI_DEEP_MODEL_CHAIN)
ok 17 gemini agentic: explicit model arg differing from tier default is honored (no chain expansion)
ok 18 gemini triage: flash rate-limited → pro tried via GEMINI_TRIAGE_MODEL_CHAIN
ok 19 claude engine: GEMINI_*_MODEL_CHAIN does not affect claude invocations

Full bats suite (tests/dev-lead/unit/ + top-level tests/*.bats):

Total OK: 303
Total not-ok: 0

Lint:

shellcheck --severity=warning -x scripts/**/*.sh   → rc=0
markdownlint-cli2 docs/engine-model-rollout.md     → 0 error(s)

Concurrency integration test: PASS.

Pre-existing failures unrelated to this change: tests/test-validate-engines.sh reports 2 failures on this branch before my changes (verified by git stash). Not in scope for #382.

Files Changed

scripts/engine.sh: gemini per-tier model overrides + _gemini_chain_invoke + chain wiring in triage/agentic/writer + chain-aware token recording; clear GEMINI_*_MODEL_CHAIN in non-gemini branches to prevent bleed
tests/dev-lead/fixtures/engines/stub-gemini: per-model exit/response/recording (parity with stub-claude)
tests/dev-lead/unit/test_engine_chain.bats: updated one assertion (! grep -q "^claude-" instead of empty-file check) since stub-gemini now records too
tests/dev-lead/unit/test_engine_gemini_chain.bats: 19 new tests covering env overrides, chain control flow, writer/agentic/triage wiring, and claude-isolation guard
docs/engine-model-rollout.md: rollout doc with phase-1/2/3 env var examples and fail-closed contract

Notes

The implementation mirrors the existing Claude in-engine chain (issue feat(dev-lead): proactive provider headroom check before engine invocation #206) for symmetry. Both helpers share the same edge-case contract: empty chain → rc=1 (config error), all-rate-limited → rc=2 (cross-provider fallback signal), first non-rate-limit failure → propagate immediately.
run_duck (cross-engine adversarial review) is intentionally not chain-wired — the duck is a one-shot adversarial pass and shouldn't itself stage rollout decisions across model variants.
The _gemini_chain_invoke helper duplicates _claude_chain_invoke rather than factoring a shared helper. Refactoring is out of scope for Enhance dev-lead and pr-review engines for Gemini 3.5 models #382; the two helpers can be consolidated in a follow-up.
gemini-3.5-flash is not hard-coded anywhere — the implementation is model-ID-agnostic, so any future Gemini model ID works via the same env vars without code changes (the "no hard-coded assumptions, rely on config" guidance in the issue's research notes).

Merge origin/main into dev-lead/issue-382-20260524-0133

feat: implement issue #382 — Enhance dev-lead and pr-review engines f…

Already working on this: an open PR exists for issue #382.

feat: implement issue #382 — Enhance dev-lead and pr-review engines f…

@dev-lead

@dev-lead the prior implementation (PR #383) has been closed — it went stale and its rebase regressed functionality that has since landed in main. Please re-implement this issue fresh from current main.

Critical constraints (these caused the regressions in #383):

Branch from the latest main and verify your changes are additive — do not remove or rewrite existing behavior in scripts/engine.sh, scripts/review-one-pr.sh, or .github/workflows/pr-review.yml.
Specifically preserve, do not regress: workflow_call for reusable callers; the PR-diff prefetch before triage; event-driven PR-review triggers; check-run dedup and the self-referential CI filter (lib/ci-status.sh, pr-review: mention-triggered runs silently lost to ci-pending skip (self-referential check runs) #469); the advisory-bot review gate (feat: implement advisory bot review gate in pr-review agent #458); the cycle-cap logic (lib/review-cycle.sh, pr-review: cycle cap counts approvals, ignores human re-requests, and escalation is unrecoverable #467); and the worktree collision-prevention in lib/pr-worktree.sh (fix(dev-lead): release stale PR-branch worktrees before checkout (#470) #494).
Do not introduce the "council" parallel-review block that referenced undefined functions — if a multi-lens review is in scope, wire it through existing helpers and ensure every code path is defined and shellcheck-clean.
Keep the change focused on Gemini-3.5 capability/config-aware routing + observability per the issue; run shellcheck --severity=warning -x and the bats suites before opening the PR; open a NEW PR off a fresh branch.

feat: implement issue #382 — Enhance dev-lead and pr-review engines f…

Navigation Menu

Repository navigation

Enhance dev-lead and pr-review engines for Gemini 3.5 models #382

Summary

Why

Research notes (official docs)

Proposed approach

Deliverables

Acceptance criteria

Risks / open questions

References

Dev-Lead Implementation Plan

Scope

Implementation Checklist

Tests to Write

Test Command

Dev-Lead: Implementation Complete

Plan Execution

Test Results

Files Changed

Notes

10 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Enhance dev-lead and pr-review engines for Gemini 3.5 models #382

Description

Summary

Why

Research notes (official docs)

Proposed approach

Deliverables

Acceptance criteria

Risks / open questions

References

Activity

don-petry commented on May 24, 2026

Dev-Lead Implementation Plan

Scope

Implementation Checklist

Tests to Write

Test Command

don-petry commented on May 24, 2026

Dev-Lead: Implementation Complete

Plan Execution

Test Results

Files Changed

Notes

don-petry commented on Jun 8, 2026

don-petry commented on Jun 8, 2026

10 remaining items

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions