feat(eval): let harbor runs use a fusion model, and fix the dead forward_keys opt-out

[ericleepi314]

A fusion model — a text-only base plus a borrowed vision model — lives in the user's global config and is selected by name. It is not addressable as provider/model, so a fresh eval container could not resolve one no matter what --model said, and vision tasks were unrunnable on a text-only base.

--model deepseek-v4-flash-luna \
--ak fusion=deepseek:deepseek-v4-flash+openai:gpt-5.6-luna

The record is seeded into the container's global config, named after --model so the two cannot drift: that name is the selector clawcodex looks up.

Two things beyond the record itself

default_provider is seeded from the fusion's BASE provider. Provider resolution runs before the fusion name is looked up, so with no default configured clawcodex fell back to anthropic and died on its missing key without ever reaching the record — the run looked like a credentials problem rather than a routing one. Seeding the base is also semantically right: the base model runs the loop.

Key forwarding already works, but by an accident worth recording. A fusion name carries no provider/ prefix, so _parsed_model_provider is empty and the env allowlist falls back to all providers — required here and only here, because a fusion run legitimately needs two vendors' keys at once.

The CLI_FLAGS distinction is load-bearing in both directions

fusion and forward_keys are explicit constructor kwargs, not CLI_FLAGS entries:

• _resolve_flag_values only walks CLI_FLAGS, so an undeclared --ak key is accepted and silently discarded. That is why the first run of this feature was a no-op — and why --ak forward_keys=false has been doing nothing since it was documented: it was read from _resolved_flags while declared nowhere, so keys were forwarded regardless of the opt-out. Fixed here.
• Declaring them in CLI_FLAGS is not the fix either: build_cli_flags() emits every entry, so clawcodex would be handed a --fusion flag it rejects.

Verified end to end

terminal-bench 2.1 code-from-image — transcribe handwritten pseudocode from a PNG and reproduce its output:

• reward 1.0 with deepseek-v4-flash + openai:gpt-5.6-luna, matching the opus-5 baseline on that task.
• Attributable to the fusion path, not the base coping: deepseek-v4-flash alone returns 400 unknown variant image_url on the same image.
• The trajectory is Read /app/code.png → Bash python3 …SALT = b'0000TBENCH-SALT'… → Write output.txt, with the salt transcribed verbatim and no OCR tool involved.
• The vision leg is first-party OpenAI on an API key (subscription_active: False, api.openai.com/v1/responses) — not OpenRouter, not the ChatGPT plan.

Tests are SKIPPED IN CI

clawcodex_agent.py imports harbor at module scope, and harbor is not a dev dependency — it is a separate uv tool installed by whoever runs evals. So these assertions run for them and nowhere else, which is worth knowing before reading a green CI as coverage of this file. The adapter had no tests at all before this. Verified passing under harbor's environment (12 tests), and mutation-tested against the two defects actually hit — the silent --ak drop and the record name drifting from --model — plus the shape-validation and forward_keys paths.

Full suite: 9587 passed, 0 failed (the 4th skip is this file).

🤖 Generated with Claude Code

[github-actions]

Test Results

    1 files      1 suites   8m 10s ⏱️
9 591 tests 9 584 ✅ 7 💤 0 ❌
9 891 runs  9 884 ✅ 7 💤 0 ❌

Results for commit 7cfcab4.