You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(headless): record OpenCode prompt provenance accurately #1563
packages/headless/harbor/opencode_agent.py::_execution_identity() hashes MAKA_SYSTEM_PROMPT and records it as systemPromptHash.
The OpenCode adapter does not inject that prompt into OpenCode. _run_with_stop_sentinel() launches the pinned OpenCode CLI with the task instruction and OpenCode's benchmark config; OpenCode supplies its own native system scaffold. As a result, the recorded identity claims provenance for model-visible bytes that the OpenCode arm never received.
Changing only the Maka system prompt therefore changes the OpenCode cell's execution identity even though the OpenCode invocation is unchanged. This can create false cohort differences and misleading audit evidence. It does not change the OpenCode score or token usage itself.
Expected behavior:
Execution identity distinguishes a Maka-injected prompt from a competitor-owned native prompt.
An OpenCode cell does not record the Maka prompt hash as model-visible provenance.
Changing only MAKA_SYSTEM_PROMPT does not change OpenCode's prompt identity.
The pinned OpenCode version/toolchain remains the authority for the competitor-owned scaffold when its exact internal prompt hash is unavailable.
How to reproduce
Run two otherwise identical OpenCode harness cells.
Set MAKA_SYSTEM_PROMPT to different values in the two environments.
Compare maka-cell-execution-identity.json.
Observe different systemPromptHash values.
Compare the OpenCode launch command/config and provider-visible task input; the Maka prompt is not injected into either invocation.
Environment
Maka: reproduced on main@213882bda and present in ee7e4ba5d
Surface: Headless / Harbor OpenCode adapter
OpenCode: pinned 1.17.18
Node.js: 26.5.0
Harbor: 0.13.2
Logs, screenshots, or additional context
This is a provenance defect, not an effectiveness or economy defect. Existing benchmark scores remain valid, but reports should not describe the recorded OpenCode systemPromptHash as the hash of OpenCode's actual native system prompt.
The fix should define the prompt-authority representation at the execution-identity contract rather than inventing a hash for inaccessible competitor-owned prompt bytes, and should include backward-compatible decoding for existing cell artifacts.
Triage note: this defect no longer exists on main. The Headless/Harbor OpenCode adapter that hashed MAKA_SYSTEM_PROMPT into the OpenCode cell's execution identity (packages/headless/harbor/opencode_agent.py::_execution_identity) was removed wholesale in #2605 when Headless was replaced by the minimal Eval kernel. In the current Eval kernel, external subjects' identity is the verified toolchain version + fingerprint (packages/eval/src/external-subject.ts, toolchain-verification.ts); the Maka system prompt is not recorded as provenance for a competitor-owned scaffold. Backward-compatible decoding of the old maka-cell-execution-identity.json artifacts is moot for the same reason — the producer is gone. This issue can likely be closed as obsolete.
Activity
UncertaintyDeterminesYou4ndMe commentedon Aug 15, 2026
Triage note: this defect no longer exists on
main. The Headless/Harbor OpenCode adapter that hashedMAKA_SYSTEM_PROMPTinto the OpenCode cell's execution identity (packages/headless/harbor/opencode_agent.py::_execution_identity) was removed wholesale in #2605 when Headless was replaced by the minimal Eval kernel. In the current Eval kernel, external subjects' identity is the verified toolchainversion+fingerprint(packages/eval/src/external-subject.ts,toolchain-verification.ts); the Maka system prompt is not recorded as provenance for a competitor-owned scaffold. Backward-compatible decoding of the oldmaka-cell-execution-identity.jsonartifacts is moot for the same reason — the producer is gone. This issue can likely be closed as obsolete.Astro-Han commentedon Aug 15, 2026
Closed as done. Thanks for triaging the issue @UncertaintyDeterminesYou4ndMe!