Deep Code runtime contract blocks exact Brev Launchable image publication #8665

Contributor preflight (required)

Before investigating or changing code:

1. Read WRITING.md in full.
2. Read CONTRIBUTING.md in full.
3. Run npm run dev:doctor from a clean checkout before creating a branch.
4. Follow both guides for code, tests, issue updates, commit messages, DCO, commit verification, documentation review, and the pull request.

Problem

The Exact staging Brev Launchable lane cannot publish an exact-candidate staging image because the cross-repository image producer fails while building the final LangChain Deep Code sandbox image.

The producer successfully validates the request and source identity and builds the NemoClaw, Hermes, and Deep Code base layers. It then aborts in the final Deep Code Dockerfile before image publication, Launchable deployment, workspace provisioning, or full E2E execution.

This is a NemoClaw-managed Deep Code image-contract failure exposed by the brevdev/nemoclaw-image producer. It is not evidence of a Brev workspace provisioning outage.

Confirmed evidence

The failure is deterministic across two different exact NemoClaw candidates:

• NemoClaw run 31337932239, candidate 1a8e6bd3198ec54c7682f98b23141bdaac716160
  • Exact staging Brev Launchable job
  • Producer run 31340168136
  • Failing producer build job
• NemoClaw run 31340979460, candidate 794c75494bcc425b7da265bd800b45b8715e92e8
  • Exact staging Brev Launchable job
  • Producer run 31342418072
  • Failing producer build job

Both producers ran from brevdev/nemoclaw-image commit 7d7f3a649ef1bc52c3dd98bd29b0b881fffd462c and failed at the same final-image Docker step in agents/langchain-deepagents-code/Dockerfile:101-110:

[stage-3 5/43] RUN discovery_contract="$(node /usr/local/lib/nemoclaw/mcp-tool-discovery-runtime/mcp-tool-discovery.mjs)" ...
ERROR: ... did not complete successfully: exit code: 1
[provision] Deep Code sandbox image pre-build failed; refusing to publish an image with a broken terminal agent

That chained command validates all of the following:

• the bundled MCP tool-discovery runtime returns the exact invalid-argument contract;
• the discovery runtime tree is root-owned and not group- or world-writable;
• the managed-startup runtime is a regular, non-symlink file;
• the managed-startup runtime can be normalized to root:root mode 0444;
• /run/nemoclaw can be created with the required ownership and mode.

The command exits without identifying which conjunct failed. The Docker warnings about the default base-image argument and the Deep Code API-key environment variable are not the failing condition.

Historical boundary — hypothesis, not confirmed root cause

PR #8642, merged as 5429cecf493f4bbe6add0f6829f851945ffee778, added the final-image regular-file, non-link, ownership, and 0444 assertions to this command. The repeated producer failure occurs after that change, but the current shell chain does not prove which assertion fails.

Reproduce the exact layer against the same base digest and inspect each check independently before choosing a fix. Do not remove or weaken the fail-closed permission contract to make the image build pass.

Implementation surface

• agents/langchain-deepagents-code/Dockerfile
• tools/mcp-tool-discovery-runtime/mcp-tool-discovery.ts
• tools/mcp-tool-discovery-runtime/reviewed-runtime-bundle/mcp-tool-discovery/mcp-tool-discovery.bundle
• tools/mcp-tool-discovery-runtime/reviewed-runtime-bundle/managed-startup-image-runtime.bundle
• src/lib/actions/sandbox/mcp-tool-discovery-runtime.test.ts
• src/lib/onboard/managed-startup-image-runtime.test.ts
• src/lib/onboard/managed-startup-bootstrap-image-runtime.test.ts
• test/langchain-deepagents-code-image.test.ts
• test/langchain-deepagents-code-image-credentials.test.ts
• test/managed-startup-image-entrypoint.test.ts

If the producer also needs better failure receipts or exact-environment regression coverage, make the companion change in brevdev/nemoclaw-image and link it here. Keep the canonical runtime and Docker contract fix in NemoClaw.

Acceptance criteria

• The exact failing conjunct is identified with evidence from the producer-equivalent build environment.
• The final Deep Code image builds successfully against the exact QA base used by the Launchable producer.
• The MCP discovery invalid-argument response remains exactly versioned and fail-closed.
• The discovery runtime tree remains root-owned and not group- or world-writable.
• The managed-startup runtime remains a regular, non-link, root:root, mode-0444 file in the final image.
• The fix does not remove, bypass, or relax the security assertions restored by fix(images): preserve managed runtime bundle permissions #8642.
• NemoClaw regression coverage executes the final Deep Code image contract with producer-equivalent base and filesystem semantics, rather than relying only on Dockerfile text inspection.
• A future contract failure identifies the failed sub-check in the producer log and handoff receipt.
• The producer continues refusing to publish an image when any required image contract fails.
• The producer publishes a READY handoff for the exact NemoClaw SHA, correlation ID, and image digest after the fix.
• Exact staging Brev Launchable consumes that exact image, verifies its receipts and baked checkout, completes full E2E, and cleans up successfully.
• OpenClaw, Hermes, and other managed-image contract checks remain green.

Targeted verification

npx vitest run --project cli \
  src/lib/actions/sandbox/mcp-tool-discovery-runtime.test.ts \
  src/lib/onboard/managed-startup-image-runtime.test.ts \
  src/lib/onboard/managed-startup-bootstrap-image-runtime.test.ts

npx vitest run --project integration \
  test/langchain-deepagents-code-image.test.ts \
  test/langchain-deepagents-code-image-credentials.test.ts \
  test/managed-startup-image-entrypoint.test.ts

Then run the producer-equivalent Deep Code Docker build with plain progress output and the Exact staging Brev Launchable workflow for the same commit. Complete the required documentation-writer review and record whether user-facing documentation changes are needed.

Related work

• NemoClaw PR #8642 restored the final managed-runtime ownership and mode assertions.
• brevdev/nemoclaw-image issue #75 tracks a different post-provisioning Deep Code keepalive and resource-limit acceptance mismatch. It does not cover this pre-publication build failure.