Skip to content

Multi-Agent V2 sends OpenAI-specific agent_message items to external Responses providers #33551

Description

@sdwolf4103

What issue are you seeing?

Codex Multi-Agent V2 sends child-agent instructions as the OpenAI-specific Responses item type agent_message. External Responses-compatible providers such as Ollama do not recognize this item type and cannot decrypt its encrypted_content when present.

This prevents a V2 parent from spawning a native Codex agent whose configured model_provider is external. The failure is at the provider request boundary, before the external model can correctly receive the delegated task.

A captured comparison showed:

  • Working V1 Kimi/GLM requests: every input item had type: "message", with no encrypted_content.
  • Failing V2 GLM request: the final input item had type: "agent_message". Even when that captured request had no encrypted payload, Ollama Responses still could not process the non-standard item type.

External-provider routing itself works under V1. The important distinction is that the Multi-Agent version is selected and stored by the root task's first parent turn, then inherited by the whole spawned agent tree:

  • If a fresh task sends its first message using a V1-capable parent model such as Luna, the root task resolves to V1 and every subsequently spawned agent also uses V1.
  • Switching models after the first message does not change the already stored Multi-Agent version.
  • Starting the task with a catalog-forced V2 parent such as Sol or Terra therefore keeps the whole tree on V2, including external-provider children.

This is related to, but distinct from, #33284. That issue concerns plaintext visibility to local hooks before execution. This issue concerns wire-format compatibility with non-OpenAI model providers after dispatch.

What steps can reproduce the bug?

  1. Configure an external Responses provider, for example an Ollama Responses endpoint, in Codex configuration.
  2. Configure a native agent role in ~/.codex/agents/*.toml whose model_provider points to that external provider.
  3. Enable Multi-Agent and Multi-Agent V2.
  4. Start a fresh task using a parent model for which the model catalog selects V2, and send the first message with that model.
  5. Invoke spawn_agent for the external-provider role.
  6. Inspect the request sent to the external Responses endpoint.

The child instruction is serialized as an input item resembling:

{
  "type": "agent_message",
  "encrypted_content": "..."
}

or as a plaintext agent_message item. Ollama cannot decrypt the encrypted form, and its Responses implementation does not accept the agent_message item type in either form.

Disabling the multi_agent_v2 feature flag alone may not avoid the issue when the model catalog forces V2 for the selected parent model.

The current workaround requires all of the following:

  1. Create a new task.
  2. Select a V1-capable parent model such as Luna before sending the first message.
  3. Set multi_agent = true and multi_agent_v2 = false.
  4. Keep using that task after its first turn has resolved to V1.

This makes the entire task and every child agent use V1. It is not a provider-specific fallback, and it does not allow an OpenAI parent to retain V2 while only an external-provider child uses a compatible transport.

Relevant implementation areas appear to include:

  • codex-rs/core/src/tools/handlers/multi_agents_spec.rs
  • codex-rs/core/src/tools/handlers/multi_agents_v2.rs
  • codex-rs/protocol/src/protocol.rs (InterAgentCommunication::to_model_input_item)
  • codex-rs/core/src/client.rs (Responses request construction)

What is the expected behavior?

Multi-Agent V2 should remain usable when either endpoint in an inter-agent route uses a non-OpenAI provider.

The requested behavior is not to downgrade external agents to V1 inside an otherwise V2 task. The root task and the entire agent tree should retain V2 lifecycle and coordination semantics. Only the wire representation at a non-OpenAI provider boundary should be adapted.

A provider-aware compatibility behavior would be:

  • Preserve the existing encrypted agent_message path for OpenAI-to-OpenAI routes.
  • For a route involving a non-OpenAI provider, keep the task text plaintext and serialize the outbound instruction as a standard Responses message item, for example:
{
  "type": "message",
  "role": "user",
  "content": [
    {
      "type": "input_text",
      "text": "..."
    }
  ]
}
  • Apply the same provider-aware conversion to initial spawn instructions, follow-up/send_message instructions, and child return messages as needed.
  • Preserve V2 lifecycle behavior such as task identity, graph state, wait, follow-up, and completion.
  • Leave existing OpenAI-provider behavior unchanged.
  • Allow catalog-forced V2 parents such as Sol or Terra to spawn explicitly configured external-provider roles without requiring the root task to start under Luna/V1.

Additional information

The compatibility decision should be made using the actual caller and recipient provider identities, not only the parent model name.

The current Luna-first workaround demonstrates that the configured external provider and native agent role can function when the payload uses standard V1 message items. It also demonstrates why that workaround is insufficient: it disables V2 for the entire root/child tree rather than making V2 interoperable with external Responses providers.

Activity

  1. added
    bugSomething isn't working
    CLIIssues related to the Codex CLI
    custom-modelIssues related to custom model providers (including local models)
    subagentIssues involving subagents or multi-agent features
    on Jul 16, 2026
  2. sfourdrinier commented on Jul 19, 2026

    @sfourdrinier

    I've also hit the same problem.

  3. jingx8885 commented on Jul 22, 2026

    @jingx8885

    I traced this to two separate OpenAI-specific assumptions in Multi-Agent V2 and have a tested implementation available for maintainer review.

    Root cause

    1. The V2 tool schemas mark inter-agent message arguments with the Responses encrypted extension. A portable Responses provider cannot produce or consume that encrypted field.
    2. Even when the communication is plaintext internally, Codex serializes it as the non-standard agent_message input item. External Responses-compatible providers generally accept only standard message items.

    The second point is why merely disabling client-side encryption is insufficient.

    Proposed compatibility mode

    The branch adds an opt-in setting:

    [features.multi_agent_v2]
    encrypt_inter_agent_messages = false

    When omitted, it defaults to true, so the existing OpenAI/Codex path is unchanged.

    When disabled:

    • spawn_agent, send_message, and followup_task expose ordinary string arguments without the encrypted schema extension;
    • the default namespace changes from the reserved collaboration namespace to agents unless explicitly configured;
    • inter-agent communications remain plaintext internally;
    • at a non-OpenAI provider request boundary, plaintext agent_message items are converted to portable Responses message/user/input_text items;
    • encrypted items are never rewritten, and OpenAI-provider requests retain the existing representation.

    An explicit mode is needed because the parent model must see the tool argument schema before it chooses an agent_type; the recipient provider is not yet known when that schema is generated.

    Implementation: https://github.com/jingx8885/codex/tree/codex/portable-inter-agent-messages
    Commit: jingx8885@20f3bcc433

    Validation

    • Targeted end-to-end integration test passes with a genuinely separate external worker provider configuration. It asserts:
      • the parent tool schema has no encrypted marker;
      • the worker receives a standard message item;
      • no agent_message item reaches the external provider.
    • codex-features: 29/29 tests passed.
    • codex-core: 2,978/3,035 tests passed. Of the remaining 57, 56 were test-environment failures caused by helper binaries not having been built at the time, and one unrelated image-budget test timed out. The helper binaries were subsequently built successfully.
    • just fix -p codex-core and just fmt completed successfully.
    • A subsequent redundant codex-features Clippy run hit a transient missing Cargo fingerprint directory while rebuilding dependencies.

    The implementation was AI-assisted and independently reviewed against the final diff and targeted behavior.

    Would a Codex maintainer confirm whether this opt-in compatibility approach matches the intended architecture and, if so, invite a pull request? I have not opened one because docs/contributing.md states that uninvited external PRs are closed without review.

  4. ysxk commented on Jul 23, 2026

    @ysxk

    Related desktop corroboration (external Responses 422)

    Alongside the encrypted-assignment / Chat Completions case in #34833, we also hit the external Responses item-type failure on MultiAgentV2:

    • Parent: OpenAI gpt-5.6-sol on Codex desktop (CLI 0.145.0-alpha.30)
    • Child role: registered skills_openapi with grok-4.5
    • Child dies immediately with:
    unexpected status 422 Unprocessable Entity:
    {"error":"Failed to deserialize the JSON body into the target type: data did not match any variant of untagged enum ModelInput"}
    url: http://<external-host>:8317/v1/responses
    
    • Empty history / 0 tools on the child before the error.
    • Same parent successfully spawns OpenAI children in the same turn.
    • Temporarily re-pointing that role at gpt-5.6-sol unblocked the work; restoring grok-4.5 reintroduces the failure.

    This matches the “V2 sends agent_message / encrypted OpenAI-specific items to external Responses providers” diagnosis here. Provider-aware conversion to a standard message item at non-OpenAI boundaries (while keeping V2 lifecycle) would cover this path.

  5. nicezic commented on Jul 24, 2026

    @nicezic

    Confirmed on Codex 0.144.6 / WSL2 with an OpenAI gpt-5.6-sol parent and a MiniMax-M3 child using an external Responses provider. Direct codex exec against MiniMax succeeds. Native V2 spawn and follow-up authenticate and complete, but return an empty result. The child rollout contains an empty visible Payload: followed only by encrypted_content. This reproduces the provider-boundary issue with MiniMax’s Responses-compatible endpoint.

  6. CCanxue commented on Aug 3, 2026

    @CCanxue

    Confirmed on codex-cli 0.146.0 with DeepSeek (wire_api = "responses", multi_agent_version = "v2"), with wire captures:

    • The V2 task is delivered as an OpenAI-specific agent_message item whose payload lives in encrypted_content; DeepSeek drops that block, so the child sees only an empty Payload: header.
    • Even when the agent_message content is rewritten to plaintext input_text, DeepSeek still ignores the item in most runs and the child replies that it has no task.
    • The same task delivered as a plain user message is consumed reliably (spawn -> PONG, followup_task -> PONG2).

    This suggests upstream should render agent-message deliveries as plain user messages (or V1-style UserInput) for non-OpenAI providers, not just fold encrypted_content into the agent_message item. Our local patch does exactly that: https://github.com/CCanxue/codex-deepseek-subagent-fix

  7. boogie-ben commented on Aug 5, 2026

    @boogie-ben

    Working local workaround: proxy that rewrites agent_message → message for non-OpenAI parents

    Environment: macOS 15.2 (arm64), Codex Desktop 26.730.61639, bundled codex-cli 0.147.0-alpha.1.2, custom DeepSeek provider (wire_api = "responses", multi_agent_version = "v2", model deepseek-v4-flash).

    Fully tested locally: subagents spawn, receive their task, execute it, and return results. Normal turns and SSE pass through unchanged.

    Why it works: for a non-OpenAI parent, encrypted_content is plaintext — Codex stores the raw spawn_agent message there (InterAgentCommunication::new_encrypted() in protocol.rs) and never rewrites agent_message items for non-OpenAI providers (client.rs). DeepSeek ignores agent_message items, so the task never reaches the child. The proxy converts them to standard message/user items. Real Fernet ciphertext (OpenAI parents) always starts with gAAAAA; the proxy passes those through untouched.

    Setup:

    node proxy.mjs   # listens on 127.0.0.1:8787 -> https://api.deepseek.com
    [model_providers.deepseek]
    base_url = "http://127.0.0.1:8787"

    Restart the session. The proxy is a single zero-dependency Node script — run it manually, wrap it as a Codex plugin, or manage it as an auto-start background process (e.g. launchd/systemd).

    Two cases matched by agent_message: ① parent → child NEW_TASK in the child's requests (critical, must be converted); ② child → parent FINAL_ANSWER replayed in the parent's history on every later request (converted too, harmless).

    Limitation: works for any non-OpenAI parent → non-OpenAI child provider combination (DeepSeek, MiniMax, Kimi, etc.); OpenAI-parent ciphertext (#36376) cannot be decrypted locally.

    Related:

    Proxy script (Node.js, zero dependencies):

    // deepseek-codex-proxy
    //
    // Local transparent proxy that fixes Codex MultiAgentV2 subagent task delivery
    // for non-OpenAI providers (DeepSeek).
    //
    // Why: Codex wraps subagent tasks in an OpenAI-specific input item type
    // `agent_message`, with the task text inside an `encrypted_content` block.
    // DeepSeek's Responses API only supports `message` / `function_call` /
    // `function_call_output` / `reasoning` / `web_search_call` input items, so it
    // drops the `agent_message` item and the subagent only sees an empty
    // "Payload:" envelope.
    //
    // Key insight: when the parent model is non-OpenAI (DeepSeek), the value inside
    // `encrypted_content` is plaintext -- `InterAgentCommunication::new_encrypted()`
    // stores the raw string from the spawn_agent tool call. Only OpenAI parents
    // produce real Fernet ciphertext (always prefixed `gAAAAA`), which cannot be
    // decrypted locally and is passed through untouched.
    //
    // Configuration: edit the constants below, then point the provider's base_url
    // at the proxy (see the issue comment for the config.toml change).
    
    import http from "node:http";
    import https from "node:https";
    
    // ── Configuration ──────────────────────────────────────────────────────
    const PORT = 8787; // local listen port
    const UPSTREAM_BASE = "https://api.deepseek.com"; // real upstream endpoint
    const PATH_STRIP = ""; // set to "/v1" if your base_url includes /v1
    // ───────────────────────────────────────────────────────────────────────
    const MAX_BODY = 64 * 1024 * 1024; // request body cap (64 MiB)
    const UPSTREAM_TIMEOUT_MS = 5 * 60 * 1000; // upstream idle timeout (5 min)
    
    const NORMALIZED_UPSTREAM = UPSTREAM_BASE.replace(/\/+$/, "");
    
    const upstream = new URL(NORMALIZED_UPSTREAM);
    const isUpstreamTls = upstream.protocol === "https:";
    const transport = isUpstreamTls ? https : http;
    
    function log(...args) {
      console.log(new Date().toISOString(), ...args);
    }
    
    // Returns true when the encrypted_content value is real ciphertext (produced by
    // an OpenAI parent, not decryptable locally). Fernet tokens always start with
    // "gAAAAA"; anything else is treated as plaintext so that legitimate long
    // space-free tasks are never misclassified.
    function looksLikeCiphertext(payload) {
      return typeof payload === "string" && payload.startsWith("gAAAAA");
    }
    
    // Convert an agent_message item into a standard user message.
    // Returns null when the payload is real ciphertext (leave the item untouched).
    function buildUserMessage(item) {
      const blocks = Array.isArray(item.content) ? item.content : [];
      const headerParts = [];
      let payload = "";
      for (const block of blocks) {
        if (block?.type === "input_text" && typeof block.text === "string") {
          headerParts.push(block.text);
        } else if (
          block?.type === "encrypted_content" &&
          typeof block.encrypted_content === "string"
        ) {
          payload = block.encrypted_content;
        }
      }
      const header = headerParts.join("").replace(/\s+$/, "");
      if (looksLikeCiphertext(payload)) return null;
      if (!header && !payload) return null; // nothing extractable, leave untouched
      const text = [header, payload].filter(Boolean).join("\n");
      const message = {
        type: "message",
        role: "user",
        content: [{ type: "input_text", text }],
      };
      if (typeof item.id === "string") message.id = item.id; // keep the original id
      return message;
    }
    
    // Rewrite a POST /responses request body.
    // Returns { body, changed, skipped }; changed = items rewritten, skipped = ciphertext items.
    function transformResponsesBody(raw) {
      let parsed;
      try {
        parsed = JSON.parse(raw.toString("utf8"));
      } catch {
        return { body: raw, changed: 0, skipped: 0 };
      }
      const input = parsed?.input;
      if (!Array.isArray(input)) return { body: raw, changed: 0, skipped: 0 };
    
      const newInput = [];
      let changed = 0;
      let skipped = 0;
      for (const item of input) {
        if (item && item.type === "agent_message") {
          const replacement = buildUserMessage(item);
          if (replacement) {
            newInput.push(replacement);
            changed += 1;
          } else {
            newInput.push(item);
            skipped += 1;
          }
        } else {
          newInput.push(item);
        }
      }
      if (!changed) return { body: raw, changed, skipped };
      parsed.input = newInput;
      return { body: Buffer.from(JSON.stringify(parsed)), changed, skipped };
    }
    
    const server = http.createServer((req, res) => {
      try {
        let path = req.url ?? "/";
        if (PATH_STRIP && path.startsWith(PATH_STRIP)) {
          path = path.slice(PATH_STRIP.length) || "/";
        }
        const pathname = path.split("?")[0];
        const isResponses = req.method === "POST" && /\/responses$/.test(pathname);
    
        const baseHeaders = { ...req.headers };
        delete baseHeaders.host;
        delete baseHeaders.connection;
    
        const requestOptions = (headers) => ({
          protocol: upstream.protocol,
          hostname: upstream.hostname,
          port: upstream.port || (isUpstreamTls ? 443 : 80),
          method: req.method,
          path,
          headers,
        });
    
        const sendUpstream = (headers, body) => {
          const uReq = transport.request(requestOptions(headers), (uRes) => {
            const respHeaders = { ...uRes.headers };
            delete respHeaders.connection;
            delete respHeaders["transfer-encoding"]; // Node adds chunked as needed
            res.writeHead(uRes.statusCode ?? 502, respHeaders);
            uRes.pipe(res);
          });
          uReq.on("error", (err) => {
            log("upstream error:", err.message);
            if (!res.headersSent) {
              res.writeHead(502, { "content-type": "application/json" });
              res.end(JSON.stringify({ error: "upstream_error", detail: err.message }));
            } else {
              res.destroy();
            }
          });
          uReq.setTimeout(UPSTREAM_TIMEOUT_MS, () =>
            uReq.destroy(new Error("upstream timeout"))
          );
          if (body) uReq.write(body);
          return uReq;
        };
    
        if (isResponses) {
          const chunks = [];
          let size = 0;
          req.on("data", (c) => {
            size += c.length;
            if (size > MAX_BODY) {
              res.writeHead(413, { "content-type": "application/json" });
              res.end(JSON.stringify({ error: "payload_too_large" }));
              req.destroy();
              return;
            }
            chunks.push(c);
          });
          req.on("end", () => {
            try {
              const raw = Buffer.concat(chunks);
              const { body, changed, skipped } = transformResponsesBody(raw);
              if (changed || skipped) {
                log(
                  `transform ${req.method} ${path} agent_message->message changed=${changed} skipped_ciphertext=${skipped}`
                );
              } else {
                log(`pass ${req.method} ${path} (no agent_message)`);
              }
              const headers = { ...baseHeaders };
              delete headers["content-length"];
              headers["content-length"] = String(body.length);
              const uReq = sendUpstream(headers, body);
              uReq.end(); // end explicitly instead of relying on implicit behavior
            } catch (err) {
              log("responses handler error:", err.message);
              if (!res.headersSent) {
                res.writeHead(500, { "content-type": "application/json" });
                res.end(JSON.stringify({ error: "internal_error", detail: err.message }));
              } else {
                res.destroy();
              }
            }
          });
          req.on("error", () => res.destroy());
        } else {
          log(`pass ${req.method} ${path}`);
          const uReq = sendUpstream(baseHeaders, null);
          req.pipe(uReq);
          req.on("error", () => uReq.destroy());
        }
      } catch (err) {
        log("handler error:", err.message);
        if (!res.headersSent) {
          res.writeHead(500, { "content-type": "application/json" });
          res.end(JSON.stringify({ error: "internal_error", detail: err.message }));
        } else {
          res.destroy();
        }
      }
    });
    
    server.listen(PORT, "127.0.0.1", () => {
      log(`deepseek-codex-proxy listening on http://127.0.0.1:${PORT} -> ${UPSTREAM_BASE}`);
    });
  8. CCanxue commented on Aug 6, 2026

    @CCanxue

    Thanks for sharing. The gAAAAA Fernet heuristic matches our finding: for non-OpenAI parents the value stored in encrypted_content is locally plaintext, while OpenAI-parent assignments are real ciphertext that cannot be decrypted locally.

    We went with a core patch instead of a wire proxy: for non-OpenAI providers, V2 spawn / followup_task / send_message are delivered as plain UserInput (standard user messages), so no request rewriting is needed for the non-OpenAI parent topology. Verified end-to-end with deepseek-v4-flash (spawn -> PONG, followup -> PONG2). The proxy is a nice zero-dependency alternative for users who cannot rebuild the CLI. Patch and evidence: https://github.com/CCanxue/codex-deepseek-subagent-fix

  9. btmorgeson commented on Aug 13, 2026

    @btmorgeson

    Sanitized corroboration from Windows with codex-cli 0.147.0 and a custom OpenAI-compatible Responses provider:

    • The root session recorded multi_agent_version: "v2" even though features.multi_agent_v2 = false; the selected model catalog/session assignment controlled the active transport.
    • spawn_agent created the child thread successfully.
    • The child task arrived as an agent_message with an input_text header plus an encrypted_content block containing the task.
    • Provider submission then failed before model execution with: invalid request body: Invalid 'input': value did not match any expected variant.
    • Four attempts across two parent sessions failed identically despite different task names and prompt lengths.
    • The child task envelope itself contained encrypted_content, so changing fork_turns would not remove this incompatibility.

    We have disabled both multi_agent and multi_agent_v2 and added a fresh-process drift gate until provider-aware standard message delivery or an explicit fail-fast compatibility check is available. No credentials, task text, repository paths, or ciphertext are included here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CLIIssues related to the Codex CLIbugSomething isn't workingcustom-modelIssues related to custom model providers (including local models)subagentIssues involving subagents or multi-agent features

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions