Repository navigation
fix(chat): accept Pi and Anthropic-shaped image parts on the chat wire - #4497
Conversation
Aside (Pi agent core, openai-completions) returns read_file screenshots as
tool-result parts of shape {type:"image", data, mimeType}. The chat
inbound translator only recognized image_url parts and silently dropped
these, so vision models answered that the image data never arrived
(live-reproduced against the running proxy: claude-opus-5 with a Pi-style
tool image reports the image missing, while the same request in image_url
form is described correctly). Claude-shaped clients sending Anthropic
source-object image parts over the same endpoint were dropped the same way.
Normalize both shapes to the data-URI form the Responses pipeline already
handles.
Local suite NOT RUN per maintainer rule; hosted exact-head CI is the gate.
|
✅ Deterministic PR hygiene checks passed. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review. 📝 WalkthroughWalkthroughThe image-part normalizer now supports OpenAI, native data, and Anthropic-style image formats. Tests cover five additional image conversion cases. ChangesImage input handling
Priority: ➖ Normal Estimated code review effort: 2 (Simple) | ~10 minutes Change: Bug fix Merge Risk: ⚪ Minimal · up to The added image formats preserve their MIME metadata and targeted tests cover the supported conversions. No actionable merge risk remains. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9b9675b202
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // (Aside read_file tool results), Anthropic-shaped clients carry a source | ||
| // object. Dropping either silently blinds a vision model, so normalize both | ||
| // to the URL/data-URI form the Responses pipeline already understands. | ||
| if (part.type === "image") { |
There was a problem hiding this comment.
When /v1/chat/completions selects an eligible openai-chat provider, handleChatCompletions dispatches the untouched body through handleNativeChatCompletions before calling this translator, so this new branch never runs and the upstream receives the unsupported {type:"image", ...} object. Moreover, chatBodyCarriesImage recognizes only image_url, meaning these parts also fail to divert explicitly text-only models through the Responses vision path. Pi and Anthropic-shaped screenshots therefore remain rejected or ignored on ordinary OpenAI-compatible targets; normalize the native body or make native eligibility recognize these shapes and cover the real handler path with a regression test.
AGENTS.md reference: AGENTS.md:L376-L379
Useful? React with 👍 / 👎.
|
Maintainer integration record (MAINTAINERS.md dev exception): integrating as repository owner without a second maintainer approval. No outstanding maintainer change requests. Exact-head verification: all required checks green on head 9b9675b (ci aggregator, gates, hygiene, enforce-target, docker smoke, api usage, CodeRabbit completed; windows/macos-control skipped by changes-gate design). Local suite NOT RUN per task rule; hosted exact-head CI is the gate. User-reported blocker (Aside/Pi screenshots blind on every model) — root cause live-reproduced pre-fix; fix verified at translator level by the added regression cases; end-to-end re-verification against the live proxy follows the merge. |
Product tree is dev at eb81eaa, byte-identical (dev already carries the 2.53.0 version line). This promotion follows a 364-commit regression audit (origin/main..981b53e) by 15 parallel subagent lanes plus two audit-spawned fixes reviewed and merged to dev: lidge-jun#4496 (live sideband connect watchdog) and lidge-jun#4497 (chat image part shapes). Zero unresolved P0/P1 at promotion time. Exact-head hosted CI green on eb81eaa (run 34750934849). Local suite NOT RUN per task rule; hosted exact-head CI is the gate.
…ol-image-parts fix(chat): accept Pi and Anthropic-shaped image parts on the chat wire
1 participant
Summary
Verification
Checklist