Background & Upstream Change
Upstream Prime Agent PR #2453 introduced sub-agent image-model routing:
- When sessions encounter image assets or multimodal tasks, sub-agents can be spawned or routed to an alternate, vision-capable model (or a cheaper vision model) instead of strictly inheriting the parent conversation's model.
Analysis for Pylon
1. Interaction with Composer Model Selector
- In Pylon, the user explicitly selects a primary model via the composer selector, or relies on workspace defaults.
- If image-model routing is strictly agent-internal (transparently dispatched by the agent runtime when inspecting images), it would run behind the scenes without overriding the user's primary composer model.
- However, if the user explicitly chose an expensive or specific model, silent fallback to an unconfigured or unapproved provider/model for vision could lead to unexpected behavioral differences or privacy/compliance leaks (e.g. sending images to an external third-party vision provider).
2. Interaction with Quota Tracking & Accounting
- Pylon tracks provider token usage and quotas per instance/workspace.
- If sub-agents route images to a secondary model:
- Token accounting and spend calculations must be correctly attributed to the secondary model's rates in the journal/ledger.
- Quota exhaustion on the image model must not wedge the primary conversation or corrupt turn settlement.
3. UI and Configuration Exposure Recommendation
- Phase 1 (Config-only / Agent-internal):
- Expose an optional setting in provider settings (e.g.
imageProcessingModel or visionModel), similar to upstream's auxiliaryModel or subagentDefaultModel.
- When unset, default to the active session model (preserving existing behavior).
- Phase 2 (UI indicators):
- In the session inspector / subagent view, indicate when an image-processing subagent ran with an alternate model.
- Do not complicate the primary composer model picker with auxiliary sub-pickers unless the user enters a specialized multimodal inspection mode.
Action Items
Background & Upstream Change
Upstream Prime Agent PR #2453 introduced sub-agent image-model routing:
Analysis for Pylon
1. Interaction with Composer Model Selector
2. Interaction with Quota Tracking & Accounting
3. UI and Configuration Exposure Recommendation
imageProcessingModelorvisionModel), similar to upstream'sauxiliaryModelorsubagentDefaultModel.Action Items
imageModelconfiguration field to Pylon's Prime provider settings.