Summary
Extend conductor doctor --models (and the underlying ProviderDiagnostic/--json output) to surface, per model:
- Reasoning effort support — which
reasoning.effort levels (low/medium/high/xhigh) a model accepts, plus its default.
- Context window — the model's prompt/output/total token limits.
Motivation
Today conductor doctor --models only lists model ids (_models_cell in src/conductor/cli/doctor.py). Users configuring reasoning.effort or worried about prompt size have no way to discover per-model limits without hitting a ValidationError at runtime or reading provider source. This data is already fetched internally for validation — it just isn't surfaced to the user.
Current state (investigated)
-
Copilot provider (src/conductor/providers/copilot.py):
list_models() already calls client.list_models(), which returns Model objects with:
capabilities.supports.reasoning_effort (bool)
supported_reasoning_efforts: list[str] | None
default_reasoning_effort: str | None
capabilities.limits.max_prompt_tokens, max_output_tokens, max_context_window_tokens
_validate_reasoning_effort_for_model already reads supported_reasoning_efforts for validation — same data, not exposed to diagnostics.
get_max_prompt_tokens() already surfaces max_prompt_tokens for the dashboard context-window bar, but doctor doesn't call it.
-
Claude provider (src/conductor/providers/claude.py):
list_models() / get_max_prompt_tokens() use Anthropic's models.list(), which only exposes max_input_tokens — no output/total split, no per-model reasoning-effort list.
- Reasoning support is a static heuristic:
is_claude_thinking_model() (prefix match on claude-3-7-, claude-opus-4, claude-sonnet-4, claude-haiku-4) — a boolean "supports extended thinking", not a level list. All 4 Conductor effort levels map to fixed token budgets (EFFORT_TO_BUDGET_TOKENS in src/conductor/providers/reasoning.py) when supported.
-
Claude Agent SDK provider: declares reasoning_effort: None in CAPABILITIES (no concept at all) — should report "n/a" for both new columns.
Proposed approach
- Add optional provider hooks alongside the existing
get_max_prompt_tokens / get_model_pricing pattern in AgentProvider (src/conductor/providers/base.py), e.g. get_model_capabilities(model) -> ModelCapabilityInfo | None, returning:
supported_reasoning_efforts: list[str] | None
default_reasoning_effort: str | None
max_prompt_tokens, max_output_tokens, max_context_window_tokens: int | None
- Default implementation returns
None (safe degradation to "n/a"), matching the existing optional-hook contract (never raise).
- Copilot: implement via the existing
client.list_models() cache.
- Claude: implement using
is_claude_thinking_model() + the 4 Conductor levels for reasoning, and max_input_tokens for prompt tokens only (no output/total available from the SDK).
- Claude Agent SDK (experimental,
reasoning_effort capability = None): leave unimplemented (base default → "n/a").
- Extend
ProviderDiagnostic (src/conductor/providers/diagnostics.py) and gather_provider()/gather() to populate this per-model when --models is passed.
- Extend
_models_cell / add new columns in src/conductor/cli/doctor.py (_render_providers) for Rich table output, and mirror new fields in DoctorReport.to_dict() for --json.
- Degrade gracefully to
n/a/— wherever a provider or model doesn't report the data — must not fail doctor or affect its exit code.
Out of scope
- Bumping the
github-copilot-sdk pin (currently >=1.0.0, installed 1.0.1, latest stable 1.0.6) — not required since the needed fields already exist in 1.0.1. Tracked separately if desired.
Summary
Extend
conductor doctor --models(and the underlyingProviderDiagnostic/--jsonoutput) to surface, per model:reasoning.effortlevels (low/medium/high/xhigh) a model accepts, plus its default.Motivation
Today
conductor doctor --modelsonly lists model ids (_models_cellinsrc/conductor/cli/doctor.py). Users configuringreasoning.effortor worried about prompt size have no way to discover per-model limits without hitting aValidationErrorat runtime or reading provider source. This data is already fetched internally for validation — it just isn't surfaced to the user.Current state (investigated)
Copilot provider (
src/conductor/providers/copilot.py):list_models()already callsclient.list_models(), which returnsModelobjects with:capabilities.supports.reasoning_effort(bool)supported_reasoning_efforts: list[str] | Nonedefault_reasoning_effort: str | Nonecapabilities.limits.max_prompt_tokens,max_output_tokens,max_context_window_tokens_validate_reasoning_effort_for_modelalready readssupported_reasoning_effortsfor validation — same data, not exposed to diagnostics.get_max_prompt_tokens()already surfacesmax_prompt_tokensfor the dashboard context-window bar, but doctor doesn't call it.Claude provider (
src/conductor/providers/claude.py):list_models()/get_max_prompt_tokens()use Anthropic'smodels.list(), which only exposesmax_input_tokens— no output/total split, no per-model reasoning-effort list.is_claude_thinking_model()(prefix match onclaude-3-7-,claude-opus-4,claude-sonnet-4,claude-haiku-4) — a boolean "supports extended thinking", not a level list. All 4 Conductor effort levels map to fixed token budgets (EFFORT_TO_BUDGET_TOKENSinsrc/conductor/providers/reasoning.py) when supported.Claude Agent SDK provider: declares
reasoning_effort: NoneinCAPABILITIES(no concept at all) — should report "n/a" for both new columns.Proposed approach
get_max_prompt_tokens/get_model_pricingpattern inAgentProvider(src/conductor/providers/base.py), e.g.get_model_capabilities(model) -> ModelCapabilityInfo | None, returning:supported_reasoning_efforts: list[str] | Nonedefault_reasoning_effort: str | Nonemax_prompt_tokens,max_output_tokens,max_context_window_tokens: int | NoneNone(safe degradation to "n/a"), matching the existing optional-hook contract (never raise).client.list_models()cache.is_claude_thinking_model()+ the 4 Conductor levels for reasoning, andmax_input_tokensfor prompt tokens only (no output/total available from the SDK).reasoning_effortcapability =None): leave unimplemented (base default → "n/a").ProviderDiagnostic(src/conductor/providers/diagnostics.py) andgather_provider()/gather()to populate this per-model when--modelsis passed._models_cell/ add new columns insrc/conductor/cli/doctor.py(_render_providers) for Rich table output, and mirror new fields inDoctorReport.to_dict()for--json.n/a/—wherever a provider or model doesn't report the data — must not faildoctoror affect its exit code.Out of scope
github-copilot-sdkpin (currently>=1.0.0, installed 1.0.1, latest stable 1.0.6) — not required since the needed fields already exist in 1.0.1. Tracked separately if desired.