Skip to content

feat: Codex model fallback — probe and cache working models (v0.12.6.0) - #543

Closed
amoeri wants to merge 4 commits into
garrytan:mainfrom
amoeri:feat/codex-model-fallback
Closed

amoeri wants to merge 4 commits into
garrytan:mainfrom
amoeri:feat/codex-model-fallback

Conversation

@amoeri

@amoeri amoeri commented Mar 27, 2026

Copy link
Copy Markdown

Summary

If your ChatGPT subscription doesn't include the model Codex defaults to, every
/codex invocation silently failed and fell back to Claude reviewing itself. The
whole point of Codex is an independent second opinion from a different AI. That was broken.

Probe-and-cache script (bin/gstack-codex-model):

  • Probes Codex CLI for a working model, tries user's config default first
  • Falls through gpt-5.3-codex → gpt-5.2-codex on "not supported" errors
  • Caches result for 24h (~6ms reads, ~6s probe on cache miss)
  • Configurable via gstack-config set codex_fallback_models

11 invocation sites updated across 4 source files + 10 regenerated skills:

  • codex/SKILL.md.tmpl — Step 0.5 model probe + error handling
  • scripts/resolvers/review.ts — 4 sites (OH voice, adversarial, structured review, plan review)
  • scripts/resolvers/design.ts — 3 sites (design lite, sketch, outside voice)
  • autoplan/SKILL.md.tmpl — 1 probe + 3 phase commands

Uses -c model="$CODEX_MODEL" which works for both codex exec and codex review
(discovered during review that codex review doesn't accept -m).

Commits

  1. feat: add gstack-codex-model probe-and-cache script — new bin/ script
  2. feat: wire model probe into all codex invocation sites — templates + resolvers
  3. chore: regenerate SKILL.md files from updated templates — 10 generated files
  4. chore: bump version and changelog (v0.12.6.0)

Pre-Landing Review

Reviewed via /autoplan (CEO + Eng phases). 12 decisions made, 10 auto-decided,
2 taste decisions confirmed by user (inline two-liners over helper abstraction,
manual verification over automated bash tests).

Key findings addressed:

  • Use -c model= not -m (codex review doesn't support -m)
  • Don't parse ~/.codex/config.toml — probe the default directly
  • Atomic cache writes (temp + mv)
  • Parse specific "not supported" errors, don't cache transient failures

Test Coverage

  • 378/378 skill validation tests pass
  • Probe script tested manually: broken model → fallback works, cache hit ~6ms, --probe bypasses cache

Test plan

  • Skill validation tests pass (378/378)
  • Skill parser tests pass
  • SKILL.md regeneration clean (29 files)
  • Manual: broken model config → probe finds gpt-5.3-codex
  • Manual: cache read ~6ms, probe ~6.6s
  • Manual: --probe bypasses cache

🤖 Generated with Claude Code

amoeri and others added 4 commits March 27, 2026 03:56
Probes Codex CLI for a working model when the user's configured
default isn't available on their subscription. Tries the user's
config first, then falls through gpt-5.3-codex and gpt-5.2-codex.
Caches result for 24h at ~/.gstack/codex-model-cache. Returns
"DEFAULT" (user config works), a model name (fallback), or "NONE"
(exit 1). Supports --probe to force re-probe and configurable
fallback list via gstack-config set codex_fallback_models.
Adds 2-line probe+guard to every codex bash block across templates
and resolvers. Uses -c model="$CODEX_MODEL" (works for both
codex exec and codex review). When probe returns NONE, existing
Claude fallback paths activate. When DEFAULT, no model flag added.

Updated sites:
- codex/SKILL.md.tmpl: Step 0.5 + error handling
- scripts/resolvers/review.ts: 4 sites (OH voice, adversarial,
  structured review, plan review)
- scripts/resolvers/design.ts: 3 sites (design lite, sketch, voice)
- autoplan/SKILL.md.tmpl: 1 probe + 3 phase commands
All 10 skills that use codex via review/design resolvers now
include the model probe pattern in their generated output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@time-attack

Copy link
Copy Markdown
Contributor

@16francej — recommend closing this PR as superseded by #1628. The branch hard-codes outdated model choices and may confuse authentication or network failures with unsupported models. Preserve contributor credit for the probing idea, but carry forward only bounded capability checks with explicit failure classification.

@capy-ai

capy-ai Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

Thanks for this, @amoeri. This is now covered on main: Codex model resolution now goes request, then GSTACK_CODEX_MODEL, then config.toml, then the default, with a probe for unusable models (shipped in v1.91.16.0, #3014). Closing as superseded; if you still see the problem on the latest gstack, please open an issue with ./setup --status output.

@capy-ai capy-ai Bot closed this Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants