Skip to content

Give Muse Spark its own reasoning effort ladder - #979

Merged
TheGreatAxios merged 3 commits into
mainfrom
cl-7867-muse-spark-cannot-select-minimal-reasoning-effort-paying-6x
Sep 14, 2026
Merged

TheGreatAxios merged 3 commits into
mainfrom
cl-7867-muse-spark-cannot-select-minimal-reasoning-effort-paying-6x

Conversation

@TheGreatAxios

Copy link
Copy Markdown
Collaborator

Closes CL-7867.

muse-spark-1.3-contributor fell through to UNKNOWN_MODEL_EFFORTS, so minimal was unreachable from the model picker, --reasoning-effort, and agent profiles.

Why it matters

Reasoning tokens are ~95% of every completion on this model — a 58-token answer ships behind 500–2000 reasoning tokens — so the effort rung is the largest single lever on token spend and latency.

Direct /responses probes, store: false, deterministic regex graders, 2–3 repeats per cell:

effort bugfix task concurrency task avg reasoning (concurrency)
minimal 3/3 3/3 328
low 3/3 3/3 787
medium 3/3 3/3 2036
high 3/3 3/3 1933

6.2x the reasoning spend at medium versus minimal, identical pass rates. high is not monotonic with medium — it reasons less while passing the same, which is what a saturated rung looks like.

The gateway accepts minimal and rejects none with HTTP 400 on reasoning.effort, so the rung set is minimal/low/medium/high. Family default is low, not minimal — both saturate the graded tasks, and low still costs a quarter of medium.

Max-tokens floor

Reasoning consumes max_output_tokens before any answer text:

effort 512 1024 2048+
minimal 2/2 2/2 2/2
low 1/2 2/2 2/2
medium 0/2 2/2 2/2
high 0/2 2/2 2/2

At 512 with medium, the model spent 397 tokens reasoning and returned 3 tokens of answer. SOURCE_MAX_TOKENS is 16384 with no override path, so nothing is broken today — the added test is what stops a future per-source override dropping below the floor.

Testing

bun run typecheck, bun run lint, full suite: 8893 pass. The 3 failures are pre-existing on main (pack-inflation-guard-delta, and an order-dependent default-sink vendor test that passes in isolation).

Confirmation gate (--repeats 5 against baseline-0286.json) is the remaining unchecked Outcome box.

@linear-code

linear-code Bot commented Sep 13, 2026

Copy link
Copy Markdown

CL-7867

muse-spark-1.3-contributor fell through to UNKNOWN_MODEL_EFFORTS, so
`minimal` was unreachable from the picker, the flag, and agent profiles.

Measured against the Go Responses endpoint, reasoning tokens are ~95% of
every completion and scale 6x from minimal to medium with no change in
pass rate on two objectively graded tasks. The gateway accepts minimal
and rejects `none` with HTTP 400, so the rung set is minimal/low/medium/
high and the family default is `low`.

Also pins SOURCE_MAX_TOKENS above the measured truncation floor:
reasoning consumes max_output_tokens before any answer text, so a 512
cap at medium effort returns 3 tokens of answer, not a shorter answer.

CL-7867
@TheGreatAxios
TheGreatAxios force-pushed the cl-7867-muse-spark-cannot-select-minimal-reasoning-effort-paying-6x branch from bbaa26f to 6a23625 Compare September 13, 2026 18:01
@TheGreatAxios TheGreatAxios changed the title feat(reasoning-effort): give Muse Spark its own effort ladder Give Muse Spark its own reasoning effort ladder Sep 13, 2026
The id list covered the two ids in packages/opencode-go and missed the
three in packages/zen: muse-spark-1.3, muse-spark-1.2, and
muse-spark-1.3-contributor-free. Nothing normalizes the model string on
the way to supportedEfforts, so those three still fell through to the
unknown-model ladder and still could not select minimal — the exact
symptom this change exists to fix, surviving on the ids nobody tested.

Confirmed against the live endpoints that minimal returns 200 and none
returns 400 for the two contributor ids and for contributor-free.
muse-spark-1.3 and -1.2 answered with a billing gate rather than a model
rejection, so their ladder is inferred from the family, not measured.

A prefix also matches isMuseSparkLeafProvider in provider-family.ts,
which already keyed off /^muse-spark/i — the two would otherwise have
disagreed about which models are Muse Spark.

CL-7867
@TheGreatAxios

Copy link
Copy Markdown
Collaborator Author

Critic review · Comment

Gives Muse Spark a minimal/low/medium/high ladder with a low family default, matched by /^muse-spark/i prefix so all five catalog ids land on it. Verified against PR head 5ff7f6163.

Evidence checked

  • src/provider/reasoning-effort.tssupportedEfforts, validateEffort, defaultEffortForModel all funnel through the one new isMuseSparkModel branch, placed after the Codex/OpenAI/grok/GLM branches (no overlap with any of them) and before the unknown-model fallback. Picker (cycleReasoningEffort in src/tui/runner/wiring.ts:355), agent profiles (validateEffort in src/subagent/agent-fleet.ts:1011), and request wiring (resolveSessionEffort in src/config/inference-sources.ts:86) all read this single choke point, so minimal is reachable from every surface named in the PR body. none stays rejected, matching the reported gateway 400.
  • Five-id claim checks out: packages/opencode-go/src/models.ts ships the two -contributor ids, packages/zen/src/models.ts:69-71 ships muse-spark-1.3, muse-spark-1.2, muse-spark-1.3-contributor-free. Prefix covers all; the test.each pins all five plus a not-muse-spark-1.3 negative.
  • Ran on PR head: bun test src/provider/reasoning-effort.test.ts src/config.test.ts — 164 pass, 0 fail. bun run typecheck — clean. (main typechecks clean too, so no drift.)
  • SOURCE_MAX_TOKENS is 16384 (src/config/index.ts:106); the new floor test is a real assertion on the exported constant.

Should-fix (comment-only, no behavior change)

  • src/provider/reasoning-effort.ts:82 — the comment claims the prefix "matches isMuseSparkLeafProvider in src/subagent/provider-family.ts, which already keyed off the same prefix." That symbol does not exist anywhere in the repo (git grep on PR head hits only this comment; provider-family.ts defines grok/kimi helpers only). Same claim is in the patch-2 commit message. A future reader grepping for it finds nothing. Reword to cite the file's actual prefix convention (e.g. the /^grok/i + trim() style already used there) or drop the sentence.

Nits (non-blocking)

  • src/config.test.ts — the truncation-floor test asserts on a config constant but lives inside describe("buildOpenAISource"); it belongs next to the constant's own tests, if any.
  • isMuseSparkModel trims and case-folds while sibling matchers (GLM_53_MODELS.includes, startsWith("grok")) do neither. Harmless — no caller passes padded ids, and it matches provider-family.ts style. Genuinely fine as-is.

Gaps (not covered)

  • Live gateway behavior (minimal→200, none→400) is author-measured, not repo-testable; taken on trust.
  • Full suite and lint not re-run here; author reports 8893 pass with 3 pre-existing main failures. Touched files are fully green above.

Verdict

No functional blockers — merge-ready once the isMuseSparkLeafProvider comment reference is corrected. Suggested keeper for testsmith (optional): a cycleReasoningEffort case for an unset muse-spark model pinning low→medium.

The prior comment named isMuseSparkLeafProvider, which exists nowhere.
@TheGreatAxios
TheGreatAxios merged commit 604c12e into main Sep 14, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant