docs(playbooks): re-resolve fable-5 model adaptation on Opus fallback - #1931
Conversation
Meta-rule 3 resolved the running model once, at arm time. The Claude Fable 5 & Claude Mythos 5 system card documents a case that misses: Fable 5's safeguard classifiers re-serve a request with the latest Claude Opus model rather than refusing, non-configurably on some interfaces (§1.5), and on Terminal-Bench for the remainder of the trajectory rather than the one request (§8.3). A session armed as Fable 5 can therefore be answered by Opus 4.8 while still running Fable-calibrated deltas, with opus-4-8.md already shipped and nothing routing anyone to it. The added sentence makes the resolution re-runnable. Phrased on a fallback signal reaching the session, not on the model noticing one: the card names a recipient for only two of the three signals it describes, and neither is the model. Names no interface as non-configurable, because the card names none. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
Claude finished @kyle-sexton's task in 4m 54s —— View job Review complete ✅
SummaryReviewed the 3 changed files ( Checks performed:
Minor, non-blocking observation: the PR description says the change "adds one sentence," but the diff appends what reads as three sentences (via em-dash and period breaks) onto meta-rule 3. Purely a PR-body wording nit — doesn't affect the shipped doctrine text, which is otherwise scoped and phrased carefully per the PR's own stated scope-discipline rules. No correctness, architecture, or maintainability issues found. Security review is out of scope for this lane per |
|
Claude finished @kyle-sexton's task in 48s —— View job Security review complete
No security issues found. All three changed files are prose/config-only, with no code-execution, injection, or trust-boundary surface:
No GitHub Actions workflows, hook contracts, |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d6b9f07f6e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…d states Terminal-Bench is the card's only report of how long a fallback lasts, and it does not say whether the persistence comes from the fallback mechanism or from how the mini-SWE-agent harness continues after a refusal. Claim only that a fallback can outlive the request that tripped it -- which is what makes a one-time model resolution unsafe -- rather than presenting trajectory-scoped persistence as a general property. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
Claude has reviewed this PR 4 times. The lane skips further automatic reviews after 5; deleting this comment resets the count. |
|
Claude finished @kyle-sexton's task in 36s —— View job Security review complete
No security issues found. This PR touches exactly the 3 files listed in the diff, all prose/config with no code-execution, injection, or trust-boundary surface:
No GitHub Actions workflows, |
|
Claude finished @kyle-sexton's task in 2m 35s —— View job Review complete ✅
SummaryThis is the fourth automated pass on the PR; the prior three (two code-review, one security) found no issues. This pass focuses on what changed since the last review: commit What that commit fixed (correctly): the prior CHANGELOG wording cited two different card sections for one claim — §8.3 (Terminal-Bench: fallback persists "for the rest of the trajectory") and §6.2.3.2 ("fallback occurred in over half of conversation transcripts") — as joint evidence that fallback is "not reliably per-request." But §6.2.3.2 is a frequency statistic (how often fallback happens across transcripts), not a duration statistic — it says nothing about whether a given fallback outlives the request that triggered it. Citing it alongside §8.3 overstated the evidence for persistence. The new text drops §6.2.3.2 from that claim, keeps only the §8.3 trajectory fact, and adds an explicit caveat that the card doesn't say whether the persistence comes from the fallback mechanism itself or from how the Terminal-Bench harness continues after a refusal. That's a real correctness fix — it narrows an evidentiary overreach rather than just rewording — and On Codex's P2 comment (fallback trigger observability): this wasn't addressed by the new commit, and I don't think it needed to be — but it's worth restating precisely, since it's a legitimate design/correctness question (code-review lane territory, not security). Meta-rule 3's new clause says fallback signals are "addressed to the surface rather than to you [the model]," then immediately instructs "so when one reaches the session, re-resolve this rule." Per the card, two of the three signals have a named recipient that is explicitly not the model (client-app UI notification, API response-object field); the third's recipient is unstated. The rule is honest that it doesn't know whether the model can ever observe the trigger it's conditioned on — that's the right call given the calibration chapter's evidence discipline (no claim beyond what the card states), and the alternative (asserting a host-side reinjection mechanism the card never describes) would be the overreach this same PR just finished removing elsewhere. So this reads as a knowingly-scoped limitation rather than an oversight: on interfaces where no signal reaches model context, the rule is inert by design rather than wrong. Flagging as a non-blocking suggestion in case a future revision wants to note that inertness explicitly, but it doesn't block this PR. Mechanical checks (all pass):
No blocking issues. Security review is out of scope for this lane per |
…llback cite "the one agentic run the card reports it for" read as though only one run saw fallback. Section 8.1 says the opposite -- fallback is why Fable scores lower than Mythos across the capability suite. Terminal-Bench is singular only in reporting how LONG a fallback lasts, which is the property the rule needs. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 1m 57s —— View job Review complete ✅
SummaryThis is the fifth automated pass. The prior four found no blocking issues; this pass focuses on what changed since the last review — commit What that commit fixed (correctly): the previous wording scoped the §8.3 duration fact with the qualifier "in the one agentic run the card reports refusal rates for" — but that qualifier describes an incidence stat (the 20.9% fallback rate on Terminal-Bench), not the duration stat ("for the rest of the trajectory") the sentence is actually building toward. Conflating the two overstated how narrowly-scoped the 20.9% figure itself is, while under-specifying what's actually singular about the source (that §8.3 is the card's only duration claim, not its only incidence claim). The new wording untangles this cleanly:
Checks performed:
No blocking issues. Security review is out of scope for this lane per Note: I was unable to run |
|
Claude finished @kyle-sexton's task in 52s —— View job Security review complete
No security issues found. This is the fifth automated pass on the PR. The newest commit, Confirming the full-PR diff against
No GitHub Actions workflows, |
…-resolve trigger The re-resolve line conditioned on a fallback signal "reaching the session", but every signal the card describes is addressed to the surface — when none lands in model-visible context the rule could never fire, leaving Fable-calibrated doctrine applied by the fallback model silently. Name that residual case as the surface's to close, and widen the trigger to any in-context evidence of fallback (relayed notice, user statement, surfaced session event). Also rephrase "non-configurably" (typos gate). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 1m 31s —— View job Security review complete
No security issues found. This is the sixth automated pass on the PR. The newest commit, Re-confirming the full-PR diff against
No GitHub Actions workflows, |
|
Claude finished @kyle-sexton's task in 2m 15s —— View job Review complete ✅
SummaryThis is the sixth automated pass. It focuses on what changed since the last review — commit What this commit does, and why it's correct: it's a direct response to Codex's P2 comment — that on interfaces where none of the card's three fallback signals lands in model-visible context, the prior wording ("re-resolve ... when one reaches the session") could never fire, silently leaving Fable-calibrated doctrine running under the fallback model. The new meta-rule 3 text does two things instead of trying to conjure observability the card doesn't document:
Also folded into the same commit: Minor, non-blocking style note: the reworded clause — "can re-serve a request with the latest Claude Opus model instead — not configurable on some interfaces, and in the one run the card reports a duration for, persisting for..." — drops the subject from "not configurable on some interfaces" (previously an adverbial phrase directly modifying the verb; now a standalone fragment joined by "and" to an unrelated clause). It's readable in context and forced by the typos-gate constraint, but it's a slightly rougher parse than the sentence it replaced. Not worth blocking on. Checks performed:
No blocking issues. Security review is out of scope for this lane per |
…gures (#1942) No linked issue ## Summary The boris orchestration reference (`plugins/playbooks/skills/boris/reference/orchestration.md`) carries a header note classifying its benchmark figures (Sections 78 and 94) as launch-day snapshots with a re-resolve trigger. Section 94/95's Fable 5 price literals ($10/M input, $50/M output, cache write $12.50, cache read $1, full 1M context at standard rate) and Sections 78/95's relative price claims ("same price as 4.7", "exactly 2x Opus 4.8") sat outside that classification, readable as current rates. This extends the note to classify pricing figures (Sections 78, 94-95) the same way — launch-day rates, verified still current against the live [pricing page](https://platform.claude.com/docs/en/about-claude/pricing) on 2026-08-04 — and routes current-rate resolution upstream. No figures added or removed; the vendored upstream mirror is untouched. Playbooks 0.6.18: serialized one patch above unmerged #1931's 0.6.17 claim per the one-in-flight-claim-per-plugin rule; whichever lands second rebases. ## Related - #1931 (playbooks 0.6.17 claim, row 240) - Doc-alignment campaign save-point: #1941 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…aces (#1950) No linked issue Row 145 of the doc-alignment campaign (lane P). Four items plus the changelog entry item (a) owes. Live pages fetched **2026-08-04** via the campaign's raw-markdown channel (`.md` appended to the `code.claude.com` URL path); the shipped docs cite the reader-facing non-`.md` URLs. ## (a) `docs/conventions/loop-lane/README.md` — pointer-not-copy The `PushNotification` paragraph enumerated four of Remote Control's requirements. The live [Requirements](https://code.claude.com/docs/en/remote-control) section carries five bullets — Subscription, Authentication, API endpoint, Feature-flag evaluation, Workspace trust — and three of the four enumerated conditions under-covered their live counterparts (evidence in item (e)). Extending the list would re-drift: the section carries version-dated clauses of its own ("As of v2.1.196…"). The paragraph already cited the page, so the enumeration is replaced by a pointer at the Requirements section **and** the page's mobile-push setup steps, with exactly one fact kept inline: `DISABLE_TELEMETRY`, `DO_NOT_TRACK`, `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC`, and `DISABLE_GROWTHBOOK` each disable the feature-flag evaluation Remote Control depends on. This repo ships OTEL and hook-telemetry conventions, so its readers are the population that sets those variables. The Team/Enterprise Owner toggle is deliberately left to the pointer — it is a research-preview-era default. Net 6 lines replacing 7: a conversion, not an extension. ## (b) `plugins/playbooks/skills/boris/reference/advanced.md` §35 — amended, per precedent Tweet-sourced (March 2026): "Available on Max, Team, and Enterprise (v2.1.74+)." Upstream now states the availability two ways in two places, and the amendment quotes **both** verbatim rather than picking one: the page note reads "in research preview and available on all plans", while Requirements reads "available on Pro, Max, Team, and Enterprise plans". The amendment names Requirements as owning the current list. This file is a repo-authored distillation, not the frozen vendor capture (`SKILL.md`: "The verbatim upstream baseline lives at `vendor/SKILL.md` for drift detection only"), and `scripts/update.sh` states that integration into `reference/*.md` is manual — so a hand-edit survives the next sync. Precedent followed rather than chosen on taste: `reference/autonomy.md` §72 lines 205–220 amends a tweet-sourced claim that upstream moved past with an **"Amended (verified DATE against LINK)"** blockquote — original preserved, upstream quoted verbatim, recheck trigger. (`reference/orchestration.md:22` was considered and rejected as the model: that is a tweet superseded by a later *tweet*, not by upstream docs.) ## (c) `plugins/playbooks/reference/model-adaptation/opus-4-8.md` — dating record The Sources note said the Opus 4.8 guide "has not been re-read at all since then". It was re-read 2026-08-04 (ROSTER row 221 close-out). Re-verified independently for this PR: literalism at lower effort, strict effort adherence with `xhigh` for coding and `high` as the intelligence-sensitive floor, reasoning favored over tool calls, fewer subagents by default, native progress updates, response-length calibration, and the code-review-harness recall effect with coverage-before-filtering — all present, no material drift. The note now records the re-read instead of an absence. ## (d) `plugins/playbooks/skills/fable-5/context/verification.md` — link text `[Slash commands]` → `[Commands]`; that page's live H1 is `# Commands`. URL unchanged. ## (e) `docs/conventions/loop-lane/CHANGELOG.md` — **8.0.1**, the entry item (a) owes Codex P2 on `README.md:274`, verified real. The convention's own §Versioning (`README.md:713-725`) states: "A firing that finds drift lands its outcome as a changelog entry; a no-drift firing refreshes the claim's verification date in place — no entry, no bump". Item (a) found drift, so it fires the entry-owing branch, and the original diff left this file at 7.0.1. The drift was re-established from raw bytes rather than from any second-hand description (`curl https://code.claude.com/docs/en/remote-control.md` → **HTTP 200, 42173 bytes**, 2026-08-04), diffed against `git show origin/main:docs/conventions/loop-lane/README.md`. Three evidenced mismatches: 1. **Feature-flag evaluation** had no counterpart in the four-item enumeration at all — the load-bearing one, and why the PR keeps that fact inline. 2. **API endpoint** under-covered the `ANTHROPIC_BASE_URL` clause, which the live bullet self-dates "As of v2.1.196". 3. **Subscription** under-covered the Team/Enterprise Owner-toggle precondition. Deliberately **not** written into the entry: any claim that the section "moved" or "changed". No prior page snapshot exists, so only a state mismatch is evidenceable, and that is how the entry is phrased. The live page's two availability statements — "all plans" in the page note, "Pro, Max, Team, and Enterprise plans" under Requirements — are page-internal looseness, not drift against this repo: the pre-change README's tier list already matched Requirements verbatim. ## Version and rebase dependencies Two separately versioned artifacts, two claims: `plugins/playbooks` 0.6.16 → **0.6.19**, and `docs/conventions/loop-lane/CHANGELOG.md` → **8.0.1**. `docs/conventions/` is not a plugin, so the loop-lane entry carries no plugin-version implication and `playbooks` stays at 0.6.19. **Playbooks chain, in order:** `#1931` (playbooks 0.6.17) → `#1942` (playbooks 0.6.18) → **this PR #1950** (playbooks 0.6.19) → a forthcoming consolidation PR (playbooks 0.6.20) **Plus a newly discovered dependency:** this PR now also claims `docs/conventions/loop-lane/CHANGELOG.md` **8.0.1**, which sits above the unmerged **8.0.0** claim in **#1830** (`feat/1525-trusted-internal-bot-logins`, not draft — the only other open PR touching that file). **This PR must therefore be rebased after #1830 lands as well.** `8.0.1` rather than `7.0.2` because the campaign's serialization rule takes patch+1 above the highest claim merged *or* unmerged; `7.0.2` would only be correct if this PR landed first, and merge ordering is not controlled here. `check-changelog-parity.sh --check-order` compares **version** order, not dates, so the post-rebase sequence `8.0.1 / 8.0.0 / 7.0.1` is ordered correctly despite 8.0.0 carrying the earlier date. ## Verification - `scripts/check-changed-skills.sh origin/main` — PASS, 0 errors (boris, fable-5) - `scripts/check-changelog-parity.sh` `--check`, `--check-bump origin/main`, `--check-order` — all pass, re-run after the 8.0.1 entry (`All 71 changelog(s) read newest-first with no duplicate versions`) - `markdownlint-cli2` on all changed markdown files — 0 errors ## Reported, not shipped `plugins/playbooks/skills/skill-authoring/reference/verification-loops-in-skills.md:73` carries the same stale `[Slash commands]` link text at the same URL. Same fix, same plugin — held out of this PR because the row's scope is fixed at four items. ## Related - Doc-alignment campaign ROSTER row 145 (Remote Control), lane P - ROSTER row 221 — the Opus 4.8 guide re-read this PR's item (c) records - #1931, #1942 — unmerged `playbooks` version claims this PR sequences behind - #1830 — unmerged `docs/conventions/loop-lane/CHANGELOG.md` 8.0.0 claim this PR's 8.0.1 sequences behind - <https://code.claude.com/docs/en/remote-control>, <https://code.claude.com/docs/en/tools-reference>, <https://code.claude.com/docs/en/commands>, <https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-opus-4-8> 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 participant
No linked issue
What
plugins/playbooks/skills/fable-5/SKILL.mdmeta-rule 3 resolved the running model once, at arm time, and routed it to itsreference/model-adaptation/chapter. This adds one sentence making that resolution re-runnable, plus the CHANGELOG entry and the plugin version bump (0.6.16 → 0.6.17).Why
Doc-alignment campaign row 240 digested the Claude Fable 5 & Claude Mythos 5 System Card — the tier
playbooks:fable-5is authored for and named after, and the last major Anthropic primary source the campaign had not ingested (Opus 5's card shipped in row 4).The card documents a case meta-rule 3 misses. Fable 5's safeguard classifiers — cybersecurity, biology and chemistry, distillation, frontier LLM development — do not merely refuse; they re-serve the request with the latest Claude Opus model:
So a session that armed as Fable 5 can be answered by Opus 4.8 from a classifier hit onward while still running Fable-calibrated deltas — and meta-rule 3's own warning that deltas are calibrated per model version is what makes that bite. The plugin already ships
opus-4-8.md; nothing routed a Fable-armed session to it.Scope discipline
SKILL.mdalready confines toreference/model-adaptation/— a directory that carries nofable-5.mdby design, since Fable 5 is the model the playbook was authored by and for.Verification
f95d413845ad8624f384ba026963f2bad2158f10f2626575bb45e823e3c2e0ca, 26,960,957 bytes, 317 pages, final URLhttps://www-cdn.anthropic.com/57a52ea7d8f0e54e8a542e908266086df425cdf5/Claude%20Fable%205%20&%20Claude%20Mythos%205%20System%20Card.pdf.markdownlint-cli2clean on both changed markdown files.Related
plugins/playbooks/reference/model-adaptation/opus-5.md; this row closes the asymmetry the roster flagged, following the same precedent that the file drawing a fact cites the card inline.plugins/playbooks/reference/model-adaptation/opus-4-8.md, which previously had no path from a Fable 5 arm.🤖 Generated with Claude Code