Skip to content

STRING_KEYS no longer names model, which the run at 2.1.306 never showed (closes #247) - #300

Merged
don-petry merged 2 commits into
mainfrom
tools/string-keys-247
Oct 22, 2026
Merged

don-petry merged 2 commits into
mainfrom
tools/string-keys-247

Conversation

@don-petry

@don-petry don-petry commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Closes #247: STRING_KEYS names one key fewer, as the run at 2.1.306 showed.

What the run showed (trials/P-13.v2-1-306.json, merged in #291):

  • Within the fields the script keeps, nine of STRING_KEYS' eleven keys held a string: type, subtype, stop_reason, service_tier, inference_geo, speed, canonicalModel, provider and costBasis.
  • tool_name is reached only when a tool is refused, and that run refused none.
  • model appeared nowhere. The model's name is a key of modelUsage and the value of canonicalModel, never under a key model.

The change:

  • tools/trial_p13_cli.sh drops model from STRING_KEYS, so a word under that key is no longer kept. It keeps tool_name, since another run may refuse a tool.
  • TrialScript pins the list as it pins KEPT_RESULT.
  • The stand-in answer plants a one-word code under model, so the test that no model text reaches OUT fails if model is named again. A mutant that names it is killed.

The other half of the item: decided, no per-field lists. usage and modelUsage held numbers and seven code words under named keys (service_tier, inference_geo, speed, type under usage; canonicalModel, provider, costBasis under modelUsage). One list already names just those words and the answer's own codes, so per-field lists would hold the same words. The script's comment says so, and the test now shows the dropped word recorded in model_text_removed.

Verification. make verify passed in the worktree, with history, at SIM_SEEDS=40.

Reviews. A fresh Claude review with a security pass on 14247d9 found no blocker and no major. It confirmed #247's other three items are on main (#263). Its minors are fixed in 30dc6b3. Muse not run (the Maintainer, 2026-10-20: Kernel/hook only).

🤖 Generated with Claude Code

View guided diff Turn on auto-fix

don-petry and others added 2 commits October 21, 2026 18:08
…wed (closes #247)

The run recorded in trials/P-13.v2-1-306.json found strings under nine of the eleven keys; model
held none, the model's name being a key of modelUsage and the value of canonicalModel. tool_name
stays, as a refused tool would fill it. TrialScript pins the list, and the stand-in plants a word
under model, which a mutant that names it again lets through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nt says one list serves every field

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@don-petry
don-petry requested a review from a team as a code owner October 21, 2026 23:34
@coderabbitai

coderabbitai Bot commented Oct 21, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 52 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: combsmith/software-factory/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c441c983-259c-471b-abc6-b8fd543ab983
📥 Commits

Reviewing files that changed from the base of the PR and between 7e03db6 and 30dc6b3.

📒 Files selected for processing: 3
  • tests/test_tools.py
  • tools/mutants.py
  • tools/trial_p13_cli.sh
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T23:47:49.112358Z 30dc6b3 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@sonarqubecloud

Copy link
Copy Markdown

@don-petry
don-petry added this pull request to the merge queue Oct 22, 2026
Merged via the queue into main with commit 3057b55 Oct 22, 2026
19 checks passed
@don-petry
don-petry deleted the tools/string-keys-247 branch October 22, 2026 00:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Trial scripts: what Muse's later rounds asked of trial_p13_cli.sh

1 participant