Skip to content

feat: implement issue #1896 — [Phase 1] Price claude-opus-5-5 in model-pricing.tsv and lock it with a regression test - #1904

Merged
don-petry merged 2 commits into
mainfrom
dev-lead/issue-1896-20260923-0307
Sep 23, 2026
Merged

don-petry merged 2 commits into
mainfrom
dev-lead/issue-1896-20260923-0307

Conversation

@don-petry

@don-petry don-petry commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

[Phase 1] Price claude-opus-5-5 in model-pricing.tsv and lock it with a regression test

From the issue: As a cost-observability maintainer, I want add a date-versioned claude-opus-5-5 pricing row and a regression test that pins its rate, so that deep-tier token-cost reports price Opus 5.5 calls correctly the moment the model is invoked, instead of falling through to no matching glob.

Risk

Low — changes automation shell logic under scripts/, covered by shellcheck (--severity=warning) and the bats suite.

Test plan

Tests added/updated: tests/model_pricing.bats. Verification: bash scripts/dev-lead-lint.sh (shellcheck --severity=warning) ran pre-commit; the bats suite runs in CI.

Rollback

Revert this PR. No non-revertible side effects (no tags, migrations, or external state).

Monitoring

This PR's Lint (shellcheck) and bats checks show pass/fail; watch subsequent dev-lead / pr-review runs for behavioral regressions.

Closes #1896

Review in cubic

…l-pricing.tsv and lock it with a regression test
@qodo-code-review

Copy link
Copy Markdown

ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing

@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 44 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 9ab6db3e-5e2d-4db7-8c0e-a36f2955f412

📥 Commits

Reviewing files that changed from the base of the PR and between 4fd4eac and 2528764.

⛔ Files ignored due to path filters (1)
  • scripts/lib/model-pricing.tsv is excluded by !**/*.tsv
📒 Files selected for processing (1)
  • tests/model_pricing.bats

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds pricing details for the new claude-opus-5-5* model in scripts/lib/model-pricing.tsv and includes corresponding tests in tests/model_pricing.bats. Feedback suggests improving the test coverage by verifying the exact boundary date (2026-09-22) when the pricing takes effect, rather than just testing the day after.

Comment thread tests/model_pricing.bats Resolved
Comment thread tests/model_pricing.bats Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread tests/model_pricing.bats Resolved
Comment thread tests/model_pricing.bats Resolved
@donpetry-bot

Copy link
Copy Markdown
Contributor

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-09-23T04:24:13Z.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Skipping the failing template-drift check — out of scope for this PR, not a root cause I can fix here.

template-drift verifies that petry-projects/repo-template's committed stub workflows have not drifted from the standards-derived baseline (it runs scripts/template_stub_drift.sh). Its failure names .github/workflows/agent-shield.yml, copilot-setup-steps.yml, and dependency-audit.yml as DRIFTED and agent-ingress.yml as MISSING in the repo-template repo — the documented remedy is a re-seed via scripts/seed-repo-template.sh (edit standards/, never hand-edit the template).

This PR's diff is confined to scripts/lib/model-pricing.tsv and tests/model_pricing.bats (pricing claude-opus-5-5 and locking it with a regression test). None of the drifted/missing template stubs are in this diff, so nothing here caused the drift and nothing here can make that check green. Making it pass would require touching unrelated standards//template files — which is exactly what the "optimize the intent, not the check" guard warns against — so I'm leaving it red with this note rather than forcing it.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — review-changes (applied)

Changes committed and pushed. Requested items addressed:

  • tests/model_pricing.bats:48 — applied
  • tests/model_pricing.bats:47 — applied
  • tests/model_pricing.bats:47 — applied
  • tests/model_pricing.bats:55 — applied

@don-petry
don-petry enabled auto-merge (squash) September 23, 2026 03:31
@sonarqubecloud

Copy link
Copy Markdown

@don-petry
don-petry disabled auto-merge September 23, 2026 03:46
@donpetry-bot

Copy link
Copy Markdown
Contributor

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-09-23T04:48:40Z.

@donpetry-bot

donpetry-bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor
Superseded by automated re-review at 2528764fdff726ee660139ca20ae2faf9737b40e — click to expand prior review.

Review — fix requested (cycle 1/3)

The automated review identified the following issues. Please address each one:

Findings to fix

Automated review — NEEDS HUMAN REVIEW

Risk: LOW
Reviewed commit: 2528764fdff726ee660139ca20ae2faf9737b40e
Review mode: triage-approved (single reviewer)

Summary

Adds a date-versioned claude-opus-5-5* pricing row (2026-09-22: 4.00 / 0.20 / 5.00 / 20.00) to scripts/lib/model-pricing.tsv and two boundary regression tests in tests/model_pricing.bats. The change is data-only plus tests, follows the file's append-only effective-dated convention, and was functionally verified by this reviewer: price_for claude-opus-5-5 returns 4.00 0.20 5.00 20.00 at and after 2026-09-22, empty before it, and the claude-opus-4-* / claude-opus-4-1 rows are unaffected (globs are disjoint; most-specific-glob selection intact). The content merits approval, but the red template-drift check on the head commit fails the all-CI-green gate, so this is escalated for a human to either re-seed repo-template or merge with an explicit override.

Linked issue analysis

Linked issue #1896 is substantively addressed — all four acceptance criteria verified: (1) the exact TAB-separated row is appended in the Anthropic Claude block; (2) price_for claude-opus-5-5 returns the pinned rate (tests sample the exact 2026-09-22 boundary, stricter than the AC's 2026-09-23, per review-bot feedback); (3) no collision with claude-opus-4-* — verified price_for claude-opus-4-8 2026-06-07 still returns 5.00 0.50 6.25 25.00 and the opus-4-1 legacy row still wins as more specific; (4) shellcheck is green and the bats suite passed in CI on the head SHA.

Findings

No code findings. All 4 bot review threads (gemini, codeant, cubic x2) are resolved — the dev-lead fix commit moved the tests to the exact effective_from boundary as requested. No unanswered human-reviewer questions. Secret scan: the run_secret_scanning MCP tool is not available in this environment; the gitleaks CI check passed (SUCCESS) and the diff contains no secret-shaped content. Sole blocker: the template-drift check is FAILURE on head. Verified out of scope: scripts/template_stub_drift.sh audits the external petry-projects/repo-template repo against the standards-derived baseline via the contents API — this PR's diff (pricing TSV + bats test) cannot cause or cure that drift, and the maintainer documented the skip rationale on the PR. The documented remedy (re-seed via scripts/seed-repo-template.sh) lives outside this PR.

CI status

All checks on head 2528764 green except: template-drift = FAILURE (pre-existing external drift in repo-template — out of scope, see Findings). Key checks: bats SUCCESS, shellcheck/ShellCheck/Lint SUCCESS, gitleaks SUCCESS, CodeQL SUCCESS, SonarCloud Quality Gate passed, agent-shield SUCCESS. The 5 CANCELLED entries (dev-lead dispatch/ci-relay/resume, Dismiss, review) are superseded duplicates from the prior commit, each with a SUCCESS or SKIPPED counterpart. Advisory-bot rate limits from the earlier sweep have cleared (CodeRabbit now SUCCESS; qodo remains billing-paused, which is not a CI check).


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

Additional tasks

  1. Resolve all unresolved review thread comments from other reviewers
  2. Ensure all CI checks pass after your changes
  3. Rebase on the target branch if behind
  4. Do NOT modify files unrelated to the findings above

The review cascade will automatically re-review after new commits are pushed.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
1. **SonarCloud Quality Gate:** Passing — reports zero new issues, zero security hotspots, zero coverage/duplication regressions on new code. No actionable defects.
2. **Test coverage:** All 20 bats tests pass, including the two new boundary tests for claude-opus-5-5:
   - `price_for: opus 5.5 pinned at effective_from (2026-09-22) → $4 / $0.20 / $5 / $20` ✓
   - `price_for: opus 5.5 before effective_from (2026-09-21) → empty (no retroactive guess)` ✓
3. **PR changes:** The dev-lead agent has already addressed feedback from Cubic's review. The test now samples the exact boundary date (2026-09-22) rather than a day after, which correctly pins `effective_from` against drift—exactly as the Gemini review suggested.
**Files changed:**
- `scripts/lib/model-pricing.tsv` — added claude-opus-5-5 pricing row
- `tests/model_pricing.bats` — added two regression tests with detailed boundary comments
**Skipped (informational):** 0
No action is required on the PR itself. The `template-drift` check failure is a known infrastructure issue being tracked separately and is not caused by these changes.

@don-petry
don-petry enabled auto-merge (squash) September 23, 2026 03:49
donpetry-bot
donpetry-bot previously approved these changes Sep 23, 2026

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: LOW
Reviewed commit: 2528764fdff726ee660139ca20ae2faf9737b40e
Review mode: triage-approved (single reviewer)

Summary

Adds a date-versioned claude-opus-5-5* pricing row (effective 2026-09-22: 4.00 / 0.20 / 5.00 / 20.00 USD/MTok) to scripts/lib/model-pricing.tsv and two boundary regression tests in tests/model_pricing.bats. Data-plus-tests only — no engine or workflow changes, zero runtime blast radius. Functionally verified by this reviewer against the head files: the row is properly TAB-separated, price_for claude-opus-5-5 returns 4.00 0.20 5.00 20.00 at and after the 2026-09-22 boundary, empty before it (no retroactive guess), and the claude-opus-4-* rows are unaffected. Confirms the triage tier's low-risk assessment.

Linked issue analysis

Linked issue #1896 is substantively addressed — all four acceptance criteria verified: (1) the exact TAB-separated row is appended in the Anthropic Claude block (raw-byte inspection confirms tabs, append-only convention preserved); (2) price_for claude-opus-5-5 returns the pinned rate — tests sample the exact 2026-09-22 effective_from boundary, stricter than the AC's 2026-09-23, adopted from review-bot feedback; (3) no collision with claude-opus-4-*: verified price_for claude-opus-4-8 2026-06-07 still returns 5.00 0.50 6.25 25.00 and the more-specific opus-4-1 legacy row still wins — existing tests lock both behaviors; (4) bats and shellcheck are green on the head SHA.

Findings

No blocking findings. Prior cycle-1 review at this same SHA (2528764) had verified the content and escalated solely over the red template-drift check; that escalation is superseded here per org policy #1795, which explicitly established that a failing NON-required check — naming template-drift — must not block review (the CI gate covers required checks only, with the failure named, never silently ignored). All 4 bot review threads (gemini, codeant, cubic ×2) are resolved — the dev-lead fix commit moved the tests to the exact effective_from boundary as requested. No unanswered human-reviewer questions; the maintainer's template-drift skip note is informational and its out-of-scope claim was independently verified (scripts/template_stub_drift.sh audits the external petry-projects/repo-template repo against the standards baseline — this PR's pricing/test diff cannot cause or cure that drift; remedy is a repo-template re-seed outside this PR). Secret scan: the run_secret_scanning MCP tool is not available in this environment; the gitleaks CI check passed and the diff contains only pricing numbers and test assertions.

CI status

All 5 REQUIRED checks green on head 2528764 (verified live via the rollup's isRequired): CodeQL, SonarCloud, agent-shield / AgentShield, dependency-audit / Detect ecosystems, duplicate-decl-gate. Non-required failure, named per #1795: template-drift = FAILURE — pre-existing external drift in petry-projects/repo-template, out of scope for this PR (see Findings). All other checks green: bats, unit-tests, shellcheck/ShellCheck/Lint, gitleaks, SonarCloud Quality Gate passed (0 new issues, 0 hotspots), holdout-guard, sync-scope-guard. CANCELLED entries are superseded duplicates from the prior commit, each with a SUCCESS/SKIPPED counterpart on head.


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

@donpetry-bot

Copy link
Copy Markdown
Contributor

pr-review approved on PARTIAL advisory evidence: 4/7 required advisory bots reported before the gate's head-age-timeout fallback proceeded. Recorded for the miss-rate metric (#1596).

@donpetry-bot
donpetry-bot dismissed their stale review September 23, 2026 03:55

Dismissing approval due to a PR issue comment lacking a verified disposition (#1813)

@don-petry
don-petry merged commit eee2898 into main Sep 23, 2026
74 of 83 checks passed
@don-petry
don-petry deleted the dev-lead/issue-1896-20260923-0307 branch September 23, 2026 03:55

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: LOW
Reviewed commit: 2528764fdff726ee660139ca20ae2faf9737b40e
Review mode: triage-approved (single reviewer)

Summary

Adds a date-versioned claude-opus-5-5 pricing row (effective 2026-09-22: 4.00 / 0.20 / 5.00 / 20.00 USD/MTok) to scripts/lib/model-pricing.tsv and locks it with two bats regression tests (exact effective_from boundary + no-retroactive-pricing). Pure data + test change — no engine, workflow, or shell-logic edits. Triage's low-risk assessment is confirmed.

Linked issue analysis

Closes #1896. All acceptance criteria substantively met: (AC-1) the TAB-separated row is appended in the Claude block, preserving the append-only effective-dated convention (no existing row mutated); (AC-2) a regression test pins price_for at the exact effective_from boundary 2026-09-22 — stronger than the AC's 2026-09-23 sample, per reviewer feedback — plus a 2026-09-21 test asserting empty output (no retroactive guess); (AC-3) the existing 'opus 4.7 at a 2026 date' test confirms the claude-opus-4-* glob still returns 5.00 0.50 6.25 25.00 (issue allows add/confirm; different major digit, no shadowing); (AC-4) bats and shellcheck are green in CI.

Findings

No blocking findings.

  • All 4 inline review threads (gemini-code-assist, codeant-ai, cubic-dev-ai ×2) are resolved; the dev-lead follow-up commit applied the requested boundary-date tightening.
  • Secret scan: the run_secret_scanning MCP tool is not available in this environment; the gitleaks CI check is green, and the diff contains only pricing data and test assertions.
  • Advisory-bot coverage was partial (4/7, head-age-timeout fallback per #1596) — recorded by the gate; no advisory bot reported an unresolved defect.

CI status

Green across the board (shellcheck, bats, unit-tests, CodeQL, SonarCloud quality gate, gitleaks, agent-shield, and all structural guards) with one exception: template-drift is FAILING. That check validates petry-projects/repo-template's stub workflows against the standards baseline; the drifted/missing files it names are not in this PR's diff, the failure predates this PR, and the repository owner explicitly documented it in a PR comment as out-of-scope infrastructure drift (remedy: re-seed via scripts/seed-repo-template.sh), deliberately leaving it red. Not caused by, nor fixable within, this change.


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Phase 1] Price claude-opus-5-5 in model-pricing.tsv and lock it with a regression test

2 participants