Skip to content

test(code-tidying): name audit-dead-code in adc-read-only-delete-request - #13204

Merged
kyle-sexton merged 2 commits into
mainfrom
host-run/12043-survey-rest
Nov 4, 2026
Merged

kyle-sexton merged 2 commits into
mainfrom
host-run/12043-survey-rest

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Refs: #12043

Summary

code-tidying adc-read-only-delete-request fired its skill in 0 of 1 runs on dispatched run 41810568160 (deterministic-only, runs 1, Claude Code 2.1.318), so its with-arm delta measured nothing. The plain prompt ("Find the dead code in this repository and delete it for me") scored 0.50 on both arms.

Fix

Triggering finding, reported on #12043 and not fixed here: a plain "find the dead code and delete it" request misses /code-tidying:audit-dead-code.

Verification

  • validate-cases.py plugins/code-tidying/evals: 0 FAIL. The only WARNs on this case are the ones it already had (case-level Bash, Edit and Write grants).
  • scripts/check-affected.sh --tests on the committed head: 14 passed, 3 skipped over 2 changed files.
  • Not run: a dispatched eval of the changed prompt. eval-browser-dispatch.yml runs only on main, so the re-dispatch comes after merge.

Remaining on #12043

Related

🤖 Generated with Claude Code

Dispatched run 41810568160 read the case as fired in 0 of 1 runs. The
prompt now names the skill, so the case tests read-only behavior once
the skill is loaded, not triggering.

Refs: #12043

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-04T11:56:33.917187Z f9b673a PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed fcd0aa85da319c4a37d22f2b2f60db3879eb22c1: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Security review (incremental, blocking only): no security issues found

This review covers only what changed since f9b673a7. One file changed: plugins/code-tidying/evals/adc-read-only-delete-request/case.yaml, and I read its whole change.

The change. It removes the natural-language tag from the eval case's tags list (line 4). Nothing else changed: the prompt, allowed_tools, max_turns, timeout_seconds and scaffold are all as before. This is eval metadata only. It touches no trust control, workflow, token scope, kill switch, merge hold, plugin manifest or standing instruction, and it matches no trusted-path glob.

Prior review. This lane's earlier review of this pull request reported no findings, so there is nothing to restate. This review was blocking only: it would have reported only CRITICAL or IMPORTANT findings, and it found none.

PR context. The context file has no embedded text aimed at this lane.

Spend. Re-dispatching the eval after merge uses prepaid model credit only. That is not a hold.

critical-findings: 0

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on f9b673a751fe8165c996cc160a113d6e2c0ad831; the summary is in this lane's pull request comment.

claude-lane: claude-review

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed fcd0aa85da319c4a37d22f2b2f60db3879eb22c1: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Code review: PR #13204 (incremental, blocking only)

Scope. This was an incremental review of the push since f9b673a7. I read the one changed hunk in plugins/code-tidying/evals/adc-read-only-delete-request/case.yaml. The prior-findings file says this lane already reviewed an earlier head, so I posted only blocking findings.

Findings: 0

  • The push removes natural-language from tags at case.yaml:4. It now reads tags: [audit-dead-code, read-only], which is exactly what my earlier advisory comment (r4467125393) suggested. That finding is resolved: a natural-language survey that selects cases by tag will no longer pick up this case, whose prompt now names the skill.
  • The other natural-language code-tidying cases still carry the tag and are unchanged, so tag-based survey runs still have natural-language triggering cases to run.

Other checks

  • The PR closes no issue (it says Refs: #12043), so there are no acceptance criteria to check against.
  • No plugin manifest changes and no settled threads.
  • The author's verification (validate-cases.py 0 FAIL, check-affected.sh) is their claim and covers the earlier head. I ran nothing.

Spend. None directly. The re-dispatch planned after merge uses eval-run subscription or API usage, which is not a hold.

…te-request

The prompt now names the skill, so a natural-language triggering survey
must not select this case and count it as a trigger pass.

Refs: #12043

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit cc654d3 Nov 4, 2026
43 checks passed
@kyle-sexton
kyle-sexton deleted the host-run/12043-survey-rest branch November 4, 2026 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant