Skip to content

ci: exempt the developer-experience labeled eval samples from typos and markdownlint - #10723

Merged
kyle-sexton merged 1 commit into
mainfrom
host-run/9936-lint-exclusions
Oct 28, 2026
Merged

kyle-sexton merged 1 commit into
mainfrom
host-run/9936-lint-exclusions

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

Refs: #9936

Summary

Exempts the developer-experience labeled eval samples (plugins/developer-experience/evals/sources/labels/**) from typos and markdownlint, ahead of the PR that commits those samples for #9936.

Fix

  • The CI typos step and scripts/check-affected.sh exclude plugins/developer-experience/evals/sources/labels/**. This is the same exemption the testing plugin's judge-calibration cases have.
  • A nested labels/.markdownlint-cli2.jsonc ignores the directory, as plugins/session-flow/scripts/tests/fixtures/ does.
  • A bump: none developer-experience changelog fragment records the change.

The samples are judge inputs captured from model runs, so correcting their spelling or list spacing would change what the judge is measured on. The exemption has to land first because the pre-push check-affected reads the merge-base copy's configs and typos exclusions, not the branch's. Until this merges, the samples branch fails at pre-push on markdown and typos.

Verification

  • scripts/check-affected-gates.test.sh: PASS=13 FAIL=0 (the runner and workflow typos exclusions match).
  • scripts/check-affected.sh --tests: 13 passed, 3 skipped.
  • scripts/check-changelog-fragments.sh --check-required origin/main exits 0.
  • markdownlint-cli2 --config .markdownlint-cli2.jsonc on a label sample, with the nested config present, reports 0 issues.

Related

🤖 Generated with Claude Code

…nd markdownlint

The samples are judge inputs captured from model runs; correcting them would
change what the judge is measured on. The pre-push check reads the base
branch configs, so the exemption lands before the samples do.

Refs: #9936

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 28, 2026 19:25
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T19:28:34.179973Z 7258717 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 7258717bc2d0d025b6745daa99a28d8ca3111a54: 0 finding(s), 0 as inline comments.

No security issues found.

critical-findings: 0

This pull request exempts plugins/developer-experience/evals/sources/labels/** from the spell check and markdown lint. I read all four changed files and the context around the one trusted-path file, .github/workflows/pr-require-checks.yml.

  • pr-require-checks.yml:921: the new glob goes only into the exclude: input of the Spell-check (typos) step. That step has continue-on-error: true and is not a merge hold. The Scan for secrets step (gitleaks, line 930) takes no exclude, uses .gitleaks.toml, and scans the commit history, so secret scanning still covers the exempted directory. No token scope, trust check, kill switch or merge hold changes.
  • scripts/check-affected.sh:397: the same typos exclusion for the local pre-push runner. It is a lint-only change.
  • labels/.markdownlint-cli2.jsonc: the nested config ignores ** only inside the labels/ directory. It affects markdown lint only.
  • .changes/...md: a bump: none changelog fragment with no executable effect.
  • Manifests: the diff touches no plugin.json or marketplace.json, so there are no manifest <field> lines.
  • Instruction surface: nothing is deleted or softened.
  • Spend: no spend or billing change.

The PR description contains no embedded directives.

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed 7258717bc2d0d025b6745daa99a28d8ca3111a54: 0 finding(s), 0 as inline comments.

No findings. I read the diff and PR context, and checked the new lint exclusions against their precedents in the repository.

  • Typos exclusion: the same glob, plugins/developer-experience/evals/sources/labels/**, is added to the exclude: list in pr-require-checks.yml and to the typos_check flags in scripts/check-affected.sh. Both follow the existing judge-calibration/cases/** entries. The PR says check-affected-gates.test.sh checks that the two lists match. I did not run it, so that is unverified here.
  • Markdownlint exclusion: the new labels/.markdownlint-cli2.jsonc ("ignores": ["**"]) has the same shape as the existing plugins/session-flow/scripts/tests/fixtures/.markdownlint-cli2.jsonc.
  • Changelog fragment: bump: none fits a change to lint configuration only, with no shipped skill or script changes.
  • Trusted path: the change to .github/workflows/pr-require-checks.yml adds one path to a lint-exclusion list. It touches no gating, permissions, secrets or triggers, so it raises no security concern.
  • Closing issue: the PR says "Refs chore(developer-experience): commit the hand-rolled eval scenario sources #9936" and has no Closing issue section, so there are no acceptance criteria to check.
  • Not run: the markdownlint and typos behaviour is author-claimed. I did not run either tool.
  • Cost and manifests: the change spends no billed resource, and it touches no plugin manifest.

@kyle-sexton
kyle-sexton enabled auto-merge October 28, 2026 19:29
@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit 9a55a7b Oct 28, 2026
84 checks passed
@kyle-sexton
kyle-sexton deleted the host-run/9936-lint-exclusions branch October 28, 2026 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant