Skip to content

fix(verification): neutralize .NET/PowerShell-flavored examples in agnostic skills - #492

Merged
kyle-sexton merged 2 commits into
mainfrom
fix/424-verification-agnostic-examples
Jul 19, 2026
Merged

fix(verification): neutralize .NET/PowerShell-flavored examples in agnostic skills#492
kyle-sexton merged 2 commits into
mainfrom
fix/424-verification-agnostic-examples

Conversation

@kyle-sexton

Copy link
Copy Markdown
Contributor

Summary

The verification plugin's confirm and measure skills are sold as ecosystem-agnostic, but their illustrative examples leaned .NET/C#/PowerShell, and two "how to check" cells assumed a Unix shell (wc -l) against the cross-platform "never assume Bash" contract. A non-.NET consumer was handed a stack-specific illustration as the universal path. This patch neutralizes the examples while keeping the guidance concrete and useful.

Fix

  • skills/confirm/context/fix.md — reproduction-test example now uses a generic descriptive name (returns_null_when_user_is_deleted, "name it per your framework's convention") and a generic failure ("the reported failure (e.g. a null-dereference crash)") instead of UserService_ShouldReturnNull_WhenUserIsDeleted / NullReferenceException.
  • skills/measure/context/metrics.md — coupling row counts "import/dependency declarations (import/require/using, package or project references)" instead of using/ProjectReference; coverage row points to "your test runner's output" instead of dotnet test; both wc -l sites are now stated shell-neutrally (wc -l on POSIX/Git Bash, Measure-Object -Line in PowerShell), mirroring the both-forms pattern established in fix: state shell requirements with a Windows path across 8 plugins #331.
  • skills/measure/context/performance.md — metric "how to measure" cells point to "your test runner", "your build tool", "your platform's profiler", and "your benchmark harness", with the shell timer given for both shells (time on POSIX/Git Bash, Measure-Command in PowerShell) instead of dotnet test/dotnet-counters/BenchmarkDotNet as the only path.
  • The named marketplace-plugin evidence pointers (dotnet-*:*) at the bottom of both measure files are unchanged — those are real named plugins used as evidence sources, not universal-path illustrations, and verification: .NET/C#/PowerShell-flavored examples and Unix-shell assumptions in stack-agnostic skills (low severity) #424 deliberately did not flag them.
  • verification bumped 0.2.20.2.3 (docs = patch) with a CHANGELOG entry.

Verification

Gates run in this Windows environment against the changed markdown; all clean, nothing suppressed:

  • markdownlint-cli2 v0.23.0 (markdownlint v0.41.0) (repo-pinned) on the four changed .md files — Summary: 0 error(s).
  • typos-cli 1.44.0 on the four changed .md files — exit 0, no output.
  • editorconfig-checker v3.8.0 on the changed .md files plus plugin.json — exit 0, no findings.
  • plugin.json re-parsed as valid JSON; version reads 0.2.3.

Related

Closes #424. Sibling stack-bias issues in the same work-readiness sweep, same pattern: #430, #429, #428 (all testing plugin).

🤖 Generated with Claude Code

…nostic skills

The confirm/measure skills are sold as ecosystem-agnostic, but their
illustrative examples leaned .NET/C#/PowerShell and one assumed a Unix
shell (`wc -l`) against the cross-platform "never assume Bash" contract.
Neutralize them so a non-.NET consumer isn't handed a stack-specific
illustration as the universal path: generic reproduction-test name and
failure, import/dependency-declaration counting instead of
`using`/`ProjectReference`, "your test runner"/"your benchmark harness"
instead of `dotnet test`/BenchmarkDotNet, and shell-neutral line counting
(`wc -l` on POSIX/Git Bash, `Measure-Object -Line` in PowerShell). The
named marketplace-plugin evidence pointers (`dotnet-*`) are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011V31qpAHP3jfs76B9d5Rfo
@claude

claude Bot commented Jul 19, 2026

Copy link
Copy Markdown

Claude finished @kyle-sexton's task in 1m 49s —— View job


PR Review — fix(verification): neutralize .NET/PowerShell-flavored examples in agnostic skills

  • Gather context
  • Read changed files
  • Review for correctness, alignment, and quality
  • Post findings

Verdict: Approve — clean, focused, well-reasoned patch. The changes are correct and do exactly what the PR description says they do. No blocking issues.


Summary

This is a documentation-only patch (5 files, 26 additions / 11 deletions) that neutralizes .NET/C#/PowerShell-flavored illustrative examples in the verification plugin's confirm and measure skills. I checked each changed file against the REVIEW.md blocking criteria — none of the security/authorization/tenancy/atomicity gates apply here. Below are observations from the content review.


✅ What's done well

fix.md — test name neutralization (line 52)
Replacing UserService_ShouldReturnNull_WhenUserIsDeleted (xUnit/MSTest PascalCase convention) with returns_null_when_user_is_deleted (name it per your framework's convention) is the right move. The parenthetical is a good way to acknowledge that snake_case itself is just one convention (Python/Ruby/Rust), without leaving the example unlabeled.

Dual-shell pattern is applied consistently (metrics.md L14, performance.md L14)
Both wc -l / Measure-Object -Line and time / Measure-Command pairs follow the pattern established in #331. They appear in every location where a shell command was previously implied as Unix-only.

Marketplace plugin evidence pointers left intact (metrics.md L69–73, performance.md L61–66)
The dotnet-*:* plugin references at the end of both files are real named marketplace plugin pointers, not illustrative examples. Leaving them untouched is the correct decision.

Version and CHANGELOG are consistent
Patch bump (0.2.20.2.3) is correct for a docs-only change. The CHANGELOG entry accurately describes what changed and explicitly calls out what did not change (the dotnet-* plugin pointers).


🟡 Nit — parenthetical inside code fence

fix.md line 52:

Reproduction test: returns_null_when_user_is_deleted (name it per your framework's convention)

Embedding (name it per your framework's convention) inside the code fence is slightly unusual — code fences typically show actual output or commands, not inline meta-commentary. A first-time reader might initially read the parenthetical as part of the test name. The surrounding prose context makes the intent clear enough, but a comment-style delimiter (e.g. # ... or a prose note before the fence) would be unambiguous. Not worth a change on its own, but worth noting.


🟡 Nit — table cell verbosity in metrics.md

metrics.md line 14:

Line counter (wc -l on POSIX/Git Bash, Measure-Object -Line in PowerShell), git diff --stat

The "How to check" cell is now quite long. These files are AI-readable instruction documents rather than rendered markdown for end users, so this is low impact. Still, if a future edit passes through, splitting the dual-shell note to a separate prose sentence below the table would improve scannability.


No concerns on REVIEW.md blocking criteria

All six blocking checks (object-level auth, tenant-scoped data access, secrets, injection, audit logging, atomicity) are inapplicable — this is a documentation-only change with no code paths, data access, or external input handling.

…t name

Address claude[bot] review nit: the "(name it per your framework's
convention)" parenthetical inside the code fence could be misread as part
of the test name. Move it to a trailing `#` comment delimiter so the
meta-note is unambiguously commentary, not the identifier.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude

claude Bot commented Jul 19, 2026

Copy link
Copy Markdown

Claude finished @kyle-sexton's task in 1m 26s —— View job


PR Review — fix(verification): neutralize .NET/PowerShell-flavored examples in agnostic skills

  • Gather context
  • Read changed files
  • Review for correctness, alignment, and quality
  • Post findings

Verdict: Approve — the follow-up commit cleanly resolves the one nit from the prior review, and the full patch is well-scoped and correct.


What changed since the last review

Commit 8c06888 addresses the nit about the parenthetical meta-note inside the code fence:

-Reproduction test: returns_null_when_user_is_deleted (name it per your framework's convention)
+Reproduction test: returns_null_when_user_is_deleted  # name it per your framework's convention

The # delimiter is the right call. A bare parenthetical inside an unlanguaged code fence could be read as part of the identifier; a # reads unambiguously as a comment in the majority of languages that use snake_case test names (Python, Ruby, Rust). The fix is minimal and targeted.


Full patch — residual observations

CHANGELOG date field (CHANGELOG.md L6)
The Keep a Changelog spec encourages ## [0.2.3] - YYYY-MM-DD, but all prior entries in this file omit the date too (## [0.2.2], ## [0.2.1], etc.). The new entry is consistent with the established pattern — no action needed.

git show <base>:<file> pipe in metrics.md (L24)
The prose now reads: "line count from git show <base>:<file> piped to a line counter — wc -l on POSIX/Git Bash, Measure-Object -Line in PowerShell". This is accurate — git show is cross-platform but the pipe operator has a different form in PowerShell (| vs. |... actually the same). The wording is correct; the split is only in the downstream command. Minor but fine.

REVIEW.md blocking criteria — all six inapplicable to a docs-only patch: no object-level authorization, no tenant-scoped data, no secrets, no injection surface, no audit-log requirements, no multi-file atomic write.


Summary

Both commits are clean. The original neutralization was well-reasoned; the follow-up precisely resolves the single nit. Ready to merge.

@kyle-sexton

Copy link
Copy Markdown
Contributor Author

Thanks for the review — verdict noted as Approve. Classification of the two 🟡 nits:

# Finding Classification Evidence Reacted
1 fix.md L52 — (name it per your framework's convention) parenthetical inside the code fence could be misread as part of the test name VALID — fixed Moved the meta-note to a trailing # comment delimiter (returns_null_when_user_is_deleted # name it per your framework's convention) so it reads unambiguously as commentary, not the identifier. Fixed in 8c06888. 👍
2 metrics.md L14 — "How to check" cell verbosity from the dual-shell note ACKNOWLEDGED (no action) The finding is scoped to "if a future edit passes through" and is low-impact by your own read; these are AI-readable instruction docs, not rendered end-user markdown. Splitting the dual-shell note out of the table cell would reduce the table's self-containedness, so leaving it. No behavior/clarity regression. 👍

Head is now 8c06888; all checks green, merge state CLEAN. Not merging (merge gate on).

@kyle-sexton
kyle-sexton merged commit 2277c98 into main Jul 19, 2026
15 checks passed
@kyle-sexton
kyle-sexton deleted the fix/424-verification-agnostic-examples branch July 19, 2026 17:03
kyle-sexton added a commit that referenced this pull request Jul 20, 2026
…ls (#531) (#609)

## Summary

Skills declared ecosystem/forge/tracker-agnostic ship bare hardcoded
stack/forge/branch/tracker defaults because agnosticism was asserted in
prose and
never enforced mechanically — the dominant review-churn class, re-caught
by the
external reviewer PR after PR (branch/remote hardcodes, seam-conditional
eval,
.NET-flavored examples). This adds a fail-closed CI lane that
mechanically detects
that coupling instead of paying for it at review time again and again.

Detection only — this PR builds the gate. Fixing the existing violations
stays
with the member issues below.

## Fix

A new `portability-lint` lane (`.github/workflows/ci.yml`) wired into
the required
`ci-status` aggregate, plus a repo-local detector:

- **`scripts/check-skill-portability.sh`** — scans skill files for
coupling tokens.
On a PR it scans only the skill files the change **touches** (mirroring
the
`skill-quality-gate` changed-diff pattern), so enabling a token class
prevents
**new** coupling without red-lining pre-existing violations — main's
push event
scans nothing (self-test is the push path), and existing hits wait for
their
  owning follow-up fix or the file's next edit.
- **`scripts/skill-portability-tokens.txt`** — the token list as
external,
extensible **data** (not logic buried in bash), so a reviewer re-catch
is a
one-line data edit. Seeded with **one active class** — the
branch/default-branch
hardcode (`origin/main` / `origin/master`) — per the ratified
one-token-class-at-
a-time rollout; further classes (`dotnet`/`Clean Arch`,
`raw.githubusercontent`,
bare `gh` tracker calls) are staged as commented entries with
enable-triggers.
- **Never-skip shape**: the job is unconditional; only the PR-diff step
is
event-gated, and a self-test step runs first so a broken detector can
never mask
  a real violation behind a green gate.

**Design decisions** (resolved per the ratified plan, not invented):

- *How a skill declares agnosticism scope* — **no new frontmatter
field**. A skill
is agnostic by default (the Design boundary already binds every plugin),
so the
gated set is the files a change touches. A hit is excused three ways,
all
reviewer-visible comments (reusing the silent-skip gate's
annotated-exemption
shape): an auto-recognized detection-first / presence-gated use; a
per-site
`portability-ok: <reason>`; or a whole-file `portability-scope:
<reason>`
declaring an inherent narrower boundary (the
forge-locked-under-a-neutral-name
case). This distinguishes **guarded** refs (fine) from **bare** ones, as
the
  guarded-forward-ref disposition requires.
- `docs/PLUGIN-PHILOSOPHY.md` gains one doctrine sentence extending the
existing
  declared-narrower-boundary allowance from OS platform to the
  forge/ecosystem/tracker axis.

No `plugins/<name>/` directory is touched, so no version bump /
CHANGELOG entry is
needed (matching the CI-gate precedent).

## Verification

- **Self-test suite** (`scripts/check-skill-portability.test.sh`, runs
in CI):
`PASS=12 FAIL=0` — covers bare-token FAIL with file:line,
detection-ladder pass,
same-line and comment-block-above `portability-ok`, annotation non-leak
past
intervening code, whole-file `portability-scope`, staged tokens staying
inactive
under the shipped list, fail-closed exit 2 on missing token list /
invalid base
  ref, the vendor/evals/`*.test.sh` exclusion set, and empty scope.

- **Catches a real violation, passes a real legit use** (the
bare-vs-guarded
  discrimination that drives this class), on live corpus files:

  ```
$ scripts/check-skill-portability.sh --paths
plugins/work-items/skills/track/actions/start.md
COUPLING: plugins/work-items/skills/track/actions/start.md:61:
origin/(main|master) -> ... git checkout -b <type>/<N>-<slug>
origin/main ...
COUPLING: plugins/work-items/skills/track/actions/start.md:65:
origin/(main|master) -> ... git checkout -b <type>/<N>-<slug>
origin/main ...
  (exit 1)

$ scripts/check-skill-portability.sh --paths
plugins/review/skills/fanout/SKILL.md
No unexcused coupling tokens in 1 skill file(s). # detection-first
origin/HEAD ladder → guarded
  (exit 0)
  ```

- **Calibrated against the full corpus** (`--all`): the active branch
class flags
exactly 3 genuine bare hardcodes in 2 files (owned by member issues;
changed-file
scoping keeps them off main) and correctly passes the detection-ladder
uses in
  `review/*` and excludes evals/test fixtures.

- **Changed-file CI path proven** against a real base ref (16 changed
skills gated,
all clean). Local checks green: `shellcheck` (repo `.shellcheckrc`),
`shfmt`,
  `actionlint`, comment-residue detector (T1=T2=T3=0), `typos`.

Closes #620

Part of #531 (umbrella stays open — stages remain; child #620 carries
this PR's shipped scope per the #603 pattern).

## Related

- Member coupling instances (this is prevention; it does not block
them): #404
#405 #406 #408 #410 #412 #415 #416 #418 #421 #422 #423 #428 #429 #432
#438 #439
  #441 #442
- #445 — sibling CI-gate backlog (mechanical conformance only;
explicitly not
  coupling)
- #412 — guarded-forward-ref disposition the gate honors (guarded ≠
bare)
- #441 — declared narrower-scope exemption the `portability-scope`
mechanism serves
- #467 — the branch/remote review-churn the seed active class targets
- #453 — seam-conditional eval re-catch of this class
- #491 #492 — .NET-flavored-example hand-fixes a future staged class
would obviate
- #611 — post-green review finding (annotation-carry pending_annot
doesn't distinguish inline vs block HTML comments), deferred out of this
PR's scope

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01KyLj6oaFVTE6xFuoYCC2KC

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
kyle-sexton added a commit that referenced this pull request Jul 30, 2026
…s (stage 3) (#1790)

Stage 3 of #531's portability-lint lane: the **stack/ecosystem opinion
class** — baked stack
defaults in skills that claim to work in any repository.

The operator ratification recorded on this issue fixed the shape: staged
per-token rollout, no bulk
activation, and per-token activation only after (1) a POSIX-safe
rewrite, (2) a green `--all` audit,
and (3) a class-scoped guard marker. This lands all three, plus the
fixtures the acceptance criteria
ask for.

## The staged patterns were inert, not merely dormant

`\bdotnet\b` and `\bClean Arch\b` could not have worked. POSIX ERE
defines no word-boundary escape,
and on gawk as well as mawk/nawk a backslash-b matches a literal
**backspace byte** — so those
patterns matched nothing. Uncommenting either would have turned the
class green while missing every
violation, which is the worst outcome a gate has.

Both are **replaced** with POSIX-safe spellings rather than uncommented,
and a fixture now fails if
a backslash-b ever reaches an ACTIVE token again.

## One token activates, on measured evidence

**`Clean Arch(itecture)?` is now ACTIVE.** It measures **0 hits / 0
files** through the gate, and a
raw grep finds the phrase nowhere in the scanned corpus. It carries no
migration debt whatsoever —
the one member of this class that can be locked in *before* it arrives,
which is exactly the
prevention #531 exists for ("prevention instead of hand-fixes plus
indefinite recurrence"). A skill
whose subject genuinely *is* that architecture declares
`portability-scope`, and a fixture pins that
escape so the active token cannot strand it.

## The ecosystem literals stay staged, with real numbers

Residue measured through the gate **itself** at `c7b0af78`, so guards,
per-site annotations, and
whole-file scope declarations are already applied to the counts:

| Pattern | Hits | Files |
|---|---|---|
| `[Dd]otnet` | 107 | 40 |
| `(^\|[^a-zA-Z0-9_])\.NET([^a-zA-Z0-9_]\|$)` | 63 | 37 |
| `\.csproj` | 26 | 17 |
| `(^\|[^a-zA-Z0-9_])C#([^a-zA-Z0-9_]\|$)` | 19 | 15 |
| `[Bb]lazor` | 13 | 5 |
| `\.sln([^a-zA-Z0-9_]\|$)` | 7 | 7 |
| `\.razor` | 1 | 1 |
| `\*\*/\*\.cs([^a-zA-Z0-9_]\|$)` | 1 | 1 |

Their shared blocker is now recorded in the token file rather than left
implicit: these are not
stray hardcodes, they are skills that legitimately need to **name** an
ecosystem and have no seam to
read one from. The fix is the `userConfig` seam work on #405 / #406 /
#421 / #428 — the lint is the
forcing function, not the fix — or a `portability-scope` declaration on
genuinely ecosystem-locked
files.

## The class-scoped guard

The class is excused by **consumer-seam evidence** co-located on the
line — never by the branch
class's evidence:

1. a `${user_config.*}` read (the seam this class exists to force),
anchored to the read syntax so
   prose *about* a seam does not count; or
2. a neutral `<placeholder>` **and** an explicit illustration lead
(`e.g.`, `for example`,
`such as`) — the portable idiom already used across this corpus, where
the ecosystem token
illustrates the placeholder rather than being the value the skill ships.

Both halves of (2) are required. A placeholder alone would excuse `run
dotnet build in <repo-root>`,
which ships the stack and parameterizes only the path; an illustration
lead alone would excuse
`e.g. run dotnet build`, the bare hardcode wearing a hedge — precisely
the generic hedge the token
file's staged-class preamble forbids as a guard.

Deliberately **not** a marker: "another ecosystem is named on the same
line". That multi-ecosystem
enumeration shape is the dominant residual in the corpus and reads as
correct, but a co-occurrence
test cannot distinguish it from a line that hardcodes one stack and
merely mentions another. It is
recorded in the token file as the shape the next activator must handle,
with `portability-ok` as its
escape.

## A bug the fixtures caught, worth calling out

Class membership is tested with `index()` on literal fragments,
**never** a regex match against the
pattern text. My first revision used `p ~ /[Dd]otnet/` — copying the
branch class's shape — which
asks for a `D`-or-`d` followed by `otnet` and therefore does **not**
match the pattern *string*
`[Dd]otnet`, whose `d` is followed by `]otnet`. It silently returned 0
for every member, leaving the
whole class unguarded while every flagging test still passed. It
surfaced only because the
honored-case fixtures were written, which is why they are part of the
deliverable rather than a
nice-to-have. `index()` removes the trap instead of re-solving the
escaping per pattern.

## Acceptance criteria

- **Same lane discipline as Stages 1–2** — patterns are data in
`skill-portability-tokens.txt`,
staged entries carry measured residue and a named blocker,
`assert_staged` pins every staged
pattern verbatim so an edit that stops catching its own defect fails
loudly.
- **Fixtures proving the class is flagged where undeclared and honored
where a `userConfig` seam
declares it** — both directions, plus prose-about-a-seam failing to
stand in for a seam, each half
of the placeholder guard failing alone, the per-site and whole-file
escapes, and class scoping
proven **both** ways (branch-resolution evidence does not excuse an
ecosystem hardcode; a seam
  read does not excuse a branch default).

## Verification

- `bash scripts/check-skill-portability.test.sh` — **PASS=65 FAIL=0**
(was 44; 21 added)
- `bash scripts/check-skill-portability.sh --all` — the only hit
repo-wide is one pre-existing
branch-class line in
`plugins/playbooks/skills/boris/reference/foundations.md`; the newly
active
Clean-Architecture token contributes **0**. CI is changed-file scoped,
so this never red-lines
  `main`.
- `bash scripts/check-shell-portability.sh --paths <the two changed
scripts>` — clean. The `\b`
assertion is written as the bracket expression `[\]b` so it does not
trip the sibling gate that
  agrees with it.
- `bash scripts/check-shell-portability.test.sh` — PASS=215 FAIL=0
(sibling gate unaffected)
- `shellcheck` on both changed scripts — clean
- `scripts/check-changelog-parity.sh --check-bump origin/main`,
`scripts/check-changed-skills.sh` —
pass. No plugin version bump: the diff is repo tooling under `scripts/`,
not a plugin.
- `typos scripts/` exit 0

## Related

- Fixes #714
- #531 — the umbrella lane this is Stage 3 of; Option A
(declaration-first, one token-class at a
  time) was ratified there
- #713 — Stage 2, whose declaration reader this class's
`portability-scope` exemption rides on
- #405, #406, #421, #428 — the `userConfig` seam members this class is
the forcing function for;
  advanced, not retired, by this change
- #491, #492 — the .NET-flavored hand-fixes that named this class

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<https://claude.ai/code/session_01RhS3T7ShwJgKTrvk2Mvd3C>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

verification: .NET/C#/PowerShell-flavored examples and Unix-shell assumptions in stack-agnostic skills (low severity)

1 participant