diff --git a/docs/conventions/detector-findings/CHANGELOG.md b/docs/conventions/detector-findings/CHANGELOG.md index de40e1f04..0cf9154b7 100644 --- a/docs/conventions/detector-findings/CHANGELOG.md +++ b/docs/conventions/detector-findings/CHANGELOG.md @@ -4,6 +4,17 @@ Notable changes to the detector-findings contract (SemVer). Changing a producer- the coexistence obligations, or an enforceability verdict is a major bump; additive guidance or a new adopter row is a minor bump; docs-only clarification is a patch. +## 2.5.1 — 2026-08-23 + +**Patch** — docs-only clarification of one adopter row's selection text. No producer-owned field's +rule moves, no coexistence obligation changes, and no enforceability verdict changes. + +- **`rule-negation-without-positive` now describes paragraph-scoped accumulation (#3195).** The + row had inherited 0.21.1's "the line must close its own sentence" gate, which withheld every + hard-wrapped prohibition. The detector now accumulates a soft-wrapped sentence before classifying + it; the crosswalk row states that, and that a finding is attributed to the first physical line of + the triggering sentence. The other eight shapes stay line-scoped. + ## 2.5.0 — 2026-08-23 **Minor under this contract's own rule** — a new adopter's rows are added; no producer-owned field's diff --git a/docs/conventions/detector-findings/README.md b/docs/conventions/detector-findings/README.md index 1476640ec..4cb34dc92 100644 --- a/docs/conventions/detector-findings/README.md +++ b/docs/conventions/detector-findings/README.md @@ -233,7 +233,7 @@ side. | ai-slop/audit/rule-stacked-hedging | Two stacked hedges in one phrase ("could potentially", "might possibly", and the catalog's list) on a prose line, outside the exempt contexts above. Mechanical phrase selection, no withholding verdict; counted declines. | A doubled hedge produces no wrong result and weakens no caller (CRITICAL fails). IMPORTANT fails: no stated rule, and redundant hedging has no nameable degradation trigger — the claim's uncertainty is stated either way. SUGGESTION holds: one hedge and two hedges both express the uncertainty; the finding is a redundancy preference. | SUGGESTION | No, remediated by `/ai-slop:audit fix` — choosing which hedge states the real uncertainty is a claim-strength judgment | | claude-config/audit-instructions/rule-coercive-emphasis | A forced-compliance emphasis marker (`CRITICAL:`, `IMPORTANT:`, `You MUST`, `MANDATORY`, `ALWAYS use`, `NEVER skip`) on an instruction line, **body-scoped**: frontmatter is fenced out, as is any body line quoting a `'trigger phrase'` that appears in the file's own `description`. Case-sensitive — the all-caps marker IS the signal (threshold: any occurrence; the fired marker travels in the `Finding` cell). Mechanical byte-sequence selection with no withholding verdict, so the fail-safe criterion is met by construction; the criteria row's carve-outs (emphasis guarding a destructive or security gate, a stated hard precondition, a document *about* the pattern) are the model lane's, and every fence decline is counted in `## Surfaces`. | CRITICAL fails every limb: instruction prose computes nothing, so no concrete input, caller, or subsequent otherwise-correct change is made to produce a wrong, unsafe, or absent result — what the emphasis changes is how likely a trigger is to fire, which is a distribution over behavior rather than a determined result. IMPORTANT's degradation-with-a-named-trigger limb then matches, and the trigger is nameable rather than hypothetical: the first session in which the over-emphasized instruction fires on input its actual condition does not cover, spending a tool or skill invocation the condition never warranted. Official guidance supplies the mechanism, not merely a preference — prompting written against undertriggering "may now overtrigger", so this is a stated upstream rule the surface violates, not a choice among phrasings that all work. SUGGESTION's catch-all is therefore never reached: the alternatives do not both work, one overtriggers. | IMPORTANT | No — contained to `Location`, but the repair is a **downgrade, never a deletion**: the directive survives verbatim and only its volume changes, and deciding the normal-conditional phrasing that preserves the instruction is a rewrite judgment. Verbatim survival is asserted apart from **sentence-initial capitalization forced by dropping a leading wrapper** (`…MUST resolve` → `Resolve`) — a change the official source's own worked example makes too (`use` → `Use`); any other wording change means the remediation overreached | | claude-config/audit-instructions/rule-blanket-tool-default | A blanket tool default ("default to using/running/calling", "if in doubt, use", "always use", "use even when") on an instruction line, under the identical body-scope fences and counted declines as the emphasis rule above. Mechanical phrase-list selection, case-folding, no withholding verdict. | The same walk, on the same mechanism and the same official source, which is why the two rules share a tier: a blanket default is the second arm of one defect — prompting written against undertriggering that no longer exists. CRITICAL fails identically (the phrasing determines no wrong result). IMPORTANT's degradation limb matches with the same nameable trigger, sharpened by the guide stating the consequence outright — "Instructions like 'If in doubt, use [tool]' will cause overtriggering" — so the cost is a named behavioral one, not a register preference, and SUGGESTION is never reached. | IMPORTANT | No — contained to `Location`, but the repair replaces the blanket with the targeted condition it stood in for; recovering that condition is judgment, and the instruction itself is kept, never deleted | -| docs-hygiene/audit-noise/rule-negation-without-positive | An **imperative** prohibition (`never`, `do not`, `don't`, `avoid`, `must not`, `should not`) opening the line, on a line that **closes its own sentence**, with no positive alternative stated in that sentence. Both scope gates are load-bearing and measured: without them the rule fired 1053 times on an 85-file sample (99% of all findings), with them 69 — descriptive prose, an already-paired mid-sentence cue, a hard-wrapped continuation and a table row are all out of scope. **Body-scoped**: `detect.sh` never leaves frontmatter, fenced code, exempt sections or opt-out-marked content, and the writer independently re-fences frontmatter and declines any body line quoting a `'trigger phrase'` that appears in the file's own `description` / `when_to_use`. Selection is per SENTENCE, on the backtick-unwrapped line, and case-folded (threshold: any occurrence; the fired prohibition travels in the `Finding` cell). | CRITICAL fails every limb: instruction prose computes nothing, so no concrete input, caller, or subsequent otherwise-correct change is made to produce a wrong, unsafe, or absent result — a prohibition without its positive leaves the target under-specified rather than determined wrong. IMPORTANT's **stated-rule** limb then matches directly, and does so without needing the degradation limb: `docs-hygiene:write-for-agents` "Prompt the positive" is a rule this fleet already adopted in writing — "Write what to do, not what to avoid" — so a bare prohibition is a surface violating a stated rule rather than one phrasing among several that all work. Official guidance supplies the same mechanism (*"Do not use markdown"* → *"Your response should be composed of smoothly flowing prose paragraphs"*). SUGGESTION's catch-all is therefore never reached. | IMPORTANT | No — contained to `Location`, but the repair rewrites to the positive target the prohibition implies, and recovering that target is a rewrite judgment: the constraint must survive while its framing changes. Matches the disposition of both `audit-instructions` rules for the same reason | +| docs-hygiene/audit-noise/rule-negation-without-positive | An **imperative** prohibition (`never`, `do not`, `don't`, `avoid`, `must not`, `should not`) opening the sentence, with no positive alternative stated in that sentence. Soft-wrapped sentences are accumulated across paragraph lines before classification; a finding is attributed to the first physical line of the triggering sentence. The imperative-only gate is load-bearing and measured: without it the rule fired 1053 times on an 85-file sample (99% of all findings) — descriptive prose, an already-paired mid-sentence cue, and a table row are all out of scope. **Body-scoped**: `detect.sh` never leaves frontmatter, fenced code, exempt sections or opt-out-marked content, and the writer independently re-fences frontmatter and declines any body line quoting a `'trigger phrase'` that appears in the file's own `description` / `when_to_use`. Selection is per SENTENCE, on the backtick-unwrapped accumulated paragraph, and case-folded (threshold: any occurrence; the fired prohibition travels in the `Finding` cell). The other eight shapes stay line-scoped. | CRITICAL fails every limb: instruction prose computes nothing, so no concrete input, caller, or subsequent otherwise-correct change is made to produce a wrong, unsafe, or absent result — a prohibition without its positive leaves the target under-specified rather than determined wrong. IMPORTANT's **stated-rule** limb then matches directly, and does so without needing the degradation limb: `docs-hygiene:write-for-agents` "Prompt the positive" is a rule this fleet already adopted in writing — "Write what to do, not what to avoid" — so a bare prohibition is a surface violating a stated rule rather than one phrasing among several that all work. Official guidance supplies the same mechanism (*"Do not use markdown"* → *"Your response should be composed of smoothly flowing prose paragraphs"*). SUGGESTION's catch-all is therefore never reached. | IMPORTANT | No — contained to `Location`, but the repair rewrites to the positive target the prohibition implies, and recovering that target is a rewrite judgment: the constraint must survive while its framing changes. Matches the disposition of both `audit-instructions` rules for the same reason | | docs-hygiene/audit-noise/rule-negation-hard-guardrail | **Non-emitting.** A prohibition whose sentence also carries a safety-critical marker (`secret`, `credential`, `token`, `password`, `api key`, `force-push`, `--force`, `rm -rf`, `destructive`, `irreversible`, `data loss`, `production`, `security`, `vulnerab`, `rewrite history`) — a hard guardrail whose constraint a positive form cannot carry, where the prohibition IS the correct shape. | **Boundary ground, not a tier test** — the contract's "Findings that never reach a relay". This disposition is argued from the Boundary because a tier test can only ever return a tier, and the claim here is that the candidate is not a defect at all: the sibling `write-for-agents` rule itself preserves the negation "when the positive form genuinely loses the constraint". Reaching for a tier test to justify the non-emission would look argued while arguing nothing. **Fail-safe direction:** the carve-out requires its marker to be PRESENT on the sentence, so absence of that evidence selects the emitting rule above — an unresolved judgment can never withhold. The same holds for the paired-positive boundary (`instead`, `rather than`, `prefer`, `in place of`, `in favour of`) and the worked-example boundary (a `->` / `→` demonstration): each requires positive evidence, and both were checked rather than only the one easier to argue. | *(never reaches the relay)* | n/a — reported to the human only, and suppressed from the human report too, so one candidate carries one disposition on every surface this producer emits to | The map is flat across the emitting rules on purpose — within each producer's set every emitting rule diff --git a/plugins/docs-hygiene/.claude-plugin/plugin.json b/plugins/docs-hygiene/.claude-plugin/plugin.json index 20f3af4c2..73238843b 100644 --- a/plugins/docs-hygiene/.claude-plugin/plugin.json +++ b/plugins/docs-hygiene/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json", "name": "docs-hygiene", - "version": "0.21.6", + "version": "0.21.7", "description": "Documentation-hygiene toolkit: compress (flavor-trim markdown with a semantic-diff safety net), audit-noise (classify markdown noise), extract-ssot (deduplicate repeated content into a single source of truth), audit-encapsulation (detect citations into skill-private surfaces), rename-references (sweep stale references after renames), audit-derivability (classify whether a whole document earns its existence — could a fresh agent re-derive it from the code?), audit-progressive-disclosure (grade instruction files against a load-tier model for split opportunities and hub/spoke disclosure defects), write-for-agents (authoring-time doctrine that fires while agent-consumed markdown is being written), and write-for-humans (the same moment for the other reader — end-user READMEs, RFCs, release notes and guides — resolving the consuming project's own style guide first).", "author": { "name": "Melodic Software", diff --git a/plugins/docs-hygiene/CHANGELOG.md b/plugins/docs-hygiene/CHANGELOG.md index 65e9a813b..5904e0001 100644 --- a/plugins/docs-hygiene/CHANGELOG.md +++ b/plugins/docs-hygiene/CHANGELOG.md @@ -1,5 +1,36 @@ # Changelog — docs-hygiene plugin +## [0.21.7] + +### Fixed + +- **`audit-noise`'s `negation` shape now accumulates a soft-wrapped sentence + before classifying it (#3195).** 0.21.1's "the line must close its own + sentence" gate withheld every hard-wrapped prohibition: `Do not use markdown` + on one line with `in the summary body.` on the next never reached a verdict, + even though no positive is paired anywhere in that sentence. That is a silent + withhold — the one failure mode the detector-findings admission test asks this + rule set to avoid — and in a hard-wrapped repo it takes every prohibition long + enough to wrap. + + `detect.sh` now joins paragraph lines before the `negation` classifier runs. + A wrapped sentence whose positive alternative sits on the continuation is not + flagged; a wrapped sentence with no positive is flagged and attributed to the + first physical line of that sentence (where the cue opens), so the fix action + lands on the instruction's start rather than its wrap continuation. Every + qualifying sentence in the paragraph is a finding (a later imperative is not + dropped after the first). Sibling list items are separate blocks, so a later + item cannot pair an earlier prohibition. Attribution offsets are taken on the + unwrapped join so an earlier inline-code span cannot pull the line number + forward. A search cursor walks the join so two identical sentences do not + both pin to the first occurrence's line. Findings print in line-number order + after the paragraph flush. The + other eight shapes stay line-scoped. Frontmatter, fenced code, exempt + sections and opt-out markers still bound the accumulation. + + The "known limitation" bullet in `SKILL.md` is removed: the limitation is + gone, and the scope-gate bullet now describes the accumulation. + ## [0.21.6] ### Fixed diff --git a/plugins/docs-hygiene/skills/audit-noise/SKILL.md b/plugins/docs-hygiene/skills/audit-noise/SKILL.md index 82897f1ae..6849f2394 100644 --- a/plugins/docs-hygiene/skills/audit-noise/SKILL.md +++ b/plugins/docs-hygiene/skills/audit-noise/SKILL.md @@ -58,7 +58,7 @@ Only a page that passes admission proceeds to the nine in-page NOISE shapes belo | `plan-reference` — plan/changeset narration | Prose pointing at the work that produced the page instead of the page's subject: `replaces the old …`, `in this PR we …`, `Task 2 of the plan` | 1 | Delete the plan/changeset frame and keep whatever the sentence asserts about the present subject, rewritten without it. A doc citing a plan artifact that still exists is a live cross-reference, not this shape — matching requires a first-person actor behind `in this PR`, so `the files changed in this PR` is not flagged | | `conversational-antecedent` — asides to the requester | Prose addressed to the person who asked for the page or to the conversation that produced it: `As you asked, …`, `As requested, …`, `Per our discussion, …`, `per your request`, `like you said` | 1 | Delete the address — the conversation is invisible to every future reader, and the assertion behind it survives verbatim once the clause is cut. Two follower classes stand the shape down, because both name something a future reader can still open: an anaphoric adverb (`as we discussed above`), and `in` / `under` / `at` / `on` ahead of a **document locator** — a `§` or `#anchor`, a section/chapter/step/table, a link or path, or a named durable document (`as we decided in §3`, `in the ADR`, `on the ADR's recommendation`). Those prepositions ahead of anything else are matched, so `as we discussed in yesterday's meeting`, `as we decided in favor of X`, `as we agreed on Tuesday`, and `as we decided at the standup` are residue; tracker nouns are deliberately not locators, since `decided in issue 88` is provenance that `ticket-pr-residue` owns. The actor-less `as requested` matches only as a clause-final adverbial, so the attribution `as requested by the client` is not matched | | `ticket-pr-residue` — tracker/PR back-references | Bare provenance offered as the reason the prose says what it says: `See PR #45 for the rationale`, `Tracked in JIRA-123`, `decided in issue 88`, `from the feature branch` | 2 | Review — delete a bare provenance reference, or relocate it to the `## Sources` / `## History` footer (already an exempt section, so a relocated reference stops flagging). **Carve-out:** a markdown task-list item (`- [ ] … #123`, `- [x] … #123`) and a `TODO(#123)`-family marker are never flagged — both denote OUTSTANDING tracked work, where the reference is the actionable part of the line, which is the markdown restatement of the sibling's sanctioned-`TODO` exception. Nothing else is carved out: an inline parenthetical (`… (tracked in #482)`) stays Tier 2 so a reviewer rules on it rather than the scanner | -| `negation` — imperative prohibition with no positive alternative | An **imperative** prohibition (`never`, `do not`, `don't`, `avoid`, `must not`, `should not`) opening the line, on a line that **closes its own sentence**, with no positive alternative stated in that sentence ("Do not use markdown."). Descriptive prose ("Older versions do not support this flag"), a mid-sentence cue (already-paired "Prefer X; never Y"), a hard-wrapped continuation, and a table row are all out of scope | 2 | Rewrite to the positive target the prohibition implies (*"Do not use markdown"* → *"Compose your response as smoothly flowing prose paragraphs"*). Keep a negation only where the positive form genuinely loses the constraint, and then pair it with the positive in the same sentence. **Never a deletion** — the constraint survives; only its framing changes. The write-side rule this completes is [`/docs-hygiene:write-for-agents`](../write-for-agents/SKILL.md) "Prompt the positive". **A hard guardrail that cannot be phrased positively is not a finding** and is never flagged (carve-outs in Hard rules) | +| `negation` — imperative prohibition with no positive alternative | An **imperative** prohibition (`never`, `do not`, `don't`, `avoid`, `must not`, `should not`) opening the sentence, with no positive alternative stated in that sentence ("Do not use markdown."). Soft-wrapped sentences are accumulated across paragraph lines before classification. Descriptive prose ("Older versions do not support this flag"), a mid-sentence cue (already-paired "Prefer X; never Y"), and a table row are out of scope | 2 | Rewrite to the positive target the prohibition implies (*"Do not use markdown"* → *"Compose your response as smoothly flowing prose paragraphs"*). Keep a negation only where the positive form genuinely loses the constraint, and then pair it with the positive in the same sentence. **Never a deletion** — the constraint survives; only its framing changes. The write-side rule this completes is [`/docs-hygiene:write-for-agents`](../write-for-agents/SKILL.md) "Prompt the positive". **A hard guardrail that cannot be phrased positively is not a finding** and is never flagged (carve-outs in Hard rules) | Consumers with their own ephemeral-path or noise conventions can refine these defaults in their repo's `CLAUDE.md` / rules; the classifier's shapes and tiers above are the skill's built-in baseline. @@ -127,27 +127,19 @@ Shared clean-tree / no-scope shape: [`../../context/clean-tree-fallback.md`](../ marker matches as a **whole word**: a bare substring would let a longer word (`secretary` for `secret`, `preferentially` for `prefer`) satisfy a *withholding* boundary and lose a real finding silently. -- **`negation` selects only an IMPERATIVE that closes its own sentence.** Two scope gates, both - measured rather than argued: without them the shape fired **1053 times on an 85-file sample** of - this repo (12.4 per file, 99% of all findings); with them, **69**. (1) The cue must open the +- **`negation` selects only an IMPERATIVE, and classifies the accumulated sentence.** Two scope + gates, both measured rather than argued: without them the shape fired **1053 times on an 85-file + sample** of this repo (12.4 per file, 99% of all findings). (1) The cue must open the **sentence**, after list, blockquote, task-list-checkbox and emphasis markers — "Prompt the positive" is a rule about *instructions*, so descriptive prose is not in its scope, and a mid-sentence cue is already the paired form - ("Prefer X; never Y"). (2) The line must end its sentence — this repo hard-wraps prose and the - pairing rule is per sentence, so a continuation line cannot be shown to lack a positive sitting on - the next line; the same test excludes a table row. The cost is stated rather than hidden: a - subject-led instruction ("The agent must not emit a bare summary") is not selected. -- **`negation` is scoped to one physical line, and WITHHOLDS on a soft-wrapped sentence (known - limitation).** `detect.sh` classifies line by line, and the scope gate requires a line to close - its own sentence, so a prohibition that markdown soft-wraps reaches no verdict at all: `Do not - use markdown` on one line with `in the summary body.` on the next is not reported, even though - no positive is paired anywhere in that sentence. Unlike every other gap here the error direction - is a **silent withhold, not a false positive** — it costs coverage rather than reviewer - attention, and in a hard-wrapped repo it takes every prohibition long enough to wrap. This is the - one place the shape does not satisfy the fail-safe-toward-emitting property that the - detector-findings admission test asks for, and it is stated rather than assumed away. - Accumulating sentence state across soft line breaks is deferred to - [#3195](https://github.com/melodic-software/claude-code-plugins/issues/3195). + ("Prefer X; never Y"). The cost is stated rather than hidden: a subject-led instruction ("The + agent must not emit a bare summary") is not selected. (2) Soft-wrapped sentences are accumulated + across paragraph lines before the classifier runs, so a prohibition that markdown wraps is judged + as one sentence. A finding is attributed to the first physical line of that sentence — the line + the cue opens on — so the fix action lands on the instruction's start, not its continuation. The + other eight shapes stay line-scoped. A table row still does not select: the cue must open the + sentence, and a row begins with `|`. - **Opt-out markers respected.** A well-formed HTML comment line `` (covers the next paragraph, through the next blank line or heading) and `` (exactly the next physical line — a blank line consumes it, so place the marker directly above the content line) skip the wrapped content. A prose mention of the marker name is not a live marker. - **Convention-path exemptions apply per matched path, never per line.** An angle-bracket slot variable (`.work//…`, `docs/topics//…`) is a schema placeholder, not a literal path; the reserved concern-scoped roots (`.work/handoffs/`, `.work/reviews/`, `.work/running-retros/`, `.work/overengineering/` — roster SSOT: topic-docs Memory, concern-scoped tier) are citable only bare or with a placeholder child — a concrete child under them flags. A convention token on a line never exempts a concrete ghost ref sharing that line; the tracked concern file (`.claude/topic-docs.yaml`) matches no ghost-ref pattern and needs no exemption. Exception: the retired `.claude/notes/` location flags even in placeholder form. - **Output deterministic.** Filenames sort lexically; per-file tier rows sort by line number; no timestamps in output. diff --git a/plugins/docs-hygiene/skills/audit-noise/scripts/detect.sh b/plugins/docs-hygiene/skills/audit-noise/scripts/detect.sh index 4435c9edf..7768da221 100755 --- a/plugins/docs-hygiene/skills/audit-noise/scripts/detect.sh +++ b/plugins/docs-hygiene/skills/audit-noise/scripts/detect.sh @@ -205,6 +205,43 @@ audit_noise_resolve_convention_roots total_t1=0 total_t2=0 total_t3=0 files_audited=0 +# Record one finding into a nameref array as a US-delimited row so the file +# can emit in line-number order after paragraph-scoped negation is flushed. +audit_noise_record_finding() { + local at_line="$1" shape="$2" excerpt="$3" marker="${4:-}" + local -n _rows="$5" + local -n _t1="$6" _t2="$7" _t3="$8" + local tier="" + audit_noise_shape_tier_into "$shape" tier + _rows+=("${at_line}"$'\x1f'"${tier}"$'\x1f'"${shape}"$'\x1f'"${excerpt}"$'\x1f'"${marker}") + case "$tier" in + 1) _t1=$((_t1 + 1)) total_t1=$((total_t1 + 1)) ;; + 2) _t2=$((_t2 + 1)) total_t2=$((total_t2 + 1)) ;; + *) _t3=$((_t3 + 1)) total_t3=$((total_t3 + 1)) ;; + esac +} + +audit_noise_print_findings() { + local file="$1" + # nameref to the caller's finding_rows; the string assignment is the name, + # not a scalar overwrite of the array. + # shellcheck disable=SC2178 + local -n _rows="$2" + local at_line tier shape excerpt marker + ((${#_rows[@]})) || return 0 + while IFS=$'\x1f' read -r at_line tier shape excerpt marker; do + printf 'File: %s\n' "$file" + printf 'Finding tier: %s\n' "$tier" + printf 'Finding shape: %s\n' "$shape" + printf 'Finding line: %s\n' "$at_line" + printf 'Finding excerpt: %s\n' "$excerpt" + if [[ "$shape" == 'negation' && -n "$marker" ]]; then + printf 'Finding marker: %s\n' "$marker" + fi + printf '%s\n' '---' + done < <(printf '%s\n' "${_rows[@]}" | LC_ALL=C sort -t$'\x1f' -k1,1n) +} + audit_file() { local file="$1" [[ -f "$file" ]] || return 0 @@ -218,16 +255,74 @@ audit_file() { local in_ignored_para=0 skip_next=0 local in_frontmatter=0 in_fence=0 local fence_char="" fence_len=0 - local -a shapes=() + # finding_rows is written via nameref in audit_noise_record_finding. + # shellcheck disable=SC2034 + local -a shapes=() finding_rows=() local shape tier excerpt line heading_text local fence_delim fence_dchar fence_dlen + local is_heading=0 + # Negation is paragraph-scoped: accumulate soft-wrapped lines, then classify. + # Other shapes stay line-scoped. Attribution is the first physical line of + # the triggering sentence — that is where the cue opens, so the fix action + # lands on the instruction's start rather than its wrap continuation. + # Offsets are tracked on the UNWRAPPED join so inline backticks cannot shift + # attribution onto an earlier line. + local neg_unwrapped="" + local -a neg_line_nums=() neg_line_texts=() neg_offsets=() + + reset_negation() { + neg_unwrapped="" + neg_line_nums=() + neg_line_texts=() + neg_offsets=() + } + + flush_negation() { + local sentences=() s idx offset attr_line attr_excerpt sentence_off + local cursor=0 rest prefix_in_rest + [[ -n "${neg_unwrapped//[[:space:]]/}" ]] || { + reset_negation + return 0 + } + # Every qualifying sentence in the paragraph is a finding. Returning after + # the first would drop a later imperative on its own physical line. + # Walk a cursor so two identical sentences attribute to their own lines: + # `${var%%"$s"*}` always anchors at the earliest match. + audit_noise_split_sentences_into sentences "$neg_unwrapped" + for s in "${sentences[@]}"; do + rest="${neg_unwrapped:cursor}" + prefix_in_rest="${rest%%"${s}"*}" + sentence_off=$cursor + if [[ "$prefix_in_rest" != "$rest" ]]; then + sentence_off=$((cursor + ${#prefix_in_rest})) + cursor=$((sentence_off + ${#s})) + fi + audit_noise_line_has_negation_without_positive "$s" "paragraph" || continue + attr_line="${neg_line_nums[0]}" + attr_excerpt="" + idx=0 + for offset in "${neg_offsets[@]}"; do + if [[ $offset -le $sentence_off ]]; then + attr_line="${neg_line_nums[idx]}" + audit_noise_trim_excerpt "${neg_line_texts[idx]}" attr_excerpt + fi + idx=$((idx + 1)) + done + [[ -n "$attr_excerpt" ]] || audit_noise_trim_excerpt "${neg_line_texts[0]}" attr_excerpt + audit_noise_record_finding "$attr_line" "negation" "$attr_excerpt" \ + "${AUDIT_NOISE_FIRED_MARKER:-}" finding_rows t1 t2 t3 + done + reset_negation + } while IFS= read -r line || [[ -n "$line" ]]; do line_num=$((line_num + 1)) + is_heading=0 # YAML frontmatter: opening --- on line 1 (or immediately after a BOM-less # blank? — SKILL exempts frontmatter; require the conventional start). if [[ $line_num -eq 1 && "$line" == '---' ]]; then + flush_negation in_frontmatter=1 continue fi @@ -245,6 +340,7 @@ audit_file() { # A bare toggle keyed on "line starts with 3+" treats the inner fence as # the outer close and then scans the remaining example as prose. if [[ "$line" =~ ^(\`{3,}|~{3,}) ]]; then + flush_negation fence_delim="${BASH_REMATCH[1]}" fence_dchar="${fence_delim:0:1}" fence_dlen=${#fence_delim} @@ -268,6 +364,8 @@ audit_file() { # an exempt ## Sources followed by an H1 stay exempt to EOF, and ### Sources # was never recognized). if [[ "$line" =~ ^(#{1,6})[[:space:]]+(.*)$ ]]; then + flush_negation + is_heading=1 heading_text="${BASH_REMATCH[2]}" heading_text="${heading_text%%$'\r'*}" if audit_noise_section_exempt "$heading_text"; then @@ -283,48 +381,66 @@ audit_file() { # merely mentions the marker name must not act as a live marker. Order # matters: -line first. if audit_noise_is_ignore_line_marker "$line"; then + flush_negation skip_next=1 continue fi if audit_noise_is_ignore_para_marker "$line"; then + flush_negation in_ignored_para=1 continue fi if [[ -z "${line//[[:space:]]/}" ]]; then + flush_negation in_ignored_para=0 fi if [[ $in_exempt -eq 1 || $in_ignored_para -eq 1 || $skip_next -eq 1 ]]; then + flush_negation skip_next=0 continue fi # Hot path: nameref APIs only — no per-line command substitutions. # Shape helpers unwrap/strip inline code internally (ghost-ref vs others). - if audit_noise_detect_shapes_into shapes "$line"; then + # Negation is classified after paragraph accumulation, not here. + if audit_noise_detect_shapes_into shapes "$line" "skip-negation"; then audit_noise_trim_excerpt "$line" excerpt for shape in "${shapes[@]}"; do [[ -z "$shape" ]] && continue - audit_noise_shape_tier_into "$shape" tier - printf 'File: %s\n' "$file" - printf 'Finding tier: %s\n' "$tier" - printf 'Finding shape: %s\n' "$shape" - printf 'Finding line: %s\n' "$line_num" - printf 'Finding excerpt: %s\n' "$excerpt" - # The fired prohibition travels with the finding so the relay writer - # reports the marker from the sentence that actually triggered. Keeping - # it here leaves ONE implementation of the sentence walk; re-deriving it - # in the writer would be a second copy free to drift from this one. - if [[ "$shape" == 'negation' && -n "${AUDIT_NOISE_FIRED_MARKER:-}" ]]; then - printf 'Finding marker: %s\n' "$AUDIT_NOISE_FIRED_MARKER" - fi - printf '%s\n' '---' - case "$tier" in - 1) t1=$((t1 + 1)) total_t1=$((total_t1 + 1)) ;; - 2) t2=$((t2 + 1)) total_t2=$((total_t2 + 1)) ;; - *) t3=$((t3 + 1)) total_t3=$((total_t3 + 1)) ;; - esac + audit_noise_record_finding "$line_num" "$shape" "$excerpt" "" \ + finding_rows t1 t2 t3 done fi + + if [[ -z "${line//[[:space:]]/}" ]]; then + continue + fi + # A new list item is its own block, not a soft-wrap continuation of the + # previous item. Flush first so `- Do not use markdown` / `- Prefer HTML.` + # cannot pair across items. Hold the regex in a variable so the unquoted + # `)` in `[.)]` is not parsed as bash syntax. + local list_item_re='^[[:space:]]*([-*+]|[0-9]+[.)])[[:space:]]' + if [[ "$line" =~ $list_item_re ]] && [[ ${#neg_line_nums[@]} -gt 0 ]]; then + flush_negation + fi + # Accumulate this physical line into the negation paragraph. Offsets are + # taken on the unwrapped join so a backticked earlier line cannot pull + # attribution forward. A heading is its own paragraph. + local trimmed uwrapped="" + trimmed="${line#"${line%%[![:space:]]*}"}" + audit_noise_unwrap_backticks "$trimmed" uwrapped + if [[ -n "$neg_unwrapped" ]]; then + neg_unwrapped+=" " + fi + neg_offsets+=(${#neg_unwrapped}) + neg_unwrapped+="$uwrapped" + neg_line_nums+=("$line_num") + neg_line_texts+=("$line") + if [[ $is_heading -eq 1 ]]; then + flush_negation + fi done <"$file" + flush_negation + audit_noise_print_findings "$file" finding_rows printf 'Summary file: %s | T1=%s T2=%s T3=%s\n' "$file" "$t1" "$t2" "$t3" } diff --git a/plugins/docs-hygiene/skills/audit-noise/scripts/detect.test.sh b/plugins/docs-hygiene/skills/audit-noise/scripts/detect.test.sh index 5ccaab901..7dc8c8c50 100755 --- a/plugins/docs-hygiene/skills/audit-noise/scripts/detect.test.sh +++ b/plugins/docs-hygiene/skills/audit-noise/scripts/detect.test.sh @@ -1084,17 +1084,167 @@ forms_count="$(printf '%s\n' "$forms_out" | grep -c '^Finding shape: negation')" assert_contains "plain, list, bolded and blockquoted imperatives all still flag" \ "count=$forms_count" "count=4" -# A hard-wrapped continuation line cannot be shown to lack a positive that sits -# on the next line, so it must not select on its own. -NEG_WRAP="$TEST_TMPDIR/negation-wrap.md" -cat >"$NEG_WRAP" <<'EOF' -# Wrapped fixture +# Soft-wrapped sentences are accumulated before the classifier runs. A positive +# on the continuation line pairs the sentence; a wrap with no positive must +# flag, attributed to the first physical line of the sentence (where the cue +# opens). The older line-scoped gate withheld the unpaired wrap. +NEG_WRAP_PAIRED="$TEST_TMPDIR/negation-wrap-paired.md" +cat >"$NEG_WRAP_PAIRED" <<'EOF' +# Wrapped paired fixture + +Do not use markdown; +compose flowing prose instead. +EOF +wrap_paired_out="$(bash "$DETECT" "$NEG_WRAP_PAIRED")" +assert_not_contains "a wrapped sentence with its positive on the next line is not flagged" \ + "$wrap_paired_out" "Finding shape: negation" + +NEG_WRAP_BARE="$TEST_TMPDIR/negation-wrap-bare.md" +cat >"$NEG_WRAP_BARE" <<'EOF' +# Wrapped bare fixture + +Do not use markdown +in the summary body. +EOF +wrap_bare_out="$(bash "$DETECT" "$NEG_WRAP_BARE")" +assert_contains "a wrapped sentence with no positive is flagged" \ + "$wrap_bare_out" "Finding shape: negation" +assert_contains "and is attributed to the first line of the sentence" \ + "$wrap_bare_out" $'Finding shape: negation\nFinding line: 3' + +NEG_WRAP_CTRL="$TEST_TMPDIR/negation-wrap-control.md" +cat >"$NEG_WRAP_CTRL" <<'EOF' +# Unwrapped control + +Never emit a bare summary. +EOF +wrap_ctrl_out="$(bash "$DETECT" "$NEG_WRAP_CTRL")" +assert_contains "an unwrapped imperative still flags" \ + "$wrap_ctrl_out" "Finding shape: negation" + +# The other shapes stay line-scoped: joining would invent a citation that no +# physical line actually contains. +LINE_SCOPED="$TEST_TMPDIR/negation-line-scoped-sibling.md" +cat >"$LINE_SCOPED" <<'EOF' +# Line-scoped sibling fixture + +Empirically +observed inside a wrap that would match if joined. +EOF +line_scoped_out="$(bash "$DETECT" "$LINE_SCOPED")" +assert_not_contains "citation does not accumulate across a soft wrap" \ + "$line_scoped_out" "Finding shape: citation" + +# Frontmatter, fences and opt-out markers still bound accumulation: a wrapped +# prohibition inside any of them must not join a later body sentence. +NEG_FENCE_WRAP="$TEST_TMPDIR/negation-fence-wrap.md" +cat >"$NEG_FENCE_WRAP" <<'EOF' +# Fence-bounded wrap + +```text +Do not use markdown +in the fenced body. +``` -Do not use markdown in your response; compose it instead as -smoothly flowing prose paragraphs. +Do not use markdown +in the summary body. +EOF +fence_wrap_out="$(bash "$DETECT" "$NEG_FENCE_WRAP")" +fence_wrap_count="$(printf '%s\n' "$fence_wrap_out" | grep -c '^Finding shape: negation')" +assert_contains "a fenced wrap is skipped and the body wrap still flags once" \ + "count=$fence_wrap_count" "count=1" +assert_contains "the body wrap is attributed past the fence" \ + "$fence_wrap_out" $'Finding shape: negation\nFinding line: 8' + +NEG_IGNORE_WRAP="$TEST_TMPDIR/negation-ignore-wrap.md" +cat >"$NEG_IGNORE_WRAP" <<'EOF' +# Opt-out-bounded wrap + + +Do not use markdown +in the ignored paragraph. + +Do not use markdown +in the summary body. +EOF +ignore_wrap_out="$(bash "$DETECT" "$NEG_IGNORE_WRAP")" +ignore_wrap_count="$(printf '%s\n' "$ignore_wrap_out" | grep -c '^Finding shape: negation')" +assert_contains "an ignored wrap is skipped and the body wrap still flags once" \ + "count=$ignore_wrap_count" "count=1" + +# Every qualifying sentence in one paragraph is a finding, attributed to its +# own first physical line. Returning after the first match would drop the +# second imperative. +NEG_TWO_WRAP="$TEST_TMPDIR/negation-two-in-paragraph.md" +cat >"$NEG_TWO_WRAP" <<'EOF' +# Two-sentence paragraph + +Do not use markdown. +Never include XML. +EOF +two_wrap_out="$(bash "$DETECT" "$NEG_TWO_WRAP")" +two_wrap_count="$(printf '%s\n' "$two_wrap_out" | grep -c '^Finding shape: negation')" +assert_contains "each imperative sentence in a paragraph flags" \ + "count=$two_wrap_count" "count=2" +assert_contains "the first sentence is attributed to its own line" \ + "$two_wrap_out" $'Finding shape: negation\nFinding line: 3' +assert_contains "the second sentence is attributed to its own line" \ + "$two_wrap_out" $'Finding shape: negation\nFinding line: 4' + +# Two textually identical sentences must not both pin to the first match. +# `${var%%"$s"*}` always anchors at the earliest occurrence, so a cursor has +# to walk past each already-attributed sentence. +NEG_DUP="$TEST_TMPDIR/negation-duplicate-sentences.md" +cat >"$NEG_DUP" <<'EOF' +# Duplicate-sentence paragraph + +Do not use markdown. +Do not use markdown. +EOF +dup_out="$(bash "$DETECT" "$NEG_DUP")" +dup_lines="$(printf '%s\n' "$dup_out" | awk '/^Finding shape: negation$/{getline; if($1=="Finding" && $2=="line:") print $3}')" +assert_contains "identical later sentence keeps its own line" \ + "$(printf '%s' "$dup_lines")" $'3\n4' + +# Sibling list items are separate blocks. Joining them would let the second +# item's `Prefer` pair the first item's prohibition. +NEG_LIST_SIBLING="$TEST_TMPDIR/negation-list-siblings.md" +cat >"$NEG_LIST_SIBLING" <<'EOF' +# List-sibling fixture + +- Do not use markdown +- Prefer HTML. +EOF +list_sib_out="$(bash "$DETECT" "$NEG_LIST_SIBLING")" +assert_contains "a later list item cannot pair an earlier item" \ + "$list_sib_out" "Finding shape: negation" + +# Inline backticks on an earlier line must not shift attribution. +NEG_TICK_ATTR="$TEST_TMPDIR/negation-tick-attr.md" +cat >"$NEG_TICK_ATTR" <<'EOF' +# Tick attribution fixture + +Use `foo`. +Do not use markdown. +EOF +tick_attr_out="$(bash "$DETECT" "$NEG_TICK_ATTR")" +assert_contains "attribution ignores earlier inline backticks" \ + "$tick_attr_out" $'Finding shape: negation\nFinding line: 4' + +# Line-scoped shapes on a later line must not print before an earlier +# negation once the file is flushed — per-file rows sort by line number. +NEG_ORDER="$TEST_TMPDIR/negation-order.md" +cat >"$NEG_ORDER" <<'EOF' +# Order fixture + +Do not use markdown +in the summary. Empirically observed later. EOF -wrap_out="$(bash "$DETECT" "$NEG_WRAP")" -assert_not_contains "a hard-wrapped continuation does not select" "$wrap_out" "Finding shape: negation" +order_out="$(bash "$DETECT" "$NEG_ORDER")" +# The printed order must be negation (line 3) then citation (line 4). +order_shapes="$(printf '%s\n' "$order_out" | awk '/^Finding shape:/{print $3}')" +assert_contains "findings print in line-number order" \ + "$(printf '%s' "$order_shapes")" $'negation\ncitation' # The imperative gate is PER SENTENCE. A line-level gate admits the whole line # on its first sentence and then lets a later DESCRIPTIVE sentence be reported — diff --git a/plugins/docs-hygiene/skills/audit-noise/scripts/lib/noise-shapes.sh b/plugins/docs-hygiene/skills/audit-noise/scripts/lib/noise-shapes.sh index b638d8141..a99c4cf53 100644 --- a/plugins/docs-hygiene/skills/audit-noise/scripts/lib/noise-shapes.sh +++ b/plugins/docs-hygiene/skills/audit-noise/scripts/lib/noise-shapes.sh @@ -302,8 +302,12 @@ audit_noise_detect_shapes_into() { # routinely the inline-code span itself (`--force`, `rm -rf`), and stripping it # would erase the very evidence the carve-out needs — turning a guardrail into # a false finding rather than a withheld one. - if audit_noise_line_has_negation_without_positive "$unwrapped"; then - _audit_noise_shapes_out+=('negation') + # detect.sh can pass skip-negation so it can classify this shape after + # accumulating a soft-wrapped sentence; other callers still get per-line. + if [[ "${3:-}" != "skip-negation" ]]; then + if audit_noise_line_has_negation_without_positive "$unwrapped"; then + _audit_noise_shapes_out+=('negation') + fi fi ((${#_audit_noise_shapes_out[@]} > 0)) } @@ -658,10 +662,15 @@ AUDIT_NOISE_FIRED_MARKER="" audit_noise_line_has_negation_without_positive() { local sentences=() s lower m + local mode="${2:-line}" AUDIT_NOISE_FIRED_MARKER="" - # Line-level gate first — cheapest, and genuinely a property of the LINE: a - # continuation cannot be shown to lack a positive sitting on the next one. - audit_noise_line_ends_sentence "$1" || return 1 + # Line-level gate first — cheapest, and genuinely a property of the LINE when + # the caller has not already accumulated a wrapped sentence. Paragraph mode + # skips it: a hard-wrapped prohibition with no positive must emit, and the + # closed-sentence test is what used to withhold that case. + if [[ "$mode" == "line" ]]; then + audit_noise_line_ends_sentence "$1" || return 1 + fi audit_noise_split_sentences_into sentences "$1" for s in "${sentences[@]}"; do # The imperative gate is per SENTENCE, so a line that opens imperatively