Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion plugins/ai-slop/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
"name": "ai-slop",
"version": "0.3.1",
"version": "0.3.2",
"description": "Detects and removes AI-writing tells (slop) in checked-in markdown prose: em dashes, emoji formatting, AI vocabulary, negative parallelisms, chatbot phrases, filler, stacked hedging, citation artifacts, and the rest of a catalog distilled from Wikipedia's Signs of AI writing. Read-only audit by default with a deterministic detector plus a judgment rubric; an explicit fix action rewrites findings behind a semantic-diff guard. Findings conform to the detector-findings convention so the review fanout fix relay can consume them.",
"author": {
"name": "Melodic Software",
Expand Down
35 changes: 35 additions & 0 deletions plugins/ai-slop/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,40 @@
# Changelog

## [0.3.2]

The catalog's cited source page lists an **Ineffective indicators** section — signals that
page's own editors consider unreliable for LLM detection. Until this release both that
section and **Comment-specific indicators** were a recorded fetch gap (the catalog-time
window ran out before those headings). That left a guardrail question unanswered: if a
rule we ship appeared there, we would be shipping a tell our own source classifies as
ineffective.

- **Negative finding, 2026-08-21.** Retrieved
<https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing> (live revision
[1370403579](https://en.wikipedia.org/w/index.php?title=Wikipedia:Signs_of_AI_writing&oldid=1370403579),
`2026-08-20T23:13:41Z`) and re-read the catalog pin (revision
[1369699198](https://en.wikipedia.org/w/index.php?title=Wikipedia:Signs_of_AI_writing&oldid=1369699198),
2026-08-16, parse section 80). Both revisions list the same eight ineffective
indicators: perfect grammar; mixed casual/formal registers; "bland"/"robotic" prose;
undifferentiated "fancy"/"academic"/"formal" prose; transition words in isolation;
unsourced content; bizarre wikitext; correct wikitext.
- **None of the 15 shipped script rules appear in that list**, including the two
candidates the gap was filed to check (`rule-em-dash`, `rule-rule-of-three`). Those
two remain valid *Style* / *Language and grammar* signs on the source page. The
em-dash entry's "most useful […] in combination with other indicators, not by
itself" sentence is a corroboration qualifier on a kept tell, not an Ineffective
indicators listing; zero-tolerance stays the shipped house-style default.
- **Comment-specific indicators** retrieved in the same visit (pin section 62).
Wikipedia talk-page scope. One of the seven tells already has a slug
(`rule-canned-policy-assurance`). The other six land as `recorded-only` /
`wikipedia-specific` entries so slug-based rechecks can track them:
`rule-misquoted-policies`, `rule-maintenance-banner-transclusion`,
`rule-sectioned-comments`, `rule-request-critic-input`,
`rule-dismiss-origin-speculation`, `rule-redirect-to-content`. Inventory count
59 → 65. No script-roster change.
- The inventory pin stays `1369699198`. Detector, emitter, evals, and rewrite guide are
untouched: this is the fetch-gap close, not a roster change.

## [0.3.1]

The audit skill told operators to keep this plugin's findings away from the very relay route that
Expand Down
126 changes: 115 additions & 11 deletions plugins/ai-slop/skills/audit/reference/catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,26 @@ The "Cursor unslop additions" section was inspired by
- **Recheck trigger**: each `ai-slop` release and each fleet audit. Per-revision rechecking was
rejected: the page was measured at 50+ edits/week (2026-08-17), so a per-revision trigger would
fire continuously.
- **Known fetch gap**: the page's "Comment-specific indicators" and "Ineffective indicators"
sections exceeded the fetch window at catalog time and are recorded as section notes below,
without entries. The recheck trigger covers closing this gap.
- **Known fetch gap**: closed 2026-08-21 for both leftover sections. The catalog-time fetch
window missed "Comment-specific indicators" and "Ineffective indicators"; this recheck
retrieved them from the catalog pin and from the live page. See those sections below.
- **Recheck logged (2026-08-21)**:
- Live page: <https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing>, MediaWiki
revisions query returned revision
[1370403579](https://en.wikipedia.org/w/index.php?title=Wikipedia:Signs_of_AI_writing&oldid=1370403579)
(`timestamp=2026-08-20T23:13:41Z`, user `Superb Owl`). Retrieved via WebFetch of the
article URL plus `action=query&prop=revisions`.
- Catalog pin: revision
[1369699198](https://en.wikipedia.org/w/index.php?title=Wikipedia:Signs_of_AI_writing&oldid=1369699198)
(2026-08-16). Retrieved via `action=parse&oldid=1369699198&prop=wikitext` (section 80 =
Ineffective indicators; section 62 = Comment-specific indicators; section 29 = Overuse of
em dashes).
- The inventory pin stays `1369699198`. This recheck closes the two-section gap; it does
not re-derive the rest of the inventory.

## Inventory

59 tells catalogued from the Wikipedia source revision, plus 7 in the "Cursor unslop additions"
65 tells catalogued from the Wikipedia source revision, plus 7 in the "Cursor unslop additions"
section at the end of this file. Entry marker: `### rule-<slug>: <name>`. The qualified id used
in crosswalk rows and findings files is `ai-slop/audit/rule-<slug>`. Fields:

Expand Down Expand Up @@ -258,6 +271,13 @@ Second pass, 2026-08-19, for the Cursor additions, against the same corpus:
plan approval): any occurrence outside code fences and inline code flags. Documents that
require em dashes opt out per-document via config path-lists or the in-file marker; the rule is
never threshold-calibrated and is excluded from the `recorded-only` demotion path.
- The source page's Style section (catalog pin and the 2026-08-21 recheck) treats this as a
**valid sign**, not an ineffective one. The same section carries the qualifier *"This sign
is most useful when taken in combination with other indicators, not by itself."* That is a
corroboration note on a kept tell, not a listing under **Ineffective indicators** (checked
explicitly; see that section). The shipped default stays zero-tolerance: this plugin is a
house-style detector, not a Wikipedia AI-authorship tribunal. A consuming repo that wants the
source's combination reading disables the rule or uses `em_dash_allowed_paths`.

### rule-emoji-formatting: Emoji as formatting

Expand Down Expand Up @@ -422,9 +442,61 @@ Second pass, 2026-08-19, for the Cursor additions, against the same corpus:

## Comment-specific indicators

Section recorded as a known fetch gap (see the upstream-drift record): the source section exceeded
the fetch window at catalog time. Its scope is Wikipedia talk-page comments, so its tells are
expected to classify wikipedia-specific. Entries land when the recheck trigger next fires.
Fetch gap closed 2026-08-21 (see the upstream-drift record). The source section is Wikipedia
talk-page comments, so every tell classifies `wikipedia-specific` / `recorded-only` — they have
no general-prose analogue worth a script rule. Quoted from the catalog pin (revision
1369699198, parse section 62) and confirmed on the live page (revision 1370403579).

One of the seven tells already has a slug under Edit summaries: downplaying AI use by
claiming policy adherence is `rule-canned-policy-assurance`. The other six land here.

### rule-misquoted-policies: Misquoted policies and invented shortcuts

- detectability: judgment
- applicability: wikipedia-specific
- v1: recorded-only
- Talk-page comments that cite made-up policy shortcuts or misstate existing ones. Wikipedia
project-page namespace; no markdown-corpus analogue.

### rule-maintenance-banner-transclusion: Transcluded maintenance banners in comments

- detectability: mechanical
- applicability: wikipedia-specific
- v1: recorded-only
- Transcluding a maintenance banner whenever the comment mentions it. Wikitext talk-page
convention.

### rule-sectioned-comments: Lengthy comments divided into titled sections

- detectability: mechanical
- applicability: wikipedia-specific
- v1: recorded-only
- Talk-page comments split into titled sections in Markdown, plain text, or level-2/3
subheadings. Distinct from `rule-verbose-edit-summaries`, which is the edit-summary field.

### rule-request-critic-input: Requests for critics to specify improvements

- detectability: judgment
- applicability: wikipedia-specific
- v1: recorded-only
- Asking critics or other editors to say exactly what to improve, as a deflection. Talk-page
register.

### rule-dismiss-origin-speculation: Dismissing AI-origin concerns as speculation

- detectability: judgment
- applicability: wikipedia-specific
- v1: recorded-only
- Treating questions about whether the comment is AI-generated as "unsubstantiated
speculation" rather than addressing the content tells. Talk-page register.

### rule-redirect-to-content: Redirecting AI concerns toward content improvement

- detectability: judgment
- applicability: wikipedia-specific
- v1: recorded-only
- Urging critics to improve the content instead of worrying that it is AI-generated.
Talk-page register.

## Edit summaries

Expand Down Expand Up @@ -539,10 +611,42 @@ emitted as findings.

## Ineffective indicators

Section recorded as a known fetch gap (see the upstream-drift record): the source section exceeded
the fetch window at catalog time. Its content lists signals the page's editors consider UNRELIABLE
for detection; when the recheck fires, its items land here as guardrails on our own rules (a
signal listed there must not become a rule).
Fetch gap closed 2026-08-21 (see the upstream-drift record). This section lists signals the
page's own editors consider **unreliable** for LLM detection — a guardrail on our roster, not
a source of new rules. A signal listed here must not become a rule.

**Verdict: no shipped rule appears here.** Compared against all 15 `v1: script` slugs in
`detect.sh`, including the two candidates named when this gap was filed (`rule-em-dash`,
`rule-rule-of-three`). Both of those live in other source sections as *valid* signs
(Style / Language and grammar). The eight ineffective indicators are the same on the catalog
pin (revision 1369699198, parse section 80, 2026-08-16) and the live recheck (revision
1370403579, retrieved 2026-08-21 from
<https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing>).

Quoted from the pin (CC BY-SA 4.0; ellipses mark dropped citation/example markup):

> False accusations of AI use can drive away new editors and foster an atmosphere of
> suspicion. […] Here are several somewhat commonly used indicators that are ineffective
> in LLM detection—and may even indicate the opposite.

- **Perfect grammar** — skilled human writers also produce this.
- **Combination of casual and formal registers**, or language that sounds both "clinical"
and "emotional" — technical-field casual writing, mixed registers, or multi-editor pages.
- **"Bland" or "robotic" prose** — LLM output has *specific* traits; "robotic" is not one.
- **"Fancy", "academic", or "formal" prose** — the page's own wording: LLMs favor *specific
words*; "the correlation does not extend to all formal, academic, or 'fancy'-sounding
prose." `rule-ai-vocabulary` is the specific-word rule, not a formality detector.
- **Transition words (in isolation)** — older output overused a few (`Additionally`,
`Consequently`, `Notably`); "this is not a strong tell." The shipped vocabulary list
already dropped `additionally` for legitimate technical use; there is no standalone
transition-words rule.
- **Unsourced content** — most uncited articles predate LLMs; modern chatbots also cite.
- **Bizarre wikitext** — random HTML/VisualEditor artifacts are *not* the LLM markup tells
already catalogued under Markup.
- **Correct wikitext** — correct formatting is normal.

None of those eight is a shipped script rule, a shipped rubric tell, or a Cursor-addition
slug. No drop or re-scope follows.

## Historical indicators

Expand Down