Repository navigation
docs: rewrite the README around the attack, the gate and the proof - #316
Conversation
The README now leads with a recorded run, then a quick start, the three attacks truecopy stops, how a pin works (with a full real lock entry), every surface it gates, CI setup, the dated ecosystem figures, the trust promises, an FAQ and a docs map. The terminal render is generated by scripts/readme/terminal.mjs from a real run of this checkout's CLI in a throwaway directory. It asserts every verdict and exit code the README quotes and writes nothing if one moved. CI gains one step on one matrix leg: scripts/check-readme-links.mjs fails when a relative link, image or anchor in the README or docs/ stops resolving. It reads files only and makes no network calls.
check-readme-links.mjs read a whole srcset value as one path, so a valid "a.png 1x, b.png 2x" failed even with both files present. It now splits the value the way HTML does (a URL runs to whitespace, descriptors run to the next comma) and checks each URL on its own. The checker exports its pieces and test/readme-links.test.mjs covers srcset, images, links, anchors, skipped external targets and the script's exit codes. The lib.mjs header said the package has no dependencies; it has one. The note now says what holds: these scripts use Node's standard library only. The asset table names the hero's source as a written brief.
A percent-encoded fragment such as #caf%C3%A9 was compared undecoded, so a working link to the heading "Café" failed. Fragments and paths are now percent-decoded first, and a malformed escape fails with its own message. An <a id> inside a fenced example counted as an anchor, so a link to it passed although the rendered page has no such anchor. Anchors and link targets are now read with fenced code and inline code spans left out. The header states what the line reader does not model. Four new fixture tests cover these cases; each fails on the previous checker.
…n slugs Any line that looked like a fence toggled the fence state, so a three backtick line inside a four backtick example closed it early: links in the example were checked and real links after it were skipped. Both readers now share proseLines, which closes a fence only on a run of the same character, at least as long, with nothing after it. slugify removed every underscore, but GitHub keeps literal ones (foo_bar, _private, snake_case in code) and drops only underscore emphasis. It now builds the heading text the way GitHub renders it and applies GitHub's slugger to that, without trimming afterwards. The slug cases in the tests come from headings rendered by GitHub's markdown API. Each new test fails on the previous checker.
The link checker found code spans with a regex that could open in the middle of a backtick run, so a line like `` [bad](missing.md) ` hid a rendered link. It now scans whole runs: a run of N backticks closes only at the next run of exactly N, a run with no closer is literal, and a backslash escapes a backtick outside code. Heading slugs now park code spans before tags are stripped, so the heading Use `<file>` gives use-file as GitHub does, and a code span's padding space is trimmed. Autolinks keep their text and HTML comments drop out. Duplicate headings take the next -N suffix not already taken, so Setup, Setup, Setup-1 give setup, setup-1, setup-1-1 (github-slugger's rule). The header now says only # headings are read. New fixtures cover each case; the slug cases match GitHub's markdown API.
A link such as [guide](docs/guide(v2).md) was cut at the first ) and
failed CI although GitHub resolves it. Inline destinations are now parsed
the CommonMark way: balanced parentheses, the <...> form, an optional
title, backslash escapes. Reference definitions take the <...> form too.
Headings decode character references after tags and links are handled,
so ## A & B gives a--b while code keeps & literal. A named
reference outside a short table gives the heading no slug, so a link to
it fails rather than passing on a guess. An image in a heading adds no
text. href, src, srcset, id and name accept single quotes and decode
references.
The private-use range in slugify is now written \p{Co}, so the source
holds no invisible characters. The slug cases match GitHub's markdown API.
sprayberry-redline
left a comment
There was a problem hiding this comment.
Verdict: request changes. The README rewrite, terminal renderer, CI step and link-checker fixtures were reviewed. The link checker still skips valid links with leading whitespace and derives incorrect anchors for headings containing parenthesized link destinations, so those paths need fixes before it becomes a required CI gate.
1. Blocking: scripts/check-readme-links.mjs:131
let j = i;
let dest;
if (text[i] === '<') {
For valid CommonMark such as [guide]( missing.md ), parsing starts at the space after (. The bare-destination loop immediately stops, and the closing/title regex cannot consume missing.md, so inlineDestination returns null. targetsOf drops the link and checkFile reports success even when missing.md does not exist. Leading whitespace is allowed before both bare and angle-bracket destinations.
Suggested fix:
Skip permitted whitespace before parsing the destination, and add fixtures showing that whitespace-prefixed links to existing files pass and links to missing files fail.
2. Blocking: scripts/check-readme-links.mjs:180
r = withoutUnderscoreEmphasis(r.replace(/![[^]]]([^)])/g, '').replace(/[([^]])]([^)])/g, '$1'));
Heading links still use a first-closing-parenthesis regex despite the balanced destination parser used for link targets. With ## [Guide](docs/guide(v2).md), this replacement leaves Guide.md), producing the slug guidemd. GitHub renders the heading text as Guide and uses guide. Consequently a working [jump](#guide) fails CI, while the nonexistent #guidemd anchor passes.
Suggested fix:
Use balanced link-destination parsing when extracting rendered heading text as well, including image destinations. Add an end-to-end fixture with a parenthesized destination in a heading that accepts #guide and rejects #guidemd.
rule:none
Fix at 4627da5
Files: Tests: |
sprayberry-redline
left a comment
There was a problem hiding this comment.
Verdict: request changes. The leading-whitespace destination parsing and parenthesized heading-link findings are fixed, with regression fixtures for both. Review of the README, terminal renderer, CI step and checker tests found one remaining character-reference parsing defect that can reject working links and accept nonexistent anchors.
1. Blocking: test/readme-links.test.mjs:189
['\\& escaped', '-escaped'],
This fixture expects the wrong rendered heading. In CommonMark, the backslash escapes the ampersand, so \& escaped renders as literal & escaped, whose heading slug is amp-escaped, not -escaped. slugify currently decodes the escaped reference anyway. Consequently a working link to #amp-escaped fails the checker and a nonexistent #-escaped passes. markdownTarget has the same ordering problem: it removes backslash escapes before decoding references, so a destination containing \& is checked as a filename containing & instead of literal &.
Suggested fix:
Preserve escaped ampersands through character-reference decoding in both heading text and markdown destinations. Correct this expected slug and add end-to-end fixtures accepting #amp-escaped, rejecting #-escaped, and resolving a filename with a literal & in an escaped destination.
rule:none
A heading reads backslash escapes and references in one pass, so \& is the literal text & (slug amp-escaped), \<b> is no tag and \_x\_ is no emphasis. Escaped characters are now parked with the code spans and put back after references are decoded. A destination differs on GitHub: its renderer decodes references first and undoes escapes after, so a\&b.md links to a&b.md and c\&d.md to c&d.md. The markdown API serves exactly those hrefs, and the checker now follows that order. A ] after an even run of backslashes now ends a link. README: the trust set is the machine's own key, the global store and truecopy.trust. A clean runner has only truecopy.trust, so the CI example says to commit every publisher key there, the public half of the CI signing key included.
sprayberry-redline
left a comment
There was a problem hiding this comment.
Verdict: request changes. The earlier whitespace, parenthesized-destination and escaped-heading findings are addressed with regression fixtures. Review of the README, terminal renderer, CI step and tests found two remaining link-checker cases that reject valid documentation, so the checker needs fixes before gating CI.
1. Blocking: scripts/check-readme-links.mjs:326
const pathPart = hash < 0 ? t : t.slice(0, hash);
A working relative URL such as [guide](docs/guide.md?plain=1) is treated as the literal filesystem name docs/guide.md?plain=1. Even when docs/guide.md exists, existsSync fails and CI rejects the link. The same problem affects image URLs with ?raw=true. Only the fragment is separated before filesystem resolution; the query is not.
Suggested fix:
Separate the URL's query from its path before percent-decoding and checking file existence, while preserving any fragment for anchor checking. Add fixtures for existing and missing files with queries, including an image URL.
2. Blocking: scripts/check-readme-links.mjs:298
for (let k = text.indexOf(']('); k >= 0; k = text.indexOf('](', k + 1)) {
The inline-link scanner never checks for an unescaped opening bracket. Prose containing \[example](missing.md) renders as literal text, not a link, but this loop finds ](, parses missing.md and fails CI. Even plain ](missing.md) is treated as a link. Checking only whether the closing bracket is escaped does not establish that a link exists.
Suggested fix:
Require a matching, unescaped opening bracket before parsing an inline destination, or use a Markdown parser for link extraction. Add fixtures that ignore escaped-opening-bracket examples and unmatched closing brackets while still checking real links on the same line.
rule:none
Fix at 62de0ca[1] Query treated as part of the file name ( [2] A The existing fixtures still pass, and Files: Tests: |
sprayberry-redline
left a comment
There was a problem hiding this comment.
Verdict: request changes. The earlier destination-whitespace, heading-link, escaped-heading, query-string and bracket findings are addressed with regression fixtures. Review of the README, terminal renderer, CI step and tests found one remaining HTML-link extraction defect that lets broken links pass the new check.
1. Blocking: scripts/check-readme-links.mjs:321
for (const m of text.matchAll(/\b(?:href|src)=(?:"([^"]*)"|'([^']*)')/g)) add(attributeValue(m[1] ?? m[2]));
Valid HTML such as Guide or
renders a link or image, but this expression requires the equals sign and quote to be immediately adjacent to the attribute name. Neither target is collected, so checkFile returns zero failures even when the files are missing. The srcset expression has the same problem.
Suggested fix:
Allow HTML whitespace around the equals sign when extracting href, src and srcset attributes. Add fixtures showing that spaced attributes resolve existing targets and fail for missing targets, including a srcset candidate.
rule:none
Fix at be0958c[1] Spaced HTML attributes were not collected ( A new fixture in Files: Tests: |
sprayberry-redline
left a comment
There was a problem hiding this comment.
Verdict: request changes. The earlier escaped-heading, query-string, bracket and spaced-link-attribute findings are fixed with regression fixtures. Review of the README, terminal renderer, CI step and checker tests found three remaining parsing cases that reject working documentation or miss broken links.
1. Blocking: scripts/check-readme-links.mjs:265
for (const a of withoutCodeSpans(line).matchAll(/<a\b[^>]*?\s(?:id|name)=(?:"([^"]*)"|'([^']*)')/g)) out.add(attributeValue(a[1] ?? a[2]));
The whitespace fix does not cover anchor declarations. With <a id = "custom"></a> followed by [jump](#custom), the browser exposes the custom anchor, but anchorsOf omits it because this expression still requires id= with no spaces. checkFile then rejects the working jump link.
Suggested fix:
Allow whitespace around = for id and name as well, and add fixtures for working links to both forms.
2. Blocking: scripts/check-readme-links.mjs:65
for (const s of codeSpans(line)) { out += line.slice(last, s.start); last = s.end; }
Deleting code spans joins text that was not adjacent in the document. For example, [example]code(missing.md) has a code span between the bracketed text and parentheses and is not a Markdown link. This function turns it into [example](missing.md), so targetsOf collects a nonexistent link and checkFile fails CI.
Suggested fix:
Preserve a non-markup placeholder for each code span instead of concatenating its neighbors. Add a fixture that ignores this example while still detecting a real broken link on the same line.
3. Blocking: scripts/check-readme-links.mjs:322
for (const m of text.matchAll(/\b(?:href|src)\s*=\s*(?:"([^"]*)"|'([^']*)')/g)) add(attributeValue(m[1] ?? m[2]));
HTML permits unquoted attribute values. <a href=missing.md>Guide</a> and <img src=missing.png> render a link and an image, but neither matches this expression. With these as the only targets, checkFile reports zero failures even when both files are missing. The srcset extractor also requires quotes.
Suggested fix:
Handle valid unquoted values for href, src and srcset, preferably through shared attribute parsing, and add fixtures for both existing and missing unquoted targets.
rule:none
Fix at 2dc5b0bAll three findings are fixed in
Because
Files: Tests: |
sprayberry-redline
left a comment
There was a problem hiding this comment.
Verdict: request changes. The earlier query, bracket, spaced-attribute, code-span and unquoted-attribute findings are fixed with regression fixtures. Review of the README, terminal renderer, CI step and checker tests found two remaining HTML parsing defects that can miss broken images or reject literal examples.
1. Blocking: scripts/check-readme-links.mjs:139
for (const t of src.matchAll(/<([A-Za-z][A-Za-z0-9-])(?=[\s/>]|$)((?:"[^"]"|'[^']'|[^"'>]))/g)) {
A valid tag with a quoted attribute spanning lines loses its continuation. For example, with <img alt="first on one line and second" src="missing.png"> on the next, the first match stops before the opening quote. Since the match does not reach the end of the line, open becomes null. The second line has no opening tag, so missing.png is never checked and a broken image passes CI.
Suggested fix:
Preserve the incomplete tag, including quote state, across lines before extracting its attributes, or use an HTML parser. Add fixtures with a multiline quoted alt followed by an existing and a missing src, and a multiline quoted attribute value itself.
2. Blocking: scripts/check-readme-links.mjs:358
const html = startTags(text, open);
HTML extraction ignores Markdown escapes. For prose such as \<img src="missing.png">, GitHub renders the escaped opening angle bracket and the rest as literal text, not an image. startTags still finds <img and collects missing.png, so the new CI check rejects a document with no broken image. The same issue makes an escaped <a id="custom"> count as an anchor that does not exist.
Suggested fix:
Distinguish escaped opening angle brackets from actual HTML tags when extracting targets and anchors, while preserving real tags after an even number of backslashes. Add fixtures for escaped image examples and escaped anchor declarations.
rule:none
Fix at 3d7ee0c
Files: Tests: |
sprayberry-redline
left a comment
There was a problem hiding this comment.
Verdict: request changes. The earlier multiline-attribute, escaped-tag, spaced-attribute and code-span findings are addressed with regression fixtures. Review of the README, terminal renderer, CI step and checker tests found one remaining parsing-order defect that can reject valid documentation or let broken images pass.
1. Blocking: scripts/check-readme-links.mjs:334
const text = withoutCodeSpans(line);
Markdown processing runs inside raw HTML attributes before tags are parsed. For <img alt="[example](missing.md)" src="img/a.png"> with img/a.png present, the inline-link loop collects missing.md from the literal alt text and incorrectly fails the check. Conversely, <img alt="" src="missing.png" title=""> has its src removed by withoutCodeSpans, so the missing image passes. Backticks and bracketed text inside these quoted HTML attributes are literal attribute content, not Markdown code spans or links. The current HTML fixtures do not cover this interaction.
Suggested fix:
Respect raw HTML tag boundaries before interpreting Markdown spans and links, including tags continued across lines. Keep attribute content intact for HTML extraction and exclude it from Markdown extraction. Add fixtures for bracketed link syntax in alt text and backticks surrounding a missing src attribute.
rule:none
Fix at 83a7285[1] Markdown parsed inside raw HTML attributes (
Not done
Files: Tests: |
sprayberry-redline
left a comment
There was a problem hiding this comment.
Verdict: request changes. The earlier parsing findings are fixed in the current implementation. Review of the README, terminal renderer, CI step and link-checker fixtures found an unterminated-tag case that hides broken links, and the latest parsing-order fix still lacks its regression tests.
1. Blocking: scripts/check-readme-links.mjs:193
if (t && i + t[0].length === line.length) return { text: text + line.slice(last, i), tags, open: line.slice(i) };
An incomplete tag consumes the rest of the document even when it never becomes HTML. For example, a file containing Use <value, a blank line, and [bad](missing.md) has a rendered Markdown link, but this branch stores <value as open. Each subsequent line extends that open value, and targetsOf reaches EOF without checking missing.md. The check therefore passes a broken link.
Suggested fix:
Only commit to excluding a tag after its closing delimiter is found. If the candidate never closes, process the buffered text as Markdown instead. Add a fixture with an unterminated tag candidate followed by a missing link, while retaining the multiline quoted-attribute fixtures.
2. Blocking: scripts/check-readme-links.mjs:194
if (t) { tags.push(tagOf(t)); text += line.slice(last, i) + '\uFFFC'; i = last = i + t[0].length + 1; continue; }
The new tag-first parsing fixes the previous review's attribute-content cases, but neither test file exercises them. Existing tests cover multiline attributes and escapes, not Markdown link syntax inside alt text or paired backticks surrounding a src attribute. Reverting this parsing-order fix would therefore leave those regressions untested.
Suggested fix:
Add fixtures showing that `<img alt="[example](missing.md)" src="img/a.png">` passes when the image exists, and that an img tag with backticks in alt and title surrounding `src="missing.png"` fails. Include a continued-tag variant to cover the shared open-tag path.
rule:none
Fix at 70656d5[1] An unterminated tag hid the rest of the file ( [2] No regression tests for the tag-first parsing order (
The README and Files: Tests: |
sprayberry-redline
left a comment
There was a problem hiding this comment.
Verdict: request changes. The earlier multiline-attribute, escaped-tag, parsing-order and unterminated-tag findings are addressed, with regression fixtures. The README, terminal renderer, CI step and checker tests were reviewed; anchor extraction still accepts heading text inside a quoted HTML attribute, allowing a broken fragment link to pass.
1. Blocking: scripts/check-readme-links.mjs:359
const m = line.match(/^#{1,6}\s+(.+?)\s*#*\s*$/); const base = m && slugify(m[1]);
Heading extraction runs before inlineParts and ignores the open HTML-tag state. For a document containing <img alt="first, then # Ghost, then last" src="img/a.png">, followed by a blank line and [jump](#ghost), anchorsOf adds ghost. With the image present, checkFile passes even though Ghost is attribute text and the rendered page has no such heading or anchor. The multiline-tag handling correctly excludes that text from link extraction but not from heading extraction.
Suggested fix:
Derive headings only from lines outside continued HTML tags, sharing the parsing state used for target extraction. Add a fixture with a heading-shaped line inside a multiline quoted attribute and assert that a link to its apparent slug fails, while a real heading after the tag still resolves.
rule:none
Fix at 4cdc96b
Files: Tests: |
sprayberry-redline
left a comment
There was a problem hiding this comment.
Verdict: approve. The earlier parsing findings are addressed, including a regression fixture proving that heading-shaped text inside a multiline HTML attribute cannot satisfy a fragment link. The README, terminal renderer, CI step and link-checker tests are consistent with the change, and the checker now shares continued-tag state between target and anchor extraction.
Bumps the version to 0.10.5. On merge, auto-release.yml tags v0.10.5, cuts the GitHub release from the new CHANGELOG section and dispatches publish.yml to npm. Ships the --version fix (#211), the read-only lock directory fix (#227, now in the CHANGELOG) and the README rewrite (#316), which npm shows only from a release. The pinned version in the docs and issue templates moves with it; docker/ stays on 0.10.4 until 0.10.5 is on npm.
A rewrite of the README, plus the script that renders its terminal and a CI step that keeps its links honest.
Builds on #315 (merged). That PR marks
.github/readme/terminal.svgas captured output. The public-text check reads the mark from the PR's base commit, so this branch carries master with #315 in it; without the mark the check fails on the em dash the CLI itself prints in the recorded run.What the README now covers
truecopy-mcpproxy,truecopy guard).truecopy.lockentry and the verdict and exit code reference.Where the claims come from
Every command, flag, status, exit code and printed string was checked against
src/and a real run. Every figure carries its run date and links to where it comes from. The lock entry is this repo's own, unedited.Files
README.md: the rewrite.scripts/readme/terminal.mjsandlib.mjs: render.github/readme/terminal.svgfrom a real run of this checkout's CLI in a throwaway directory, with HOME, USERPROFILE and CANON_HOME redirected and the keychain off. The script asserts every verdict and exit code the README quotes and writes nothing if one moved.scripts/readme/README.mddocuments both README assets.scripts/readme/fonts/: Space Mono (OFL, licence included), embedded in the SVG so it renders the same everywhere.scripts/check-readme-links.mjsand oneci.ymlstep: fails when a relative link, image,srcsetcandidate or anchor in the README ordocs/stops resolving. Eachsrcsetcandidate is checked on its own, without its descriptor, a?queryis not part of the file name, and fragments are percent-decoded before they are matched. Link destinations are read the way GitHub serves them: spaces may stand around them, parentheses balance, the<...>form and titles parse, and character references are decoded before backslash escapes are undone, which is the order GitHub's renderer uses. References are decoded in attribute values and headings too; a heading with a named reference outside the checker's short table gets no slug, so a link to it fails instead of passing on a guess. Raw HTML tags are read the way GitHub reads them: an attribute may be double-quoted, single-quoted or unquoted, with spaces around its=, and a quoted value may run across lines. A code span or a tag, whichever starts first, takes its whole text, so link syntax and backticks inside an attribute are attribute text, and a#line inside a tag still open is no heading. An escaped<is text, and so is a tag that does not close before its paragraph ends, so the lines after it are still checked. Links and<a id>anchors inside fenced code or inline code spans are examples, so they are not counted; a fence closes only on a run of the same character, at least as long, and a code span only on a backtick run of exactly its opener's length. Heading slugs follow GitHub's slugger on the rendered heading text, so literal underscores, the text of code spans and backslash-escaped characters stay as written (\&is the text&,\<b>is no tag), a link in a heading leaves only its text even when its destination holds parentheses, and a repeated slug takes the next suffix not already taken. It reads markdown line by line, so it reads only#headings, and its header lists what that misses. It runs on one matrix leg and makes no network calls.test/readme-links.test.mjs: fixture tests for the checker (srcset with one and several candidates, missing images, files and anchors, encoded and malformed fragments, anchors that only appear in code, nested fences, unmatched and escaped backticks, file names with parentheses, destinations with spaces around them, character references and backslash escapes in GitHub's order, escaped characters in headings, headings that link to a file with parentheses, GitHub heading slugs and duplicate suffixes, skipped external targets) and for the script's own exit codes.test/readme-links-html.test.mjscovers raw HTML: quoted attributes that span lines (alt,srcset), escaped tags, a tag that never closes, a heading-shaped line inside an attribute, and link syntax and backticks insidealtandtitle.package.jsonis untouched, so this does not cut a release. npm and Glama keep showing the old README until the next release.Checked
npm teston the merged master: 329 tests, 325 pass, 4 skipped, 0 fail.node scripts/check-readme-links.mjs: 35 links and anchors in the README resolve, plus those indocs/.