Repository navigation
docs(plan): shape-based secret redaction with a prose corpus (S-12 of #234) - #288
Merged
Merged
Conversation
Plan only: src/ and tests/test_auth.py are byte-identical to main @ 860636a. The redactor is keyword-anchored, so it redacts the word after `token`, `secret`, `session`, `password` and `bearer` in prose while letting keyword-less credentials (AKIA…, ghp_…, xoxb-…, a token in a URL path) through. The plan specifies separator-anchored keyword rules with the key as the identifier's last segment, a value gate for whitespace-separated keywords and for `Bearer`, an OAuth-only reading of `code`, a shape layer (vendor supplement, prefixed tokens, class-transition scoring of opaque runs with a 256-char payload bound), a padding-safe split in `redact_secrets`, and a truncation cut that never ends inside a token. Both directions are acceptance criteria: a prose corpus that must survive byte-identically and a 29-class credential corpus that must not. The inherited spike was measured, not trusted: ten defects were found and fixed (elicitation-URL path redaction, JSON-RPC `code`, `bearer token`, the `\bbearer` boundary, `0x` addresses, image-sized base64, pod names, newline-spanning separators, `param=value` after a keyword, base64-padding split). Its tests/test_auth.py edit is dropped: both original fixtures pass unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013G8iK4noj1iNbSTqJVFEX6
Plan only: src/ and tests/test_auth.py are byte-identical to main @ 860636a. The redactor is keyword-anchored, so it redacts the word after `token`, `secret`, `session`, `password` and `bearer` in prose while letting keyword-less credentials (AKIA…, ghp_…, xoxb-…, a token in a URL path) through. The plan specifies separator-anchored keyword rules with the key as the identifier's last segment, a value gate for whitespace-separated keywords and for `Bearer`, an OAuth-only reading of `code`, a shape layer (vendor supplement, prefixed tokens, class-transition scoring of opaque runs with a 256-char payload bound), a padding-safe split in `redact_secrets`, and a truncation cut that never ends inside a token. Both directions are acceptance criteria: a prose corpus that must survive byte-identically and a 29-class credential corpus that must not. The inherited spike was measured, not trusted: ten defects were found and fixed (elicitation-URL path redaction, JSON-RPC `code`, `bearer token`, the `\bbearer` boundary, `0x` addresses, image-sized base64, pod names, newline-spanning separators, `param=value` after a keyword, base64-padding split). Its tests/test_auth.py edit is dropped: both original fixtures pass unchanged. Rev 2: the board found three blocking defects in rev 1, all reproduced and fixed with tests and mutation rows M18-M20 -- the payload bound was applied after path splitting (JPEG base64 `/9j/...` was destroyed), the truncation back-up was bounded on the whole trailing run (a credential after a long path kept its prefix; offset 0 kept a partial token), and quoted values stopped at an escaped quote (`{"password": "a\"hunter2"}` left `hunter2"`). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013G8iK4noj1iNbSTqJVFEX6
Plan only: src/ and tests/test_auth.py are byte-identical to main @ 860636a. The redactor is keyword-anchored, so it redacts the word after `token`, `secret`, `session`, `password` and `bearer` in prose while letting keyword-less credentials (AKIA…, ghp_…, xoxb-…, a token in a URL path) through. The plan specifies separator-anchored keyword rules with the key as the identifier's last segment, a value gate for whitespace-separated keywords and for `Bearer`, an OAuth-only reading of `code`, a shape layer (vendor supplement, prefixed tokens, class-transition scoring of opaque runs with a 256-char payload bound), a padding-safe split in `redact_secrets`, and a truncation cut that never ends inside a token. Both directions are acceptance criteria: a prose corpus that must survive byte-identically and a 29-class credential corpus that must not. The inherited spike was measured, not trusted: ten defects were found and fixed (elicitation-URL path redaction, JSON-RPC `code`, `bearer token`, the `\bbearer` boundary, `0x` addresses, image-sized base64, pod names, newline-spanning separators, `param=value` after a keyword, base64-padding split). Its tests/test_auth.py edit is dropped: both original fixtures pass unchanged. Rev 2: the board found three blocking defects in rev 1, all reproduced and fixed with tests and mutation rows M18-M20 -- the payload bound was applied after path splitting (JPEG base64 `/9j/...` was destroyed), the truncation back-up was bounded on the whole trailing run (a credential after a long path kept its prefix; offset 0 kept a partial token), and quoted values stopped at an escaped quote (`{"password": "a\"hunter2"}` left `hunter2"`). Rev 3: two more board defects, both reproduced and fixed with tests and mutation rows M21-M24 -- the Bearer pass ran before the keyword pass with a value class that ate the closing quote (`{"password": "hunter2 Bearer x"}` left `hunter2` visible; the Authorization rule missed a JSON-quoted key), and a truncation cut inside a quoted multi-word password leaked its first word (128 of 1452 swept cuts). Keyed values are now redacted first, every looser value class stops at quotes and brackets, and an unterminated quoted value runs to the end of its line; the sweep is 0 of 1452 with coverage asserted. mypy src/ is clean and in the verification. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013G8iK4noj1iNbSTqJVFEX6
Plan only: src/ and tests/test_auth.py are byte-identical to main @ 860636a. The redactor is keyword-anchored, so it redacts the word after `token`, `secret`, `session`, `password` and `bearer` in prose while letting keyword-less credentials (AKIA…, ghp_…, xoxb-…, a token in a URL path) through. The plan specifies separator-anchored keyword rules with the key as the identifier's last segment, a value gate for whitespace-separated keywords and for `Bearer`, an OAuth-only reading of `code`, a shape layer (vendor supplement, prefixed tokens, class-transition scoring of opaque runs with a 256-char payload bound), a padding-safe split in `redact_secrets`, and a truncation cut that never ends inside a token. Both directions are acceptance criteria: a prose corpus that must survive byte-identically and a 29-class credential corpus that must not. The inherited spike was measured, not trusted: ten defects were found and fixed (elicitation-URL path redaction, JSON-RPC `code`, `bearer token`, the `\bbearer` boundary, `0x` addresses, image-sized base64, pod names, newline-spanning separators, `param=value` after a keyword, base64-padding split). Its tests/test_auth.py edit is dropped: both original fixtures pass unchanged. Rev 2: the board found three blocking defects in rev 1, all reproduced and fixed with tests and mutation rows M18-M20 -- the payload bound was applied after path splitting (JPEG base64 `/9j/...` was destroyed), the truncation back-up was bounded on the whole trailing run (a credential after a long path kept its prefix; offset 0 kept a partial token), and quoted values stopped at an escaped quote (`{"password": "a\"hunter2"}` left `hunter2"`). Rev 3: two more board defects, both reproduced and fixed with tests and mutation rows M21-M24 -- the Bearer pass ran before the keyword pass with a value class that ate the closing quote (`{"password": "hunter2 Bearer x"}` left `hunter2` visible; the Authorization rule missed a JSON-quoted key), and a truncation cut inside a quoted multi-word password leaked its first word (128 of 1452 swept cuts). Keyed values are now redacted first, every looser value class stops at quotes and brackets, and an unterminated quoted value runs to the end of its line; the sweep is 0 of 1452 with coverage asserted. mypy src/ is clean and in the verification. Rev 4: the architecture changes so both defect classes close by construction. Every pass now reads the original text and returns spans; overlaps are merged and all replacements land in one step, so no pass can consume a boundary a later pass needs (the round-3 URL-eats-escape leak). process_output redacts BEFORE it truncates, over a bounded window (max_bytes + 16 KiB), so the cut never lands inside a credential the redactor has not seen whole (the round-3 backslash-cut leak); the trailing-run back-up and the end-of-line quoted-value rule are gone. Property tests replace hand-picked cases: 3000 random printable passwords x 5 keyed forms on both surfaces, a 23 713-cut truncation sweep with coverage asserted, and 2000 generated prose lines. Mutation rows M25 (mutate in place) and M26 (truncate then redact) are red under the property tests. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013G8iK4noj1iNbSTqJVFEX6
Plan only: src/ and tests/test_auth.py are byte-identical to main @ 860636a. The redactor is keyword-anchored, so it redacts the word after `token`, `secret`, `session`, `password` and `bearer` in prose while letting keyword-less credentials (AKIA…, ghp_…, xoxb-…, a token in a URL path) through. The plan specifies separator-anchored keyword rules with the key as the identifier's last segment, a value gate for whitespace-separated keywords and for `Bearer`, an OAuth-only reading of `code`, a shape layer (vendor supplement, prefixed tokens, class-transition scoring of opaque runs with a 256-char payload bound), a padding-safe split in `redact_secrets`, and a truncation cut that never ends inside a token. Both directions are acceptance criteria: a prose corpus that must survive byte-identically and a 29-class credential corpus that must not. The inherited spike was measured, not trusted: ten defects were found and fixed (elicitation-URL path redaction, JSON-RPC `code`, `bearer token`, the `\bbearer` boundary, `0x` addresses, image-sized base64, pod names, newline-spanning separators, `param=value` after a keyword, base64-padding split). Its tests/test_auth.py edit is dropped: both original fixtures pass unchanged. Rev 2: the board found three blocking defects in rev 1, all reproduced and fixed with tests and mutation rows M18-M20 -- the payload bound was applied after path splitting (JPEG base64 `/9j/...` was destroyed), the truncation back-up was bounded on the whole trailing run (a credential after a long path kept its prefix; offset 0 kept a partial token), and quoted values stopped at an escaped quote (`{"password": "a\"hunter2"}` left `hunter2"`). Rev 3: two more board defects, both reproduced and fixed with tests and mutation rows M21-M24 -- the Bearer pass ran before the keyword pass with a value class that ate the closing quote (`{"password": "hunter2 Bearer x"}` left `hunter2` visible; the Authorization rule missed a JSON-quoted key), and a truncation cut inside a quoted multi-word password leaked its first word (128 of 1452 swept cuts). Keyed values are now redacted first, every looser value class stops at quotes and brackets, and an unterminated quoted value runs to the end of its line; the sweep is 0 of 1452 with coverage asserted. mypy src/ is clean and in the verification. Rev 4: the architecture changes so both defect classes close by construction. Every pass now reads the original text and returns spans; overlaps are merged and all replacements land in one step, so no pass can consume a boundary a later pass needs (the round-3 URL-eats-escape leak). process_output redacts BEFORE it truncates, over a bounded window (max_bytes + 16 KiB), so the cut never lands inside a credential the redactor has not seen whole (the round-3 backslash-cut leak); the trailing-run back-up and the end-of-line quoted-value rule are gone. Property tests replace hand-picked cases: 3000 random printable passwords x 5 keyed forms on both surfaces, a 23 713-cut truncation sweep with coverage asserted, and 2000 generated prose lines. Mutation rows M25 (mutate in place) and M26 (truncate then redact) are red under the property tests. Rev 5: key coverage as a property. The lead's own property test found keys the redactor names but did not redact in every form (passwd/pwd via the policy default only, quoted Authorization stopping at a space, private_key/ credentials/auth uncovered). AUTH_DIAGNOSTIC_SECRET_KEYS is widened, the public WEAK_SECRET_KEYS keeps plain words after auth/code/credential(s), a quoted Authorization value is redacted whole, a bare value may start with punctuation, and an existing [REDACTED] is inert to every span. The new test derives its keys from the source (31 keys x 4 forms, 6158 strings, 0 leaks); M27 drops passwd from the strong set and the property goes red. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013G8iK4noj1iNbSTqJVFEX6
Plan only: src/ and tests/test_auth.py are byte-identical to main @ 860636a. The redactor is keyword-anchored, so it redacts the word after `token`, `secret`, `session`, `password` and `bearer` in prose while letting keyword-less credentials (AKIA…, ghp_…, xoxb-…, a token in a URL path) through. The plan specifies separator-anchored keyword rules with the key as the identifier's last segment, a value gate for whitespace-separated keywords and for `Bearer`, an OAuth-only reading of `code`, a shape layer (vendor supplement, prefixed tokens, class-transition scoring of opaque runs with a 256-char payload bound), a padding-safe split in `redact_secrets`, and a truncation cut that never ends inside a token. Both directions are acceptance criteria: a prose corpus that must survive byte-identically and a 29-class credential corpus that must not. The inherited spike was measured, not trusted: ten defects were found and fixed (elicitation-URL path redaction, JSON-RPC `code`, `bearer token`, the `\bbearer` boundary, `0x` addresses, image-sized base64, pod names, newline-spanning separators, `param=value` after a keyword, base64-padding split). Its tests/test_auth.py edit is dropped: both original fixtures pass unchanged. Rev 2: the board found three blocking defects in rev 1, all reproduced and fixed with tests and mutation rows M18-M20 -- the payload bound was applied after path splitting (JPEG base64 `/9j/...` was destroyed), the truncation back-up was bounded on the whole trailing run (a credential after a long path kept its prefix; offset 0 kept a partial token), and quoted values stopped at an escaped quote (`{"password": "a\"hunter2"}` left `hunter2"`). Rev 3: two more board defects, both reproduced and fixed with tests and mutation rows M21-M24 -- the Bearer pass ran before the keyword pass with a value class that ate the closing quote (`{"password": "hunter2 Bearer x"}` left `hunter2` visible; the Authorization rule missed a JSON-quoted key), and a truncation cut inside a quoted multi-word password leaked its first word (128 of 1452 swept cuts). Keyed values are now redacted first, every looser value class stops at quotes and brackets, and an unterminated quoted value runs to the end of its line; the sweep is 0 of 1452 with coverage asserted. mypy src/ is clean and in the verification. Rev 4: the architecture changes so both defect classes close by construction. Every pass now reads the original text and returns spans; overlaps are merged and all replacements land in one step, so no pass can consume a boundary a later pass needs (the round-3 URL-eats-escape leak). process_output redacts BEFORE it truncates, over a bounded window (max_bytes + 16 KiB), so the cut never lands inside a credential the redactor has not seen whole (the round-3 backslash-cut leak); the trailing-run back-up and the end-of-line quoted-value rule are gone. Property tests replace hand-picked cases: 3000 random printable passwords x 5 keyed forms on both surfaces, a 23 713-cut truncation sweep with coverage asserted, and 2000 generated prose lines. Mutation rows M25 (mutate in place) and M26 (truncate then redact) are red under the property tests. Rev 5: key coverage as a property. The lead's own property test found keys the redactor names but did not redact in every form (passwd/pwd via the policy default only, quoted Authorization stopping at a space, private_key/ credentials/auth uncovered). AUTH_DIAGNOSTIC_SECRET_KEYS is widened, the public WEAK_SECRET_KEYS keeps plain words after auth/code/credential(s), a quoted Authorization value is redacted whole, a bare value may start with punctuation, and an existing [REDACTED] is inert to every span. The new test derives its keys from the source (31 keys x 4 forms, 6158 strings, 0 leaks); M27 drops passwd from the strong set and the property goes red. Rev 6: three rules inside the span design leaked, two of them regressions against main, and are replaced. The URL pass yields component spans (userinfo, secret query values, fragment) instead of a rewritten URL that swallowed every keyed and shape span inside its query; a literal [REDACTED] in the input is a span that merges with what touches it instead of a shield that disarmed the rule next to it; and process_output keeps only the cap plus the tail of any span that starts before it, so the window's far edge is never emitted. `redact_auth_url` is byte-identical to main again. New properties: every span the redactor collected is absent from the output (over the corpora and generators for query strings and literal markers), the window edge never emits an unredacted value, and a 42-input corpus of what main redacts still vanishes. Authorization gains the plain-word gate and same-line separator; `==` is a comparison. Mutation rows M28-M30, M32, M33. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013G8iK4noj1iNbSTqJVFEX6
Plan only: src/ and tests/test_auth.py are byte-identical to main @ 860636a. The redactor is keyword-anchored, so it redacts the word after `token`, `secret`, `session`, `password` and `bearer` in prose while letting keyword-less credentials (AKIA…, ghp_…, xoxb-…, a token in a URL path) through. The plan specifies separator-anchored keyword rules with the key as the identifier's last segment, a value gate for whitespace-separated keywords and for `Bearer`, an OAuth-only reading of `code`, a shape layer (vendor supplement, prefixed tokens, class-transition scoring of opaque runs with a 256-char payload bound), a padding-safe split in `redact_secrets`, and a truncation cut that never ends inside a token. Both directions are acceptance criteria: a prose corpus that must survive byte-identically and a 29-class credential corpus that must not. The inherited spike was measured, not trusted: ten defects were found and fixed (elicitation-URL path redaction, JSON-RPC `code`, `bearer token`, the `\bbearer` boundary, `0x` addresses, image-sized base64, pod names, newline-spanning separators, `param=value` after a keyword, base64-padding split). Its tests/test_auth.py edit is dropped: both original fixtures pass unchanged. Rev 2: the board found three blocking defects in rev 1, all reproduced and fixed with tests and mutation rows M18-M20 -- the payload bound was applied after path splitting (JPEG base64 `/9j/...` was destroyed), the truncation back-up was bounded on the whole trailing run (a credential after a long path kept its prefix; offset 0 kept a partial token), and quoted values stopped at an escaped quote (`{"password": "a\"hunter2"}` left `hunter2"`). Rev 3: two more board defects, both reproduced and fixed with tests and mutation rows M21-M24 -- the Bearer pass ran before the keyword pass with a value class that ate the closing quote (`{"password": "hunter2 Bearer x"}` left `hunter2` visible; the Authorization rule missed a JSON-quoted key), and a truncation cut inside a quoted multi-word password leaked its first word (128 of 1452 swept cuts). Keyed values are now redacted first, every looser value class stops at quotes and brackets, and an unterminated quoted value runs to the end of its line; the sweep is 0 of 1452 with coverage asserted. mypy src/ is clean and in the verification. Rev 4: the architecture changes so both defect classes close by construction. Every pass now reads the original text and returns spans; overlaps are merged and all replacements land in one step, so no pass can consume a boundary a later pass needs (the round-3 URL-eats-escape leak). process_output redacts BEFORE it truncates, over a bounded window (max_bytes + 16 KiB), so the cut never lands inside a credential the redactor has not seen whole (the round-3 backslash-cut leak); the trailing-run back-up and the end-of-line quoted-value rule are gone. Property tests replace hand-picked cases: 3000 random printable passwords x 5 keyed forms on both surfaces, a 23 713-cut truncation sweep with coverage asserted, and 2000 generated prose lines. Mutation rows M25 (mutate in place) and M26 (truncate then redact) are red under the property tests. Rev 5: key coverage as a property. The lead's own property test found keys the redactor names but did not redact in every form (passwd/pwd via the policy default only, quoted Authorization stopping at a space, private_key/ credentials/auth uncovered). AUTH_DIAGNOSTIC_SECRET_KEYS is widened, the public WEAK_SECRET_KEYS keeps plain words after auth/code/credential(s), a quoted Authorization value is redacted whole, a bare value may start with punctuation, and an existing [REDACTED] is inert to every span. The new test derives its keys from the source (31 keys x 4 forms, 6158 strings, 0 leaks); M27 drops passwd from the strong set and the property goes red. Rev 6: three rules inside the span design leaked, two of them regressions against main, and are replaced. The URL pass yields component spans (userinfo, secret query values, fragment) instead of a rewritten URL that swallowed every keyed and shape span inside its query; a literal [REDACTED] in the input is a span that merges with what touches it instead of a shield that disarmed the rule next to it; and process_output keeps only the cap plus the tail of any span that starts before it, so the window's far edge is never emitted. `redact_auth_url` is byte-identical to main again. New properties: every span the redactor collected is absent from the output (over the corpora and generators for query strings and literal markers), the window edge never emits an unredacted value, and a 42-input corpus of what main redacts still vanishes. Authorization gains the plain-word gate and same-line separator; `==` is a comparison. Mutation rows M28-M30, M32, M33. The CLI seats' supplementary cases (userinfo plus a non-secret query key, an operator pattern in a URL path, a window shrink from ordinary keyed values) are named tests. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013G8iK4noj1iNbSTqJVFEX6
Plan only: src/ and tests/test_auth.py are byte-identical to main @ 860636a. The redactor is keyword-anchored, so it redacts the word after `token`, `secret`, `session`, `password` and `bearer` in prose while letting keyword-less credentials (AKIA…, ghp_…, xoxb-…, a token in a URL path) through. The plan specifies separator-anchored keyword rules with the key as the identifier's last segment, a value gate for whitespace-separated keywords and for `Bearer`, an OAuth-only reading of `code`, a shape layer (vendor supplement, prefixed tokens, class-transition scoring of opaque runs with a 256-char payload bound), a padding-safe split in `redact_secrets`, and a truncation cut that never ends inside a token. Both directions are acceptance criteria: a prose corpus that must survive byte-identically and a 29-class credential corpus that must not. The inherited spike was measured, not trusted: ten defects were found and fixed (elicitation-URL path redaction, JSON-RPC `code`, `bearer token`, the `\bbearer` boundary, `0x` addresses, image-sized base64, pod names, newline-spanning separators, `param=value` after a keyword, base64-padding split). Its tests/test_auth.py edit is dropped: both original fixtures pass unchanged. Rev 2: the board found three blocking defects in rev 1, all reproduced and fixed with tests and mutation rows M18-M20 -- the payload bound was applied after path splitting (JPEG base64 `/9j/...` was destroyed), the truncation back-up was bounded on the whole trailing run (a credential after a long path kept its prefix; offset 0 kept a partial token), and quoted values stopped at an escaped quote (`{"password": "a\"hunter2"}` left `hunter2"`). Rev 3: two more board defects, both reproduced and fixed with tests and mutation rows M21-M24 -- the Bearer pass ran before the keyword pass with a value class that ate the closing quote (`{"password": "hunter2 Bearer x"}` left `hunter2` visible; the Authorization rule missed a JSON-quoted key), and a truncation cut inside a quoted multi-word password leaked its first word (128 of 1452 swept cuts). Keyed values are now redacted first, every looser value class stops at quotes and brackets, and an unterminated quoted value runs to the end of its line; the sweep is 0 of 1452 with coverage asserted. mypy src/ is clean and in the verification. Rev 4: the architecture changes so both defect classes close by construction. Every pass now reads the original text and returns spans; overlaps are merged and all replacements land in one step, so no pass can consume a boundary a later pass needs (the round-3 URL-eats-escape leak). process_output redacts BEFORE it truncates, over a bounded window (max_bytes + 16 KiB), so the cut never lands inside a credential the redactor has not seen whole (the round-3 backslash-cut leak); the trailing-run back-up and the end-of-line quoted-value rule are gone. Property tests replace hand-picked cases: 3000 random printable passwords x 5 keyed forms on both surfaces, a 23 713-cut truncation sweep with coverage asserted, and 2000 generated prose lines. Mutation rows M25 (mutate in place) and M26 (truncate then redact) are red under the property tests. Rev 5: key coverage as a property. The lead's own property test found keys the redactor names but did not redact in every form (passwd/pwd via the policy default only, quoted Authorization stopping at a space, private_key/ credentials/auth uncovered). AUTH_DIAGNOSTIC_SECRET_KEYS is widened, the public WEAK_SECRET_KEYS keeps plain words after auth/code/credential(s), a quoted Authorization value is redacted whole, a bare value may start with punctuation, and an existing [REDACTED] is inert to every span. The new test derives its keys from the source (31 keys x 4 forms, 6158 strings, 0 leaks); M27 drops passwd from the strong set and the property goes red. Rev 6: three rules inside the span design leaked, two of them regressions against main, and are replaced. The URL pass yields component spans (userinfo, secret query values, fragment) instead of a rewritten URL that swallowed every keyed and shape span inside its query; a literal [REDACTED] in the input is a span that merges with what touches it instead of a shield that disarmed the rule next to it; and process_output keeps only the cap plus the tail of any span that starts before it, so the window's far edge is never emitted. `redact_auth_url` is byte-identical to main again. New properties: every span the redactor collected is absent from the output (over the corpora and generators for query strings and literal markers), the window edge never emits an unredacted value, and a 42-input corpus of what main redacts still vanishes. Authorization gains the plain-word gate and same-line separator; `==` is a comparison. Mutation rows M28-M30, M32, M33. The CLI seats' supplementary cases (userinfo plus a non-secret query key, an operator pattern in a URL path, a window shrink from ordinary keyed values) are named tests. Rev 7: the differential against main is the never-worse test. The board seat's 4000-row corpus is regenerated in-test and main @ 860636a's recorded removals are a committed oracle fixture (beside this plan; the implementer copies it to tests/fixtures/); a row where rev 7 keeps a piece main removed fails unless it is in an accepted class decided from the row's own key, separator and value (0 unaccepted; 226 row-surfaces in 5 classes; 1852 rows better than main). Four regressions against main are fixed (a quote before Bearer; httpie := and ==; percent-encoded query values; Ruby =>), the serialisation layer is in scope -- structured results are redacted as structure before json.dumps -- and M28 is a two-rule mutant red for its stated reason. Mutation rows M34-M38. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013G8iK4noj1iNbSTqJVFEX6
Embeds the frozen rev-8 code (origin/wip/234-redactor-rev8-code @ d316364) verbatim: the src/ patch against 860636a, tests/test_redaction.py (149 tests) and the main oracle fixture (now with dict_types). - Structured layer removed; serialisation is decision (c), scoped out to #290. Keyed quoted values are redacted inside their quotes, so a dict result round-trips as a dict (14 str of 400, all among main's 30) and no corpus JSON row breaks (0 of 258; main 7). - CRLF / no-break-space regressions fixed; decode depth bounded at 3, failing closed; policy patterns asked about decoded query values; `if token == expected:` a comparison on both surfaces; keyed-list pass. - Differential re-measured on rev 8: 0 unaccepted, 294 accepted row-surfaces in 6 classes, 1 737 rows better; per-key and property counts re-measured. Mutation rows Q1-Q3, D1, D2 red on the frozen code; Q4 equivalent; M37 retired. Full suite 4213 passed, 3 skipped, 25 deselected. - Stale text fixed (path_transform, %5BREDACTED%5D, the redact-before- truncate non-goal, _TRAILING_*); new residual stated (policy defaults eat the opening quote of a plain-text quoted value, as on main). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The code branch dropped stale rev-7 wording and the no-op `not inner` check in `_keyword_list_spans`. Re-embeds `git diff 860636a origin/wip/234-redactor-rev8-code -- src/` and tests/test_redaction.py at 56d7f80, updates the sha256 values and How-to-apply ranges, and removes the text that described the check as still present. D1, D2, Q1-Q3 re-run on 56d7f80 with the same failures (Q3 now a 2-line diff). The oracle is unchanged (cmp). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ViperJuice
added a commit
that referenced
this pull request
Sep 26, 2026
grok, codex and gemini on plan rev 8 (#288); each reproduced on main vs this tree before fixing: - `==` then whitespace is a separator on the engine (main's `[\s:=]+`): `password == hunter2` is redacted; a non-credential-shaped value is a comparison (`if token == expected:` survives). - A percent-encoded query KEY is decoded and the `key=value` pair asked of the engine (`?api%2Dkey=hunter2`). - `Bearer` continues onto an unindented line only with a credential-shaped value that is not a flag/bullet (`Bearer\r\nhunter2`). - The whitespace-rule and Authorization bare values never end on a backslash (a serialised leaf's `\"` stays escaped; dict stays dict). - The keyed-list body is quote-aware (`["first]", "hunter2"]` -- leaked on main too) and honours the `code` qualifier exemption (`error_codes: [...]`). - Dropped the `&` accepted-class arm: it read the output and could not fire (gemini). Seat JSON fuzz: see commit message of the next plan revision. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017HCiJU6dVVrNdcz461oN33
…gs (see #234) The board round on rev 8 came back DISAGREE from all four seats; every blocking finding sat outside the corpus the never-worse test generated. Embeds the frozen rev-9 code (origin/wip/234-redactor-rev9-code @ 8e7d98c) verbatim: the src/ patch against 860636a, tests/test_redaction.py (306 tests), the oracle fixture (now with fuzz_types) and the adapted oracle regenerator. - Revision 9 entry: B1-B5, N1-N4 (claude), G1 (grok), C2/C4/C6/C7 (codex), gemini's `&` arm, and the six classes the widened differential found (list straddle, JSON escape, whitespace suffix, case-folded keys, dash-led value, other-quote straddle), each with its rule change. - Three-tree before/after table (main, rev 8, rev 9) for every class. - The generator's axes stated; accepted classes re-measured: 8 000 rows, 1 902 better, 1 792 worse pairs in 10 classes, 0 unaccepted; dict 786 dict->dict, 14 str->dict, 0 dict->str; JSON fuzz 3 272 in scope, 0 broken; 810 JSON rows, 0 broken (main 8). - Red/green: 144 of 306 red on rev 8, all green on rev 9. The seven round-3 mutants and D1, D2, Q1-Q3 re-run on rev 9: all red. M01-M38 marked not re-run. - Full suite on 8e7d98c: 4370 passed, 3 skipped, 25 deselected. - Rev 8's opening-quote residual is fixed; new residuals stated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#234) Three board rounds each found inputs worse than main on an axis the differential never generated, because every axis was added after a finding. With the maintainer's approval the never-worse differential is now derived from the oracle's grammar: main's keyword, Authorization, Bearer and URL rules and policy defaults, plus every escape and scalar json.dumps emits. - Revision 10 entry: the method change and why; the fixes (F1-F5, F8, F9, N1, N2, N4b, N5, N6/N7, N8, N9, the URL backslash) and the maintainer's seven decisions. - Audit table A (one row per construct of main's regexes and policy defaults) and table B (the serialiser's escapes and scalars), each row with rev 10's equivalent and a verdict: a superset, or a named class. Evidence is the tier-2 corpus observed on both trees, per bucket. - The generator's axes, the density of both tiers, and runtimes. Accepted classes: tier 1 12 535 rows, tier 2 166 183, a second seed pair: 0 unaccepted on each. C7 is never used. - Red/green (157 of 472 red on rev 9) and 21 of 21 mutants killed, each run over the whole default tier. - Cross-reference: F1-F10 and every audit gap, to its fix or class and evidence. - Embeds e79d75f verbatim: the src/ patch, test_redaction.py, _redaction_grammar.py, regen_redaction_main_oracle.py, the pyproject marker, and the oracle (cmp-identical). The rev-9 class counts are marked superseded. - Full suite on e79d75f: 4536 passed, 3 skipped, 40 deselected. The coordinator's teardown error is environmental. - No Cc/Cf/Zl/Zp or U+0085 characters left in the plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…p (see #234) Revisions 7-10 re-implemented main's rules and each was shown worse than main on some input. Rev 11 replays main's rules unchanged as candidate spans (the floor); rev 10's rules only add; a floor span is dropped only by one of 18 named suppression predicates. With the maintainer's decisions: the four syntax predicates approved, N11 narrowed (not after Bearer/Authorization, not on a credential-shaped value), and rev 10's false-positive fixes that main does not make left as main behaves. The plan states the design, the guarantee and its proof (fidelity oracle, construction tests on both tiers, the board rows and the fuzz, B1-B5 and N11 red/green with mutants, suppression counts), the rev-10 findings and their resolution, a condensed history, Verification with CI cost, Acceptance criteria, Non-goals and Unverified. The code is embedded as one patch against main 876fd33 (floor branch 3e49b95), proven to apply and to reproduce the branch's files byte for byte. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LT8D8tRmExyQve69DCWRik
… generated timing sweep (see #234) Round 1 on rev 11 (claude seat, DISAGREE) found the floor held on an independent differential but three helpers grew quadratically and N10, N3 and D2's plain number fired wider than approved. Rev 12: * linear floor helpers (right-to-left closing-wrapper scan, reverse search for an unindented break, predicates once per part of a match), and a timing sweep generated from the redactor's own 66 patterns (46 096 shapes through every entry point; 11 in the default tier, all in the slow tier); * N10 only on a segment of an arn:/urn: name; N3 only on a glued run; a plain number is at most 10 decimal digits; every URL position labelled; * a Rev 11 board findings table (red on 3e49b95, green, mutant), updated suppression counts and residuals, and CI cost. The code is embedded as one patch against main 876fd33 (floor branch b7b071d), re-proven to apply and to reproduce the branch's files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LT8D8tRmExyQve69DCWRik
…234) The proof paragraph (extract, cmp, git apply on a fresh main, diff -rq against the floor branch b7b071d, gates and the redaction test files on that tree) and one unreproduced default-tier failure under Unverified. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LT8D8tRmExyQve69DCWRik
…n a work count, shapes from the loops (see #234) Round 2 on rev 12 (claude seat, DISAGREE): round 1 stayed closed and the floor held, but two quadratic paths reached only by composed passes (B-3, a URL marker standing for the whole URL; B-4, every keyword match asking every resource name), flaky clock-ratio tests (T-1) and camelCase keys read as glued (N-5). Rev 13 records the fixes, a Rev 12 board findings table (red on b7b071d, green, mutant), a per-function complexity argument for the floor, the additive rules, the merge and the policy surface, the backtracking argument per pattern, the work-count method and the composition shapes, residuals N-6 and N-7, CI cost, three default-tier runs (one under load) and the slow tier. T-1 accounts for rev 12's unexplained failure. The code is embedded as a patch against main 876fd33 (floor branch 4b5cac8), proven to apply unchanged to main 8de49e4. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LT8D8tRmExyQve69DCWRik
…, memory linearity, shapes inside atomised pieces (see #234) Round 3 on rev 13 (claude seat, DISAGREE on B-5): a re-encoded URL component is one piece, and every later match inside it copied the whole piece (B-3's class). Rev 14 records the fix as a class (a per-step frontier in _Tracked.source; removed text copied only where a predicate reads it; one application per input range), an audit of every place a span or value can be larger than its match, exact work charges, a tracemalloc linearity check, the atomised-piece shape family (156 of 476 super-linear on 4b5cac8, 0 here), S-1's backward walks, a Rev 13 board findings table, N-8 in the residuals, CI cost, and the runs. The code is embedded as a patch against main 876fd33 (floor branch b664588), proven to apply unchanged to main 8de49e4. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LT8D8tRmExyQve69DCWRik
#234) The maintainer's decision: additive only. The redactor runs as it is, and rev 10's shape-based and separator-anchored rules run over its output, writing only markers, so the floor holds by construction. The replay-based floor of revs 11-14 is abandoned (one paragraph says why), with its suppression predicates and recorded oracle (the oracle file is removed). The plan states the design, the four guarantees and their tests (floor against a vendored copy of the redactor, JSON, linear work and memory, markers) with a mutant each, the residuals including the dropped exceptions as possible future changes, CI cost measured on GitHub runners through a closed draft PR (test jobs 12-14 minutes), acceptance, non-goals, and the change embedded as a patch against main 260cc1a with its proof. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LT8D8tRmExyQve69DCWRik
…yer (see #234) Every seat held the floor by construction. Rev 16 records the additive layer's fixes, each as a class with a Rev 15 board findings table (red on ff73fd0, green, mutant): markers left whole on every surface (a span over a marker is dropped, a policy value stops at & in a query), string values only for JSON (no key collisions), an unambiguous list body plus a sweep over every quantifier of every redactor regex, unrolled string patterns (peak memory ~40 MB at 1 MB), the key prefilter as the patterns' own alternation (the reported multi-character fold gap checked: Python's re does not apply them), and upper-case URL schemes. Residuals restate the slow-input cost and the 400-character cut. The change is embedded as a patch against main 260cc1a (wip/234-additive e77c691), with its proof. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LT8D8tRmExyQve69DCWRik
…ch (see #234) Round 2 on rev 16 held the floor, the markers, the prefilter and linearity, and found one regression (F1): every additive span overlapping a marker was dropped, so a strong key's value the redactor partly marked showed the rest. Rev 17 cuts a span that starts before a marker around the markers; a span that starts on one is still dropped. The plan adds a Rev 16 board findings table (red on e77c691, green, mutant), tests derived from the redactor's own key list, a rev 15 against rev 17 differential on the credential corpora (every difference is a marker rev 15 dropped, re-spelled or swallowed, or a whitespace escape rev 15 read as a Bearer value), and discloses keys left unredacted (F2) and the Bearer escape check (F3; a first cut at changing it was quadratic, found by the quantifier sweep, and reverted). The change is embedded as a patch against main 260cc1a (wip/234-additive 8aae554). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LT8D8tRmExyQve69DCWRik
…glued to it (see #234) Rev 17's board, round 3: the merge rule held; F-A found the residual rev 17 disclosed as one shape to be a class. The redactor's keyword rule stops at the first character outside its value class and rev 17 dropped a span starting on that marker, so the rest of the value stayed. - adopts the seat's measured fix: keep the marker, replace the run glued to its end (to whitespace, `&`, `#`, a quote, a backslash or the next marker); an AWS partition under an `...arn` key is exempt; - tests read the end of a value on all four surfaces (red on 8aae554: 902 of 4 800; green: 0; rev 17's rule as the mutant); - residuals restated as measured classes; rev 17's 248 count superseded by a rev 15 against rev 18 differential with its classification; - two further changes measured and left to the maintainer (D1, D2); - patch regenerated against 260cc1a from eacfdb7; embedding proof re-run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LT8D8tRmExyQve69DCWRik
…ter or digit (see #234) Maintainer decisions on rev 18: - D1 adopted: a glued run without a letter or a digit is left as written; prose, 2 000 generated lines and 2 352 key word + word + punctuation lines equal the redactor's output (360 differ under rev 18's rule, the mutant); the punctuation-only tail is disclosed; - D2 not approved: the tail under a whitespace-keyed flag stays disclosed; - rev 15 against rev 18.1 differential re-run and re-classified; tail counts unchanged; the header no longer quotes the value class; - patch regenerated against 260cc1a from 05feaad; embedding proof re-run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LT8D8tRmExyQve69DCWRik
…lows on across markers (see #234) Rev 18.1's board (round 4), maintainer decisions: - F-1: the glued run also stops at `,;([{<`; code/log test added (red on 05feaad: 852; green 0; rev 18.1's stops as mutant); the design text now states the stops the code uses; the tail cost is disclosed with the seat's numbers; - F-2: the glued run follows on across a marker it ends at (408 of 3 600 -> 0; mutant: no follow-on); - F-3: an ARN body is kept under any key and percent-encoded; - F-4: residuals restated at scale (stop tails, flag form 353/500, Bearer + whitespace escape 13-28 %, `|`/`:` log fields); - rev 15 against rev 19 differential re-classified; patch regenerated from 4f2ca0a against 260cc1a; embedding proof re-run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LT8D8tRmExyQve69DCWRik
ViperJuice
added a commit
that referenced
this pull request
Sep 27, 2026
…g redactor (see #234) (#310) The existing redactor runs unchanged. Shape-based rules then apply to its output, and they only ever write the redaction marker, so everything the existing redactor removes stays removed. Other guarantees: - markers are never split, re-spelled or swallowed; - a JSON document stays valid and object keys are never redacted; - the added pass runs in linear time and memory. Implements the plan merged in #288. Residuals are listed in the plan, CHANGELOG and SECURITY.md. Approved by the cross-vendor review panel. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LT8D8tRmExyQve69DCWRik
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Detailed plan for see #234 — the redactor is keyword-anchored, so it mangles ordinary prose while letting real credentials through. Plan only;
src/andtests/are unchanged.Plan:
.consiliency/plans/detailed-234-redactor-shape-based-20260923-0921.mdDesign in brief
access_token=fires;token_type=,token_endpoint=don't). A keyword followed by whitespace only redacts a credential-shaped value, sotoken bucketsurvives.Verified independently by the integrator
I applied the plan's embedded patch and test file to a scratch worktree off current main:
tests/test_auth.py);ruff checkandruff format --checkclean.sk-credential are all redacted; a git SHA, a UUID, a traceback frame and an Okta authorize URL survive intact.The implementer will need to widen
SECURITY.mdlines 120–122, and should expect GitHub push protection to reject detector-shaped sample tokens (the plan documents which).🤖 Generated with Claude Code
https://claude.ai/code/session_013G8iK4noj1iNbSTqJVFEX6
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.