Skip to content

Epic: risk_documentation becomes a per-unit coverage ratio (Phase 4 score contract; child of #2812) #2908

Closed

Description

@squid-protocol

Child of #2812 (Phase 4 · score contracts). First formula to get a score contract; the pattern here is meant to be reused for risk_api_exposure, risk_tech_debt and func_complexity_gini, the other score-layer rows still carrying open-defect cells.

Labels: epic metrics core-engine roadmap

Why this epic exists (one paragraph)

_calc_documentation does not measure documentation. On the rosetta corpus every term but one is flat: no function has impact above 50 (the "opaque execution" term is dead), every file is under the 50-line evidence floor (the denominator is a constant 70), and the multipliers do not move the observed values. What remains is a sigmoid over 2 × api, so one adjusted-api hit per file is worth about 13 points on a 0–100 scale. The adjusted api count is assembled from two overlapping sources (the language's api rule plus the orphan-census credit, minus a tokenizer-dependent overlap), and every way that assembly fails lands in a different score cluster: 79 where a name is counted twice (m4, makefile, scheme, cobol), 55 where the rule sees one token and the census credits three (css, agc_assembly, yacc), 18 where the census is unanswerable or broken (dockerfile, html, yaml, haskell), 0 where the rule matches nothing (jcl, sqlite, objective-c main). On real code the same equation flips regime: a 500-line file with the same three undocumented public functions scores about 15, because the denominator is now the file length. That is the −0.83 length leak the bias report already flags, seen from the other side. Four of the eleven open-defect cells on the scoreboard and the corpus's single confirmed length leak are this one formula.

The goal restated: risk_documentation is the share of a file's callable surface a reader cannot recover from documentation. A ratio over units, not a density over lines. Identical intent scores identically by construction, not by tuning.

The contract

risk_documentation — of the units extracted from a file (functions, methods, paragraphs, steps), the weight-share that is undocumented. A unit is public when its name is in the file's public name set. A unit is documented when a doc-rule match is anchored to its header window. Public units weigh double; a unit's weight rises by one per reflection_metaprogramming hit inside it. A folder-level documentation umbrella shields the whole file multiplicatively. A file with no extracted units has no value (n/a), not zero.

for each extracted unit u:
    weight(u)  = (2 if public(u) else 1) + reflection_hits_inside(u)
    exposed(u) = weight(u) if not documented(u) else 0

risk_documentation = 100 × Σ exposed(u) / Σ weight(u) × (1 − umbrella_shield × doc_umbrella)
                   = n/a when Σ weight(u) = 0

Definitions that carry the contract:

What leaves the equation, with the reason:

term why it goes
doc_loc × 0.33 a comment-line count is not coverage, and it is one of the two length terms
ownership × 0.5 who maintains a file is a different question, already carried by silo/authorship views
opaque_execution (impact > 50 ∧ no docstring) dead on small functions; its intent (load-bearing undocumented code) is what public weighting expresses
/ (_mass_loc(loc) + 20) and the sigmoid the length regime; the ratio is already 0–100
popularity and silo multipliers, path modifier mp blast radius sits next to the score in the same report row; a coverage ratio does not change because the file lives in a config folder
doc_weight, ownership_weight, doc_loc_weight, loc_smoothing, threshold_base, sigmoid_slope six knobs become two: public_weight (2.0), umbrella_shield (0.5)

Weights are counts, not lengths. Per-function impact contains 0.05 × effective_loc; weighting by it re-imports length through the side door. Impact stays the hitlist's ranking key.

Decisions to approve before Phase 2 starts

  • D1 The score is a ratio over extracted units; no per-LOC term, no sigmoid, no small-file floor.
  • D2 public(u) is the name-set union above. This is the same set risk_api_exposure should read; that formula's own contract is a separate epic but shares Phases 1–2 here.
  • D3 documented(u) is the header-anchored doc match, not docstring. docstring is left as is (it feeds the brief's text), not deleted.
  • D4 Ownership, doc_loc, opaque_execution, popularity/silo multipliers and mp leave the equation. path-and-extension-modifiers in the ledger drops risk_documentation from its signal list when this lands.
  • D5 No damping knob for one-unit files (0 or 100). Roll-ups are already mass-weighted. Revisit only if Phase 5 shows hitlist noise.
  • D6 Engine keeps emitting a float; n/a is inferred by the reporting layer from unit count 0, the same convention the corpus tooling already uses for rule absence (no None in the risk vector).

Expected values (acceptance numbers)

file today after
rosetta a/b/c, every language 17.9 → 79.2 depending on api assembly 100 (3 public units, 0 documented)
rosetta main, entry documented 33.4 75 where the api rule sees entry, 86 where it does not (an api contract cell, surfaced on purpose)
css, yaml, html, sqlite, markdown 0 → 55 n/a
500-line file, 10 public units, 2 documented ≈ 15 80
20-line file, 3 public units, 0 documented 40.6 100
length-leak row (bias_report.md) rho −0.83, leak rho 0 by construction

Phases

  • 0 · Design freeze gitgalaxy — docs/risk_documentation_contract.md (the contract sentence, the equation, the leave-list table, D1–D6), and the per-unit decomposition that produced the cluster table above committed as tests/tools/audit_documentation_inputs.py (scan → per-file arch_api, credit, doc_loc, score; reproduces every corpus score from the inputs). Approve D1–D6 on this issue.
    done when the contract doc is merged as draft and the acceptance table above is agreed.

  • 1 · The name set is right before anything reads it gitgalaxy + keyword-rosetta — prerequisites, all already filed or owed:

  • 2 · Per-unit attributes in the detector gitgalaxy (T2) — three attributes on every extracted unit: is_public (name-set union, D2), is_documented (header-window doc match, D3), reflection_hits (the unit's own reflection_metaprogramming count, already sliced per function). Recorded as three function_data columns and in the audit JSON. Counts do not move: golden masters change by new columns only.

  • 3 · The equation gitgalaxy (T2, label rosetta:rebless-owed) — replace _calc_documentation with the ratio; delete the six tuning keys in analysis_lens.py, add public_weight and umbrella_shield; call site in signal_processor.py passes units + umbrella only.

    • tests: tests/core_engine/test_signal_processor.py (new pins), test_uef_length_invariance.py and tests/tools/audit_length_invariance.py (documentation is now exactly length-invariant: assert equality, not tolerance), tests/tools/audit_risk_equations.py (no tier/fid term left to audit for this formula)
    • golden masters: both legs re-blessed via tests/tools/crucible_check.py --update --yes; every file's documentation score moves, so the diff is scoped by the decomposition tool before blessing
    • prose that prints the equation: llm_recorder.py ("12. Documentation Risk Exposure …"), docs/wiki/08-03-transforming-regex-counts.md (the dynamism row), docs/gitgalaxy_architecture_brief.md self-scan regen
      done when both golden legs are blessed with a scoped diff, the length audit reads rho 0, and the corpus reads the acceptance table.
  • 4 · Corpus re-bless and scoreboard keyword-rosetta — manifests to the acceptance values; ledger: path-and-extension-modifiers drops risk_documentation, m4-makefile-api-scores-above-stratum and the api-declared-orphans-blind-to-non-word-names documentation half retire on Phase 1's verifying scan; bias_report.py: risk_dependencies() learns the per-unit inputs so a documentation cell attributes to a unit, reads_constant flips to False (the formula stops reading fid, so the metric is no longer banded within a strictness stratum, which today hides yacc at +28% against the global median), the leak table row becomes the proof.
    done when risk_documentation is 46/46 in band (n/a where declared), 0 open, and the report bands it against the global median.

  • 5 · Real-code validation gitgalaxy — before/after on the language-crucible and on the two README example repos (curl, cics-genapp): score distribution, top-20 documentation hitlist diff, count of one-unit files at 100 and whether any reaches a hitlist. Decide D5 on evidence. docs/incidence_report.md in keyword-rosetta gains the documentation row.
    done when the before/after comment is on this epic and D5 is confirmed or a damping design is filed as its own issue.

  • 6 · Closure — the contract row goes stated (score contracts get their own section in signal_contracts.py / docs/signal_contracts.md, the first entry); docs/contract_roadmap.md §4 Phase 4 marks this formula done; README "Accuracy, measured" copy refreshed with the live numbers (the paragraph still carries 2026-09-06's 90% / 2.7%; docs/validation.md carries 75% / 3 of 33).
    done when open-defect cells attributable to risk_documentation = 0, the sheet row is stated, and the next score contract (risk_api_exposure) can copy this epic's phase list.

Sequencing

0 design freeze ──► 1 name set (4 items, independent of each other)
                         │
                         ▼
                    2 per-unit attributes (counts unchanged)
                         │
                         ▼
                    3 equation + golden bless ──► 4 corpus re-bless ──► 5 real-code check ──► 6 close

Phase 1 and Phase 2 can run in parallel; Phase 3 must not start until both are merged, because the ratio reads the name set and a wrong set would be blessed into the golden masters. One layer per engine PR (rule | slicer | scoring), as #2812 rule 4.

Non-goals

Issue disposition

issue action
#2827, #2871 Phase 1 prerequisites; keep, unchanged
#2872 closed (engine half #2902); the corpus re-plant is Phase 1 item 2
#2771 (raw_arch_api granularity) answered by D2's per-unit name set; close when Phase 2 lands
#2719 (dynamism replaces flat irc) kept, moved per unit (D1); no reversal
#2655 (evidence-mass floor) the floor stays for the other density formulas; this formula stops reading it
bias report length-leak row for risk_documentation resolved by construction in Phase 3; the row stays as the proof
keyword-rosetta risk-documentation-zero-evidence-guard (retired) the guard's role (no surface → 0) becomes n/a (D6); note on the entry, no revival

Evidence behind the "why"

Sensitivity of today's equation to the adjusted api count per file (all other inputs at the corpus values, doc_loc = 2):

adjusted api / file 0 1 2 3 4 5 6 7
risk_documentation 0 17.9 27.9 40.6 54.8 68.2 79.2 86

Where the corpus lands and why (per-file scan of 20 languages against engine main, 2026-09-09; every score reproduced from the formula):

adjusted api / file languages (a/b/c) mechanism
6 makefile, m4, scheme, cobol raw 3 + orphan credit 3: overlap check never recognises the names (#2827, #2872)
5 c (a.c only) globals plants match the api rule as top-level declarations
4 css, agc_assembly, yacc raw 1 + credit 3; yacc adds 4 reflection hits
3 the mainstream pack; objective-c via credit alone (raw 0) intended
1 dockerfile, html, yaml (census n/a), haskell (#2871) raw 1, no credit
0 jcl, sqlite, objective-c main zero-evidence guard

Activity

  1. added
    epicLarge-scale architectural milestone or multi-issue initiative
    roadmapLong-term strategic planning and upcoming feature tracking
    metricsHeuristics, risk exposures, and topological math updates
    core-engineModifications to the central physics and parsing engine
    on Sep 9, 2026
  2. squid-protocol commented on Sep 9, 2026

    @squid-protocol
    OwnerAuthor

    claiming Phase 0 — Fable 5.1 — branch design/risk-documentation-contract

    status Phase 0: done pending CI — PR #2910 (docs/risk_documentation_contract.md as draft + tests/tools/audit_documentation_inputs.py). 46/46 corpus languages, 185 files, every recorded risk_documentation reproduced from its inputs at engine 81b895c.

    Filed from the decomposition:

    D1–D6 are on the contract doc for review. Phase 1 next (#2907 first: smallest, unblocks c's cell).

  3. squid-protocol commented on Sep 9, 2026

    @squid-protocol
    OwnerAuthor

    status Phase 1 / #2907: engine PR #2911 open (rosetta:rebless-owed), CI running. rule_probe crucible 1141 → 1040, rosetta 15 → 13, nothing gained; every lost match read. Corpus effect: c a.c api 5 → 3 only. Corpus re-bless of data/c opens after #2911 merges. Remaining Phase 1: #2827, #2871, m4/makefile re-plant.

  4. squid-protocol commented on Sep 9, 2026

    @squid-protocol
    OwnerAuthor

    status Phase 1: #2907 merged (#2911); corpus re-bless keyword-rosetta#110 open and green. #2827 → engine PR #2914 open (rosetta:rebless-owed): cobol + scheme a/b/c api_orphan_credit 3 → 0, nothing else moves; golden masters blessed (14 lua/perl/zig files on the four api-fed keys). Corpus branch rebless/2827-declared-orphans (both manifests + ledger retirement) prepared, opens after #2914 merges. Remaining Phase 1: #2871 (haskell), m4/makefile re-plant.

  5. squid-protocol commented on Sep 9, 2026

    @squid-protocol
    OwnerAuthor

    status Phase 1: all four items now have PRs.

    When #2914/#2915 and the three corpus PRs land, the scoreboard's 11 open-defect cells drop to 1 (makefile func_complexity_gini, its own ledger entry, no issue yet) and Phase 1's done-when holds: arch_api reads 3 in every a/b/c that has an api rule. Phase 2 (per-unit is_public / is_documented / reflection_hits) is next and needs D1–D6 approved.

  6. squid-protocol commented on Sep 9, 2026

    @squid-protocol
    OwnerAuthor

    Efficiency assessment of this plan (2026-09-09, after Phase 0 + Phase 1)

    Written from the session that ran Phases 0 and 1, so the numbers are what it actually cost, not a guess. The plan's phases are right. The way they were run was the expensive way, and the repo already owns the cheap way for rule contracts (.claude/skills/rule-contract-audit, .claude/agents/contract-audit-scout.md, tests/tools/bless_scope.py). None of it was used here because none of it names the score layer. The fix is to make the score-layer twin of that kit before Phase 2, not after, because this will not be the last formula (risk_api_exposure, risk_tech_debt, func_complexity_gini are queued on this same epic's pattern).

    Where the tokens went (measured on Phases 0–1)

    cost centre what happened what should have happened
    Hand-rolled scripts, repeated the per-file decomposition was written inline before it became a tool; the round-trip-preserving manifest/ledger JSON edit was written 4×; the lost/gained-lines diff for a rule change was written 2×; the scoped golden-master diff (golden_diff + bucketing) was written 2× although bless_scope.py already does it one tool each, run by the scout
    Everything in the orchestrator context rule_probe dumps, crucible windows, 46-language audits, ledger verdicts (single entries run to 3 KB) were all read in the main context scout returns digests; orchestrator holds ten digests, never a dump
    Serial by hand three Phase 1 fixes were built one after another in one context; each is independent (different files, different languages) three build+scout pairs in parallel, orchestrator reads three digests
    Environment traps a worktree created at the wrong path (relative to -C), a background job launched from the wrong cwd (the shell's cwd had drifted), a stale corpus worktree that produced a phantom high_risk_execution mismatch, a cd inside a gh command that moved the shell for every later call one worktree_env script that creates/refreshes and prints the exports; absolute paths always
    CI polling gh pr checks polled by hand a dozen times across four PRs one --watch in the background per PR, then nothing until it notifies
    PR bodies the same four sections (Measured / Corpus / Golden masters / Tests) typed four times generated from the tools' own output

    The irreducible orchestrator work was small and is the only part worth the strong model: the contract sentence and D1–D6; the read-every-lost-line pass on a rule change (the first #2907 candidate lost 21 Doom K&R definitions — a scout would have reported "21 lost lines start with a bare return type", the decision was one line); the per-cell verdicts in the ledger; and the go/no-go on each bless scope. Everything else was retrieval, measurement and formatting.

    What to build before Phase 2 (each is an issue: gitgalaxy #2916, keyword-rosetta#113)

    Engine (tests/tools/):

    • audit_score_inputs.py <risk_metric> — generalise audit_documentation_inputs.py: an adapter table per _calc_* (DB columns → call arguments, the EQUATION_CASES shape audit_length_invariance.py already uses), reproduce every recorded risk_<metric> from its recorded inputs, print the cluster table by dominant input and the sensitivity table, exit 1 on residual. Phase 0 of any score contract becomes one command and one digest. Also the tool that says, for a formula nobody has looked at, which input actually drives it.
    • rule_probe.py --diff-lines before.json after.json — the lost/gained matched-lines listing (with 2 lines of context for multi-line matches). The read-every-lost-line pass is the orchestrator's; the listing is not.
    • contract_pr_check.py — runs the gauntlet legs in parallel from one worktree (extraction tests for the touched languages, rosetta_audit against the corpus worktree at origin/main, crucible_check, bless_scope on the result, audit_check) and prints the PR-body block. One invocation, one digest, no retyping.
    • worktree_env.sh <name> [--corpus] — git fetch, create or move gitgalaxy-worktrees/<name> (and keyword-rosetta-worktrees/<name>) to origin/main, print export PYTHONPATH= GITGALAXY_PATH= GALAXYSCOPE_BIN= LANGUAGE_CRUCIBLE_PATH= KEYWORD_ROSETTA_PATH= for copy-paste. Refuses to run inside the primary checkout.
    • .claude/agents/contract-audit-scout.md: the environment section names /srv/storage_16tb/projects/gitgalaxy/v6/.venv, which is not this machine's venv (the working one is ~/nyx_projects/gitgalaxy/.venv, full precision at .crucible_venvs/full_precision). A scout dispatched today runs the wrong engine or fails on import. Fix the paths, add audit_score_inputs and contract_pr_check to its job list.
    • .claude/skills/score-contract-audit/SKILL.md — the score-layer twin of rule-contract-audit: same tier table (orchestrator / scout / build / adversary), same "never delegate" list, phases = this epic's phases with the exact commands, the D-decision template, the acceptance-table template, and docs/_score_contract_template.md with the sections risk_documentation_contract.md uses (sentence · equation · what today's formula measures · leave-list with reasons · decisions · acceptance · consumers · phase map).

    Corpus (keyword-rosetta/tools/):

    • rebless.py <lang> --engine <worktree> --note "<dated sentence>" [--retire <ledger-id> --resolved-by gitgalaxy#N --verdict "..."] — scan with the named engine, diff observed vs manifest, apply only the cells that moved (printing them), append the note, retire the ledger entry, preserve the formatting the files actually use (manifest indent=2, ensure_ascii=False; ledger ensure_ascii=True), re-run verify_language. The 4× hand-rolled python, once.
    • screen_plant.py --engine <worktree> — it hard-codes the primary checkout today (the skill already warns about this).
    • open_rebless_pr.sh <N> (or a bias-history.yml step): when engine main carries Closes #N and a pushed branch rebless/<N>-* exists, open the corpus PR with the standard body. The engine-first merge order is inherent; the remembering is not. Three corpus branches sat pushed-but-unopened this session waiting on human merges.
    • ledger_show.py <id> [--short] — id, signal, languages, disposition, status, the first and last 300 chars of the verdict. Reading a whole entry to check one field cost ~3 KB each time.

    Per-phase checklists for THIS epic (copy into the phase comment when claiming it)

    Phase 2 · per-unit attributes (gitgalaxy)

    • worktree_env.sh fix-2908-p2; claim comment on this issue.
    • ▸ scout: audit_score_inputs.py risk_documentation digest at engine main — the baseline every later step is compared to.
    • orchestrator: write the exact spec for is_public (name-set union: api header match ∪ export-list names ∪ healed orphans), is_documented (header-window doc match; k lines before start_line or leading body lines), reflection_hits. Name the header window's k and the docstring-position rule per lexical family before any code.
    • ▸ build: detector attributes + three function_data columns + audit JSON; contract tests per declaration family (docstring-position · preceding-block · export-list · positional-invocation).
    • ▸ scout: rosetta_audit 46/46 (counts must not move — this phase adds columns only); crucible_check (expect new columns only; bless_scope leaf-key table must show no signal key).
    • ▸ scout: per-language table of units_public / units_documented for a/b/c/main → orchestrator decides the 3-vs-4 dispatcher split per language against the api contract and ledgers it.
    • corpus: verify_language.py reads the two new keys; manifests gain them (rebless.py --add-keys); one ledger entry for the split.
    • contract_pr_check.py → PR body; label rosetta:rebless-owed; push the corpus branch before the engine PR merges.

    Phase 3 · the equation (gitgalaxy)

    • orchestrator: the equation is already written (§1 of the contract doc); confirm D1–D6 are approved on this issue. No code before that.
    • ▸ build: replace _calc_documentation; delete six tuning keys, add two; update test_signal_processor.py, test_uef_length_invariance.py, audit_length_invariance.py (equality, not tolerance), audit_risk_equations.py; the brief text in llm_recorder.py; wiki 08-03.
    • ▸ scout: audit_score_inputs.py risk_documentation at the new formula — every corpus file must read the acceptance table (100 / 75-or-86 / n/a); bless_scope on the golden masters — leaf-key table must be Documentation Exposure + its aggregates only; length-leak row rho 0.
    • orchestrator go/no-go on the scope table; bless; contract_pr_check.py → PR body; rosetta:rebless-owed.

    Phase 4 · corpus re-bless (keyword-rosetta)

    • rebless.py for all 46 (one loop); path-and-extension-modifiers drops risk_documentation; bias_report.py's risk_dependencies() learns the per-unit inputs and reads_constant flips; na_check/decoy_check/ledger_orphan_check clean; open_rebless_pr.sh.

    Phase 5 · real-code check — ▸ scout: before/after top-20 documentation hitlist on curl + cics-genapp + the crucible, count of one-unit files at 100; orchestrator decides D5 from the digest.

    Phase 6 · closure — sheet row stated; roadmap tick; README + validation.md numbers refreshed (both still carry September-6 figures).

    Rules of thumb that fell out of this session

    1. Measure with a tool or not at all. Anything measured inline gets re-measured next session.
    2. The orchestrator reads digests and lost lines, never dumps. If a tool's output is more than ~40 lines, it needs a --summary.
    3. Every environment fact is a script, not a memory. Worktrees, exports, corpus location, venv: worktree_env.sh prints them; nothing else should.
    4. Pre-build the corpus branch; automate the open. The merge order is the only serial dependency; nothing else in a phase should wait on a human.
    5. Independent fixes run in parallel agents. Phase 1 was three independent engine PRs; run them as three build+scout pairs.
    6. decoy_check after any api-rule change; a rule narrowing can drop a comment decoy below the floor (it did, for c).
  7. 5 remaining items

  8. squid-protocol commented on Sep 10, 2026

    @squid-protocol
    OwnerAuthor

    claiming Phase 3 — Fable 5 — branch feat/2908-p3-equation

  9. squid-protocol commented on Sep 10, 2026

    @squid-protocol
    OwnerAuthor

    status Phase 3: done pending CI — engine PR #2938 (rosetta:rebless-owed, 2 commits).

    The equation is the contract doc §1 verbatim: _calc_documentation(functions, doc_umbrella), weight = (public_weight 2.0 if public else 1) + the unit's own reflection hits, exposed if not documented, × (1 − umbrella_shield 0.5 × doc_umbrella); 0.0 emitted at unit count 0 (D6). Six tuning keys → two; loc/doc_loc/fid/ownership/opaque_execution/popularity/silo/mp all left per D4.

    One decision surfaced by measurement (folded into the formula + contract doc §5): the unit population excludes synthetic slices (#2691/#2792). sqlite's CREATE_Statement buckets otherwise scored a/b/c at 100 where the contract declares n/a — and the score could not reproduce from function_data, which excludes those buckets. Corollary from the same scan: the corpus n/a class is html, markdown, sqlite (+ javascript's package.json), not Phase 2's six — dockerfile, jcl and yaml DO extract units (the grammar-recognised keyword constructs #2728 kept) and read 100 in a/b/c.

    Done-when accounting:

    • audit_score_inputs.py documentation: 185 files, 0 residuals at the new formula; a/b/c read 100 in all 129 unit-bearing files; mains read the acceptance shapes (16×75, 18×85.71, plus the ledgered splits: 2×57.14, 71.43 scheme, 87.5 css, 60, 50 — all api-contract cells for the risk_api_exposure epic).
    • Length: audit_length_invariance.py now checks documentation over the WHOLE sweep (equality, not tolerance) — invariant by construction; the bias-report rho −0.83 row flips to the proof in Phase 4. audit_risk_equations.py: confirmed tier-blind (moved to NOT_TIER_PARAMETERIZED).
    • Golden masters: both legs blessed, verification green; bless_scope: 2203 diffs, 0 topological, ALL under Documentation Exposure + its declared aggregates. Crucible avg_documentation 7.52 → 52.36 (the re-price; Phase 5 examines the distribution and decides D5).
    • contract_pr_check --score: ALL GREEN (rosetta_audit 46/46 zero count regressions; audit_check battery clear).
    • Local full suite: 8667 passed; the 10 test_security_auditor failures are the mixed local venv missing pandas/xgboost (identical on main; CI installs them).

    Phase 4 (corpus re-bless + bias report) opens after #2938 merges: manifests to the measured values above, path-and-extension-modifiers drops risk_documentation, reads_constant flips, ledger retirements per the epic.

  10. squid-protocol commented on Sep 10, 2026

    @squid-protocol
    OwnerAuthor

    claiming Phase 4 — Fable 5 — corpus branch rebless/2908-p3-documentation (keyword-rosetta)

  11. squid-protocol commented on Sep 10, 2026

    @squid-protocol
    OwnerAuthor

    status Phase 4: done pending CI — corpus PR keyword-rosetta#120.

    No rebless: Phase 3 moved no counts, and manifests carry counts, not risk values — the corpus half of this phase turned out to be report machinery, not manifest edits. What landed:

    • Banding: reads_constant flipped to False on its own at the first post-merge regen (the formula no longer takes fid), so risk_documentation is banded against the global median — this also un-hides the strictness-stratum banding artifact the epic called out (yacc is now measured against the global median like everyone else).
    • Attribution: risk_dependencies() AST-derives the per-unit edges (functions_found, units_public, units_documented, units_public_documented, per-unit reflection); the totals are cached as an ungated attribution group, so a documentation cell attributes to a unit.
    • D6 n/a: population-based n/a — html/markdown/sqlite (the measured n/a class, amended from Phase 2's six: dockerfile/jcl/yaml DO extract units) read n/a with review state inherited from functions_found's own accounting. They had been coasting on path-and-extension-modifiers, which now drops risk_documentation per D4.
    • The leak row is the proof: exact-hold for pure per-unit ratios (banded holding read main's unit composition as rho +0.90); the row now reads invariant, +0.00 across 15 languages sharing the modal unit profile. 0 leaks in the table.

    Done-when accounting: 43/43 comparable cells in band, 3 n/a where declared → 46/46; open-defect cells attributable to risk_documentation: 0 (report total 3/2633, all pre-existing: raw_arch_api ×2, func_complexity_gini — the next contracts' cells); banded against the global median ✓. na_check / ledger_orphan_check / ledger_issue_check clean.

    One decay note for the Phase 6 sweep: jcl-steps-have-no-invocation-by-name's risk_documentation token died (jcl reads 100, in band, under the new equation) — scope-down candidate, baselined, advisory.

    Phase 5 (real-code before/after on curl + cics-genapp + the crucible; decide D5) is next once #120 merges.

  12. squid-protocol commented on Sep 10, 2026

    @squid-protocol
    OwnerAuthor

    claiming Phase 5 — Fable 5 — before/after on curl + cics-genapp + the language-crucible; D5 decided on the evidence, comment to follow here

  13. squid-protocol commented on Sep 10, 2026

    @squid-protocol
    OwnerAuthor

    status Phase 5: done — the before/after, and D5 is confirmed: no damping for one-unit files. Corpus PR keyword-rosetta#121 (incidence row) open; CI pending.

    Before/after (engine ef9b8e6 vs 90ddb3b, full precision)

    language-crucible (from the two golden-master versions, 2649 files):

    • distribution: mean 8.0 → 71.1, median 0 → 100; before 1754 files at 0 and 8 at 100, after 333 at 0 / 1159 at 100 / 574 n/a (no units).
    • top-20 (score desc, coding-LOC tiebreak): overlap 1/20. The before list was fixtures and wrappers (an html qunit fixture, .ksh wrappers, alonesym-obj.asm — several now 0 or n/a). The after list is the heaviest genuinely undocumented surface: gnucobol's parser.y (15,682 LOC, 1,148 units), cpython typeobject.c (418 units), racket internals, exiftool.

    curl (696 → 521 scored + 175 n/a):

    • mean 6.6 → 66.6; bimodal after: 134 files at 0 (curl's documented lib/ and headers), 297 at 100.
    • top-20 overlap 1/20 — and the direction of the change is the headline: the OLD top-20 was almost entirely include/curl/*.h and src/tool_*.h — curl's famously well-documented public API headers, flagged as the WORST documentation risk by the api×2 sigmoid. They all read 0 now (every unit documented). The new top-20 leads with tests/ftpserver.pl — the exact file the README already features for the function hitlist — then src/tool_getparam.c, the test servers, tool_operate.c: heavy, undocumented, real.

    cics-genapp (127 → 60 scored + 67 n/a):

    • the old formula was nearly dead here (mean 3.0, median 0 — COBOL's api rule barely fired). After: a uniform 50.0 on all 60 programs — every unit undocumented (100) halved by the folder-README umbrella (shield 0.5). Ordering within the tie is mass-driven, so the list leads with the big DB2 programs (lgipdb01, lgapdb01, lgupdb01 — the README's featured file). Distribution note: a repo with uniform (non-)documentation and one umbrella collapses to a single value by construction; the differentiation is the hitlist ordering, which mass provides.

    D5 (one-unit files: 0 or 100, no damping)

    corpus one-unit files at 100 in top-20
    crucible 438 (21% of scored) 263 0
    curl 213 72 0
    cics-genapp 19 0 (all 50, umbrella) 0

    No one-unit file reaches any top-20: the mass/LOC-weighted ordering keeps them out (curl's heaviest one-unit-at-100 has mass 185 vs a top-20 floor of 780), and the roll-ups are mass-weighted as D5 assumed. The heaviest one-unit case anywhere — c/micropython/vm.c, 1,151 LOC in one unit at 100 — is a correct flag (a giant undocumented VM interpreter loop), not noise. D5 confirmed as written; no damping issue filed.

    Method note: scans via the p4 full-precision venv; before = detached worktree at ef9b8e6, after = merged main; hitlists ranked score-desc with structural-mass (DBs) / coding-LOC (golden JSON) tiebreak — the product's own hitlist ranks by impact, which correlates with the mass tiebreak used here.

    Phase 6 (closure: contract row → stated, roadmap tick, README/validation numbers refresh) is the remaining phase.

  14. squid-protocol commented on Sep 10, 2026

    @squid-protocol
    OwnerAuthor

    claiming Phase 6 — Fable 5 — branch docs/2908-p6-closure: contract row → stated, roadmap §4 tick, README "Accuracy, measured" + validation.md refresh

  15. squid-protocol commented on Sep 10, 2026

    @squid-protocol
    OwnerAuthor

    status Phase 6: done pending CI — PR #2939.

    • Sheet: signal_contracts.py gains the score-contract section; risk_documentation is its first entry, stated, rendered into docs/signal_contracts.md. signal_contract_audit --ci clean (no new findings).
    • contract_roadmap.md §4: first formula marked done; risk_api_exposure, risk_tech_debt, func_complexity_gini copy this epic's phase list.
    • README "Accuracy, measured" + docs/validation.md: the 2026-09-06 figures (90% / 2.7%; 75% / 3-of-33) retire for the live report — 95% average in-band share, 0.1% open-defect cells (3 of 2,633), weakest metrics named.
    • docs/risk_documentation_contract.md: draft → stated with the phase record.
    • raw_arch_api bands a per-file declaration count against a per-function median — one granularity mismatch, three red columns #2771 closed per the disposition table (answered by D2's per-unit name set).

    Epic done-when, verified: open-defect cells attributable to risk_documentation = 0 (the corpus's 3 remaining opens are raw_arch_api ×2 and func_complexity_gini — the next contracts' cells); the sheet row is stated; the phase list is copyable as-is (the score-contract-audit skill names this epic as the worked precedent). Phase 6's checkbox ticks when #2939 merges — after that the epic can close.

  16. squid-protocol commented on Sep 11, 2026

    @squid-protocol
    OwnerAuthor

    Epic complete — ready to close

    All phases (0–6) are done and every phase's "done when" gate is met:

    One residual (doc hygiene, outside the formal gate)

    The README "Accuracy, measured" paragraph and docs/validation.md still carry the 2026-09-06 numbers (90% / 2.7% and 75% / 3 of 33). These want a refresh with live numbers from the next measurement run. This is a documentation-copy follow-up, not part of Phase 6's acceptance criteria — flagging so it isn't lost when this closes.

    Safe to close as epic complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    core-engineModifications to the central physics and parsing engineepicLarge-scale architectural milestone or multi-issue initiativemetricsHeuristics, risk exposures, and topological math updatesroadmapLong-term strategic planning and upcoming feature tracking

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions