Repository navigation
fix(core-engine): dependency_density is coupling per line, not per control-flow ratio (#2770) - #2789
Merged
Conversation
…ntrol-flow ratio (#2770) `dependency_density` divided by `max(int(coding_loc * control_flow_ratio), 1)`. `control_flow_ratio` is `branch / (branch + structural_boundaries)`, so it is 0 for ANY file with no branches -- config, constants modules, data classes, most yaml/dockerfile -- and the denominator then collapsed onto its own `max(..., 1)` floor, making the "density" report the raw import COUNT. Measured on keyword-rosetta (one 12-probe program in 46 languages): 4 of the 5 files in every language have branch == 0, so the column read 1.000 for python's b.py and 2.000 for embedded_python's a.py -- their import counts -- against 0.048 for yaml's main.yml, which has branches. A 40x spread on one program. Three failures stacked: 1. the discontinuity at branch == 0 (above), up to 20x on one added `if`; 2. `int()` truncated the product BEFORE the floor, so any product under 2.0 also landed on 1; 3. where a language's `structural_boundaries` rule never fires, control_flow_ratio is 1.0 and the denominator silently became coding_loc -- the right answer, reached by accident (makefile). A coupling measure has no reason to read a control-flow ratio at all. `analysis_lens.ENGINE_CONSTANTS` already states the doctrine this column was missed by: "every per-file density equation divides by max(coding_loc, EVIDENCE_MASS_FLOOR)" (#2655). `dependency_density` now does, matching the per-function treatment its sibling `func_internal_density` got in #2705 six lines above it. security_auditor.py carried the same `coding_loc * control_flow_ratio` idiom. `logic_loc` is kept there unchanged -- `log_logic_loc` is an input to the pre-trained model and redefining a trained input is a retrain, not a bug fix (the frozen `func_internal_density: 0.0` placeholder is the precedent) -- but its defect is now recorded so the retrain can price it. The dead half is removed: control_flow_ratio is bounded [0, 1] by construction, so `logic_loc <= coding_loc` always and `max(logic_loc, coding_loc, 1)` could never select `logic_loc`. Written as the `max(coding_loc, 1)` it always was; no behavior change. Tests: tests/tools_recorders/test_dependency_density_floor.py pins the same four properties its #2705 sibling pins (invariance below the floor, no cliff at it, the density regime above it, zero/empty safety) plus the one specific to this fix -- the column does not move with control_flow_ratio. 14 of its 16 cases fail against the old formula. Verification: full suite 7791 passed. Golden masters unchanged in BOTH modes (`pytest -m golden_crucible`, full_precision and zero_dependency) -- the column is sqlite-only and appears in neither fixture. ruff/mypy baseline-gated checks green; the ruff baseline is regenerated for pure line shifts in record_keeper.py (every (file, rule) count identical before and after). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016BHBDqfybkLJqnQt9AoyTt
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes part of #2770 — shape (c), the piece that is a live defect rather than a comparability question. Shapes (a) and (b) are deliberately not in this PR; see "What this does not do".
The bug is worse than #2770 describes
record_keeper.py:#2770 frames this as "a gated metric divides by an ungoverned vocabulary tally". It is also, and more simply, discontinuous.
control_flow_ratioisbranch / (branch + structural_boundaries), so it is 0 for any file with no branches — config, constants modules, data classes, most yaml/dockerfile — and the denominator then collapses onto its ownmax(..., 1)floor. The column reports the raw import count.Measured per file on keyword-rosetta (one 12-probe program, 46 languages, engine at this branch's parent):
Four of the five files in every one of the 46 languages have
branch == 0. A 40× spread on one program, andassembly 0.063 / embedded_python 1.35— the two outliers #2770 names — are just the two ends of this collapse.Three failures stacked:
branch == 0. Add oneifto a branchless file and its denominator jumps from 1 toint(coding_loc × cfr)— up to 20× on the corpus. This is within-language, so thecontrol-flow-ratio-denominator-is-a-vocabulary-tallyledger entry's "fine within one repository" reading does not cover it.int()truncates the product before the floor, so anything under 2.0 also lands on 1.cfr == 1.0whereverstructural_boundariesnever fires (makefile), silently making the denominatorcoding_loc— the right answer, reached by accident.The fix is an existing doctrine this column was missed by
analysis_lens.ENGINE_CONSTANTSstates it outright:dependency_densitydid not. Its per-function siblingfunc_internal_densitygot the analog treatment in #2705, six lines above it in the same function. Now:Imports per line of code, floored — which is what the name describes, and what a coupling measure should be.
Second site:
security_auditor.pyNot mentioned in #2770. The same
coding_loc * control_flow_ratioidiom appears there:Handled in two halves:
logic_locis kept unchanged.log_logic_locis an input to the pre-trained model, and redefining a trained input is a retrain, not a bug fix — the same call the frozenfunc_internal_density: 0.0placeholder documents 20 lines down. Its defect is now recorded in place so the retrain can price it.control_flow_ratiois bounded[0, 1]by construction, sologic_loc <= coding_localways andmax(logic_loc, coding_loc, 1)could never selectlogic_loc. Rewritten as themax(coding_loc, 1)it always computed. No behavior change.What this does not do
control_flow_ratio's own denominator atcoding_loc— is a separate PR. It movesangle(detector.py:6928) for every function in every golden master, needs both the file-level and per-function definitions moved together, and needs an explicit[0, 1]clamp the current formula gets for free. Measured on the rosetta data, it takes the metric from 34.8% to 68.9% cross-language consistency.structural_boundaries— stays open and stops being on the critical path once (b) lands, sincecontrol_flow_ratiowill no longer read it. It is still worth doing for the signal in its own right: aSIGNAL_SCHEMAslot, charted as "Sequential Logic Declarations", one of the statistical auditor's 32 keys, and an input tolog_density_hit_structural_boundaries.Tests
tests/tools_recorders/test_dependency_density_floor.py, shaped after its #2705 siblingtest_func_internal_density_floor.py: invariance below the floor, no cliff at it,import / coding_locabove it, zero/empty-file safety, and the property specific to this fix — the column does not move withcontrol_flow_ratio. 14 of its 16 cases fail against the old formula.Verification
pytest -m golden_crucible,full_precisionandzero_dependency) — the column is sqlite-only and appears in neither fixture.ruffandmypybaseline-gated checks both exit 0. The ruff baseline is regenerated for pure line shifts inrecord_keeper.py: every(file, rule)pair has an identical count before and after, and the diff is 17 removals against 17 identical additions once line numbers are stripped. Themypybaseline is deliberately left alone — its one resolved entry (full_api_network_map.py:32) is pre-existing drift onmain, confirmed against a clean checkout, and not this PR's to absorb.Cross-repo
Companion: keyword-rosetta owes a re-bless. Merge order and what moves:
rosetta:rebless-owedperkeyword-rosetta/docs/GATING.md§ "Cross-repo flow". Nothing in the corpus repo is edited beforehand.main:dependency_densityis a bias-report metric derived from theimport_countandcoding_locDB columns, not aSIGNAL_SCHEMAkey — so noexpected_signals.jsonmanifest changes andverify_languagegates do not move. What moves isdocs/bias_report.md/bias_data.json, whichbias-history.ymlregenerates on push.control-flow-ratio-denominator-is-a-vocabulary-tallynarrows —dependency_densityleaves its blast radius, and the entry's standing "no engine change is wanted" verdict needs the branchless-file discontinuity recorded against it.no-dependency-capture-languageslistsdependency_densityamong its affected signals and should be re-checked at the new values.Measured effect on the corpus: every rosetta file is under the 50-line floor, so the column becomes a scaled import count —
0.020for a 1-import file,0.040for 2 — replacing today's0.048–2.000. Cross-language consistency82.6% → 95.6%, and the 82.6% was itself a floor artifact (a constant denominator of 1 for most cells), not evidence of health.🤖 Generated with Claude Code
https://claude.ai/code/session_016BHBDqfybkLJqnQt9AoyTt