Skip to content

fix(core-engine): dependency_density is coupling per line, not per control-flow ratio (#2770) - #2789

Merged
squid-protocol merged 1 commit into
mainfrom
fix/2770-dependency-density-mass-floor
Sep 6, 2026
Merged

squid-protocol merged 1 commit into
mainfrom
fix/2770-dependency-density-mass-floor

Conversation

@squid-protocol

Copy link
Copy Markdown
Owner

Closes part of #2770 — shape (c), the piece that is a live defect rather than a comparability question. Shapes (a) and (b) are deliberately not in this PR; see "What this does not do".

The bug is worse than #2770 describes

record_keeper.py:

logic_loc_denom = max(int(coding_loc * control_flow_ratio), 1)
dependency_density = import_count / logic_loc_denom

#2770 frames this as "a gated metric divides by an ungoverned vocabulary tally". It is also, and more simply, discontinuous. control_flow_ratio is branch / (branch + structural_boundaries), so it is 0 for any file with no branches — config, constants modules, data classes, most yaml/dockerfile — and the denominator then collapses onto its own max(..., 1) floor. The column reports the raw import count.

Measured per file on keyword-rosetta (one 12-probe program, 46 languages, engine at this branch's parent):

lang file coding_loc cfr denominator imports dependency_density
python b.py 13 0.000 1 1 1.000
embedded_python a.py 13 0.000 1 2 2.000
python main.py 25 0.250 6 1 0.167
makefile b.mk 14 1.000 14 1 0.071
yaml main.yml 28 0.750 21 1 0.048

Four of the five files in every one of the 46 languages have branch == 0. A 40× spread on one program, and assembly 0.063 / embedded_python 1.35 — the two outliers #2770 names — are just the two ends of this collapse.

Three failures stacked:

  1. The discontinuity at branch == 0. Add one if to a branchless file and its denominator jumps from 1 to int(coding_loc × cfr) — up to 20× on the corpus. This is within-language, so the control-flow-ratio-denominator-is-a-vocabulary-tally ledger entry's "fine within one repository" reading does not cover it.
  2. int() truncates the product before the floor, so anything under 2.0 also lands on 1.
  3. cfr == 1.0 wherever structural_boundaries never fires (makefile), silently making the denominator coding_loc — the right answer, reached by accident.

The fix is an existing doctrine this column was missed by

analysis_lens.ENGINE_CONSTANTS states it outright:

Evidence-mass floor (#2655): every per-file density equation divides by max(coding_loc, EVIDENCE_MASS_FLOOR).

dependency_density did not. Its per-function sibling func_internal_density got the analog treatment in #2705, six lines above it in the same function. Now:

dependency_density = import_count / max(float(coding_loc), EVIDENCE_MASS_FLOOR)

Imports per line of code, floored — which is what the name describes, and what a coupling measure should be.

Second site: security_auditor.py

Not mentioned in #2770. The same coding_loc * control_flow_ratio idiom appears there:

logic_loc = max(int(round(coding_loc * cfr)), 1)
safe_denom = max(logic_loc, coding_loc, 1)

Handled in two halves:

  • logic_loc is kept unchanged. log_logic_loc is an input to the pre-trained model, and redefining a trained input is a retrain, not a bug fix — the same call the frozen func_internal_density: 0.0 placeholder documents 20 lines down. Its defect is now recorded in place so the retrain can price it.
  • The dead half is removed. control_flow_ratio is bounded [0, 1] by construction, so logic_loc <= coding_loc always and max(logic_loc, coding_loc, 1) could never select logic_loc. Rewritten as the max(coding_loc, 1) it always computed. No behavior change.

What this does not do

  • Shape (b) — repointing control_flow_ratio's own denominator at coding_loc — is a separate PR. It moves angle (detector.py:6928) for every function in every golden master, needs both the file-level and per-function definitions moved together, and needs an explicit [0, 1] clamp the current formula gets for free. Measured on the rosetta data, it takes the metric from 34.8% to 68.9% cross-language consistency.
  • Shape (a) — a written contract for structural_boundaries — stays open and stops being on the critical path once (b) lands, since control_flow_ratio will no longer read it. It is still worth doing for the signal in its own right: a SIGNAL_SCHEMA slot, charted as "Sequential Logic Declarations", one of the statistical auditor's 32 keys, and an input to log_density_hit_structural_boundaries.

Tests

tests/tools_recorders/test_dependency_density_floor.py, shaped after its #2705 sibling test_func_internal_density_floor.py: invariance below the floor, no cliff at it, import / coding_loc above it, zero/empty-file safety, and the property specific to this fix — the column does not move with control_flow_ratio. 14 of its 16 cases fail against the old formula.

Verification

  • Full suite: 7791 passed, 4 skipped, 9 xfailed, 3 xpassed.
  • Golden masters unchanged in both modes (pytest -m golden_crucible, full_precision and zero_dependency) — the column is sqlite-only and appears in neither fixture.
  • ruff and mypy baseline-gated checks both exit 0. The ruff baseline is regenerated for pure line shifts in record_keeper.py: every (file, rule) pair has an identical count before and after, and the diff is 17 removals against 17 identical additions once line numbers are stripped. The mypy baseline is deliberately left alone — its one resolved entry (full_api_network_map.py:32) is pre-existing drift on main, confirmed against a clean checkout, and not this PR's to absorb.

Cross-repo

Companion: keyword-rosetta owes a re-bless. Merge order and what moves:

  1. This PR merges first, labelled rosetta:rebless-owed per keyword-rosetta/docs/GATING.md § "Cross-repo flow". Nothing in the corpus repo is edited beforehand.
  2. Corpus side, against engine main: dependency_density is a bias-report metric derived from the import_count and coding_loc DB columns, not a SIGNAL_SCHEMA key — so no expected_signals.json manifest changes and verify_language gates do not move. What moves is docs/bias_report.md / bias_data.json, which bias-history.yml regenerates on push.
  3. Ledger: control-flow-ratio-denominator-is-a-vocabulary-tally narrows — dependency_density leaves its blast radius, and the entry's standing "no engine change is wanted" verdict needs the branchless-file discontinuity recorded against it. no-dependency-capture-languages lists dependency_density among its affected signals and should be re-checked at the new values.

Measured effect on the corpus: every rosetta file is under the 50-line floor, so the column becomes a scaled import count — 0.020 for a 1-import file, 0.040 for 2 — replacing today's 0.048–2.000. Cross-language consistency 82.6% → 95.6%, and the 82.6% was itself a floor artifact (a constant denominator of 1 for most cells), not evidence of health.

🤖 Generated with Claude Code

https://claude.ai/code/session_016BHBDqfybkLJqnQt9AoyTt

…ntrol-flow ratio (#2770)

`dependency_density` divided by `max(int(coding_loc * control_flow_ratio), 1)`.
`control_flow_ratio` is `branch / (branch + structural_boundaries)`, so it is 0
for ANY file with no branches -- config, constants modules, data classes, most
yaml/dockerfile -- and the denominator then collapsed onto its own `max(..., 1)`
floor, making the "density" report the raw import COUNT.

Measured on keyword-rosetta (one 12-probe program in 46 languages): 4 of the 5
files in every language have branch == 0, so the column read 1.000 for python's
b.py and 2.000 for embedded_python's a.py -- their import counts -- against
0.048 for yaml's main.yml, which has branches. A 40x spread on one program.

Three failures stacked:

  1. the discontinuity at branch == 0 (above), up to 20x on one added `if`;
  2. `int()` truncated the product BEFORE the floor, so any product under 2.0
     also landed on 1;
  3. where a language's `structural_boundaries` rule never fires,
     control_flow_ratio is 1.0 and the denominator silently became coding_loc --
     the right answer, reached by accident (makefile).

A coupling measure has no reason to read a control-flow ratio at all.
`analysis_lens.ENGINE_CONSTANTS` already states the doctrine this column was
missed by: "every per-file density equation divides by max(coding_loc,
EVIDENCE_MASS_FLOOR)" (#2655). `dependency_density` now does, matching the
per-function treatment its sibling `func_internal_density` got in #2705 six
lines above it.

security_auditor.py carried the same `coding_loc * control_flow_ratio` idiom.
`logic_loc` is kept there unchanged -- `log_logic_loc` is an input to the
pre-trained model and redefining a trained input is a retrain, not a bug fix
(the frozen `func_internal_density: 0.0` placeholder is the precedent) -- but
its defect is now recorded so the retrain can price it. The dead half is
removed: control_flow_ratio is bounded [0, 1] by construction, so
`logic_loc <= coding_loc` always and `max(logic_loc, coding_loc, 1)` could
never select `logic_loc`. Written as the `max(coding_loc, 1)` it always was;
no behavior change.

Tests: tests/tools_recorders/test_dependency_density_floor.py pins the same
four properties its #2705 sibling pins (invariance below the floor, no cliff at
it, the density regime above it, zero/empty safety) plus the one specific to
this fix -- the column does not move with control_flow_ratio. 14 of its 16
cases fail against the old formula.

Verification: full suite 7791 passed. Golden masters unchanged in BOTH modes
(`pytest -m golden_crucible`, full_precision and zero_dependency) -- the column
is sqlite-only and appears in neither fixture. ruff/mypy baseline-gated checks
green; the ruff baseline is regenerated for pure line shifts in record_keeper.py
(every (file, rule) count identical before and after).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016BHBDqfybkLJqnQt9AoyTt
@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

@squid-protocol
squid-protocol merged commit 08a08c6 into main Sep 6, 2026
42 checks passed
@squid-protocol
squid-protocol deleted the fix/2770-dependency-density-mass-floor branch September 6, 2026 02:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rosetta:rebless-owed Intentionally moves keyword-rosetta counts; audit warns, corpus re-blesses after merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant