Skip to content

comment_lines conflates blank lines with real documentation (engine-wide, found via #2593) #2625

Closed

Description

@squid-protocol

Found while triaging #2593 (python's rosetta cross-language-consistency sweep). Not
python-specific — the bug is in the shared comment_lines computation, so it likely biases
every language's comment_lines metric toward languages whose source style uses more blank
lines, not just ones with more actual comments.

The bug

comment_lines is computed in keyword-rosetta's tools/bias_report.py:

struct["comment_lines"] += max(0, (row["total_loc"] or 0) - (row["coding_loc"] or 0))

i.e. total_loc - coding_loc, reading the two DB columns galaxyscope writes per file.

  • coding_loc (gitgalaxy/core/prism.py split_streams, ~line 315-322) is computed from
    non-blank lines only: total_active_lines = len([l for l in content.split("\n") if l.strip()]), then coding_loc = len([l for l in final_code.split("\n") if l.strip()]),
    doc_loc = max(0, total_active_lines - coding_loc). Blank lines never enter either side.
  • But the DB's total_loc column (galaxyscope.py ~line 640, filter_res.get("total_loc", 0)) is the raw line count from aperture.py's ingestion filter — includes blank lines.

So comment_lines = total_loc(incl. blanks) - coding_loc(excl. blanks) silently folds every
blank line into "documentation," on top of real comment/docstring lines. A file with lots of
blank-line-separated functions reports inflated comment_lines for a reason that has nothing
to do with how well-documented it is.

Evidence (python, keyword-rosetta corpus)

data/python/{main,a,b,c}.py: engine reports comment_lines = 52 total, median across 46
languages is 30 (+73%, red band). Direct count: 31 of those lines are blank, only ~10-12
are real #-comment or docstring content. The blank-line count alone accounts for most of the
deviation — python's shell files happen to have a higher blank-line-to-code ratio than most
other languages' shells (stylistic choice per file, not a signal of anything).

Two independent fixes, either or both

  1. Corpus-side (keyword-rosetta): normalize blank-line density across all 46 language
    shells so comment_lines stops being dominated by formatting style. Doesn't fix the
    underlying engine metric, just removes this specific cross-language noise source.
  2. Engine-side (this repo): comment_lines/doc_loc shouldn't count blank lines as
    documentation at all. Since coding_loc already excludes blanks, the parallel total_loc
    used for the subtraction should too (or bias_report.py/whatever consumes doc_loc
    directly should use prism.py's own doc_loc output instead of re-deriving via
    total_loc - coding_loc against a blank-inclusive total). Needs a full re-bless per the
    Differential Scan protocol since it changes doc_loc for every language, every file.

Recommend (1) as a quick corpus-side mitigation and (2) as the real fix, scoped as its own
issue since it touches every language's golden master, not just python's.

Activity

  1. added
    bugUnintended behavior or logic failure in the engine
    metricsHeuristics, risk exposures, and topological math updates
    core-engineModifications to the central physics and parsing engine
    on Sep 1, 2026
  2. squid-protocol commented on Sep 1, 2026

    @squid-protocol
    OwnerAuthor

    Both halves merged and verified end-to-end: engine (#2632 — doc_loc persisted + rehydrator fix) and corpus (keyword-rosetta#16 — comment_lines switched to the honest column, 44/46 languages now in the green band). rosetta-audit dispatched on main against the bumped pin: success. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugUnintended behavior or logic failure in the enginecore-engineModifications to the central physics and parsing enginemetricsHeuristics, risk exposures, and topological math updates

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions