Repository navigation
Fix #825: Extraction hardening for scala - #898
Merged
Merged
Conversation
Hardens all four extraction gauntlets (func_start, args, class_start,
_dependency_capture) for scala per tests/extraction/how_to_harden_extraction.md.
A fresh sweep surfaced three real bugs:
- func_start/args shared the flat `\[[^\]]*\]` Rule-11 square-bracket
generic-parameter step-over, breaking any nested generic bound
(`def foo[T <: Comparable[T]](x: T): T = {`) -- the same bug class
already fixed for java/typescript/python/rust/csharp/kotlin/swift.
- func_start/args never accepted backtick-quoted arbitrary identifiers
(Scala's escape hatch for reserved-word/space-containing method names,
e.g. Java-interop or ScalaTest-style spec names). Fixed as an
alternative capture group resolved via match.lastindex, not a widened
class. Kotlin shares this exact gap, missed during #823 -- filed as a
separate follow-up.
- _dependency_capture had no statement-boundary logic at all: the flat
`[\w.{}\s,]+` class let `\s` bleed across a second real import
statement into the next unrelated line on multi-import files, so the
second import was silently never detected. It also truncated Scala 3
wildcard imports (`*`) and `=>`-renamed block imports. Replaced with a
bounded segmented-dotted-path grammar.
crucible_check.py showed a real diff this time (not a zero diff):
confirmed genuine via the real Kafka scala corpus, where several files'
detected dependency counts roughly doubled once the bleed-over bug was
fixed (e.g. one file's direct-upstream count jumped 24 -> 53). The diff
also rippled into 9 unrelated language/repo pairs plus global ecosystem
aggregates -- confirmed as expected cross-repo dependency-graph/PageRank/
3D-projection ripple (every non-scala diff line was a global or derived
metric, never a raw per-file signature count for an untouched language),
not a confinement violation. Golden master fixtures re-blessed.
Checked test_language_standards_strict.py for intentional scala behavior
before finalizing -- no conflicts; existing scala regression tests pass
unmodified. Migrated scala's cases out of the four monolithic dict files
into tests/extraction/languages/test_scala.py (63 tests, up from ~15).
Two known limitations documented: func_start's Mode-B string-shielding
gap reproduces via triple-quoted strings (ninth language confirmed);
_dependency_capture's unshielded-content_buffer gap reproduces via a
commented-out import (same pipeline-wide issue as every other language).
Epic #813 and the methodology doc updated with recurring bug classes
33-35.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part of the extraction-hardening epic (#813). Closes #825.
Hardens all four extraction gauntlets (
func_start,args,class_start,_dependency_capture) for scala pertests/extraction/how_to_harden_extraction.md. This was a fresh sweep (no pre-flagged known bug) and surfaced three real bugs:func_start/argsshared a flat\[[^\]]*\]Rule-11 square-bracket generic-parameter step-over, breaking any nested generic bound (def foo[T <: Comparable[T]](x: T): T = {, a realistic bounded generic method) — the same bug class already fixed for java/typescript/python/rust/csharp/kotlin/swift.func_start/argsnever accepted backtick-quoted arbitrary identifiers at all — Scala's escape hatch for reserved-word/space-containing method names (e.g. Java-interop methods or ScalaTest-style spec names likedef `should handle edge cases`(): Unit = {}). Fixed as an alternative capture group resolved viamatch.lastindex(existing infrastructure the pipeline already had for this shape, e.g. java's(init)|(constructor)groups), not a widened class. Kotlin shares this exact identifier-grammar gap, missed during Extraction hardening: kotlin #823's pass — filed as a separate follow-up (kotlin func_start/args missing backtick-identifier support (found during #825) #899) rather than reopening Extraction hardening: kotlin #823._dependency_capturehad no statement-boundary logic at all: the previous flat[\w.{}\s,]+class let\s(which matches newlines) bleed across a SECOND real import statement into the following unrelated line on multi-import files, so the second import was silently never detected. It also truncated Scala 3 wildcard imports (import scala.util.chaining.*) at the*and=>-renamed block imports (import scala.util.{Try => STry}) at the=>. Replaced with a properly bounded segmented-dotted-path grammar instead of just widening the character class further.Differential Scan
crucible_check.pyshowed a real diff this time, not a zero diff — confirmed genuine, not a regression:KafkaRaftManager.scala's direct-upstream count jumped 24 → 53,ReplicaManager.scalajumped 79 → 144)._dependency_capturefeeds the cross-repo dependency DAG (network_risk_sensor.py's PageRank +spatial_mapper.py's 3D projection), which is computed over the whole scanned corpus as one graph, so this ripple is the expected behavior of fixing a real dependency-detection bug, not a bug of its own.tests/golden_master_audit.json,tests/golden_master_zero_dep_audit.json) re-blessed viacrucible_check.py --update --yes; both modes now PASS cleanly.Checked
test_language_standards_strict.pyfor intentional scala behavior before finalizing (per c's #822 lesson) — no conflicts found; existing scala regression tests pass unmodified.Also documents two known, unfixed limitations:
func_start's Mode-B string-shielding gap reproduces via triple-quoted strings (now confirmed on a ninth language);_dependency_capture's unshielded-content_buffergap reproduces via a commented-out import (same pipeline-wide issue already documented for every other language).Test plan
pytest tests/extraction/languages/test_scala.py(63 tests) — run via barepytestfrom an unrelated CWD to reproduce CI's invocation stylepython -m pytest tests/— 3995 passed, 1 deselected, 1 xfailed (pre-existing, unrelated)python tests/tools/audit_check.py— clean (ruff/mypy/dead-key baselines unchanged)python tests/tools/crucible_check.py— PASS on both venvs after re-blessing (see Differential Scan above)tests/extraction/languages/test_scala.py🤖 Generated with Claude Code