yacc: %union type declarations have real record/struct morphology but class_start is wired None #2644

Activity

1. squid-protocol

   added

   enhancementNew feature, sensor, or structural signatureNew feature, sensor, or structural signature

   core-engineModifications to the central physics and parsing engineModifications to the central physics and parsing engine

   on Sep 2, 2026
2. squid-protocol commented on Sep 2, 2026

   

   squid-protocol

   on Sep 2, 2026

   OwnerAuthor

   More actions

   Pulled from Batch B.3 wave 2 — this cannot ship as a registry-only change. Implemented, measured, and reverted; here is why.

   Wiring class_start swaps "no classes" for "phantom classes"

   detector.py ~L1091 gates named-class extraction in two stages:

   if "class_start" in rules and rules["class_start"] is None:
       class_matches = []                     # hard zero
   else:
       class_start_pattern = (
           rules.get("class_start") if self.primary_lang_id in _CLASS_START_NAMED_EXTRACTION_LANGS else None
       )
       if class_start_pattern is not None:
           ...                                # the language's own rule
       # else -> the LEGACY GENERIC FALLBACK: class|struct|interface|trait|enum

   yacc is not in _CLASS_START_NAMED_EXTRACTION_LANGS (37 languages, yacc absent). So the moment class_start stops being None, yacc does not start using its own %union rule — it falls through to the generic fallback, which then matches every bare struct Foo in the embedded C action code that fills a real grammar file.

   Measured on the two real .y files in language-crucible/data/yacc/, toggling the rule in a scratch copy:

   	config.y	jailparse.y
   class_count before (class_start: None)	0	0
   class_count after (rule wired, allowlist unchanged)	17	9
   the new %union rule's own raw signal	1	1

   There are 57 bare struct <name> occurrences across those two files (struct opt, struct device, struct cfparam, …), all ordinary C in action blocks. So the fix as filed would replace an honest zero with 26 phantom classes on real grammars, and drag classes_found and everything downstream with it. The %union rule itself is correct — exactly 1 match per file, correctly anchored at config.y:1 and jailparse.y:45.

   What it actually needs

   Adding yacc to _CLASS_START_NAMED_EXTRACTION_LANGS, which is a detector.py change — the slicer layer. #2669's stacking rule keeps that out of a registry-regex PR, so this issue is now blocked on a companion slicer-layer change and belongs in its own PR with the two together.

   The rest of the design work is done and holds up, so whoever picks this up starts from:

   • Rule (validated, 1/1 on both real files): re.compile(r"^[ \t]*%union\b(?:[ \t]+([a-zA-Z_]\w*))?", re.M) — optional group 1 for bison's rarer named-tag form (%union name {), and deliberately no group 2, since a union has no inheritance parent for detector.py's group-2-is-parent convention to misread. The common anonymous form leaves group 1 unset and falls to _resolve_class_start_match's "Anonymous_Class" path, matching assembly's own no-name class_start precedent.
   • Corpus plant (screened against yacc's full rules dict — contains no (, ,, =, &, <, void, const, static, comment markers, or line-ending :, so it fires nothing but class_start), the verbatim freebsd/config.y:1-6 shape:

     %union {
     	char	*str;
     	int	val;
     	struct	file_list *file;
     }
     

     Note the rosetta corpus's four .y files use no struct/class/enum at all, so there classes_found would correctly stay 0 even with the fallback active — the phantom-class problem is real-world only, which is exactly why the rosetta gate would not have caught it.
   • Docs: docs/language_status/yacc.md §1 and §4 currently document class_start: None as deliberate and go stale when this lands.

   The #2672 doc-family half of yacc shipped separately and is corpus-inert (yacc's planted doc line is single-star /* @param … */, which /\*\* never matched, so doc stays 1).
3. squid-protocol

   added

   status: blockedWaiting on upstream dependencies or user clarificationWaiting on upstream dependencies or user clarification

   on Sep 2, 2026
4. squid-protocol

   added a commit that references this issue on Sep 3, 2026

   fix(core-engine): Rosetta batch 3 wave 2 — doc-comment pairing, rule …

   ...

   deca2bf
5. squid-protocol commented on Sep 5, 2026

   

   squid-protocol

   on Sep 5, 2026

   OwnerAuthor

   More actions

   Unblocked and fixed in #2737, with both halves in one PR as the comment above concluded they had to be.

   Took the validated rule verbatim (^[ \t]*%union\b(?:[ \t]+([a-zA-Z_]\w*))?, optional group 1 for the named-tag form, no group 2) and added yacc to _CLASS_START_NAMED_EXTRACTION_LANGS in the same change.

   Re-measured the phantom-class counterfactual on current main before writing anything, and it is worse than the two-file figure in the comment above: 109 classes across all four real grammar files in the crucible corpus (config.y 17, jailparse.y 9, gnucobol parser.y 56, scanner.l 27) where the true total is 2. parser.y is the sharpest case — 56 phantom classes on a grammar that declares no union at all, since it uses %define api.value.type. With the allowlist entry the four files read 1 / 1 / 0 / 0.

   One correction to the verification plan: ctags cannot corroborate this either. ctags --list-kinds-full=YACC exposes exactly one kind, l (label) — which is why CTAGS_CLASS_KINDS["yacc"] is empty on purpose and §9 covers functions only. So the direct source cross-check is the only external check available, the same standing abap/cobol/jcl/sqlite have on this allowlist, and that is how it was verified.

   Golden-master diff is exactly the two lines the issue predicted: Class/Entity Declarations 0 -> 1 on config.y and jailparse.y, both modes, nothing else moving. All CI green including rosetta-audit — the rosetta yacc probes contain no %union, so no corpus cell moves and no re-bless is owed. Step 4 (the corpus plant that makes the yacc/class_start n/a cell plantable) stays open as the follow-up, described in the PR body.
6. squid-protocol

   added a commit that references this issue on Sep 5, 2026

   test: re-bless golden master after merging main (#2644)

   ...

   3938d24
7. squid-protocol

   closed this as completedin 69c1c51on Sep 5, 2026
8. squid-protocol

   added this to the Rosetta Program Comparison milestone on Sep 5, 2026