Skip to content

[GOLD-RING] parser: the Zig subset lexer.t27 needed (Closes #2426) - #2427

Merged
gHashTag merged 7 commits into
masterfrom
feat/zig-subset
Aug 23, 2026
Merged

gHashTag merged 7 commits into
masterfrom
feat/zig-subset

Conversation

@gHashTag

Copy link
Copy Markdown
Collaborator

[GOLD-RING] — the seal move in this pull request is made on the Architect's explicit authorisation, given in response to #2368's last section, which asked for exactly that before touching bootstrap/src/compiler.rs.

Result

12 specs move from the compile-debt ledger to generating, with zero regressions.

before:  712 specs, 541 generate, 171 known-broken
after:   712 specs, 553 generate, 159 known-broken

The ledger is updated in the same commit and with the same compiler that changed, so the gate now holds those twelve compiling rather than excusing them. Measuring a ledger with the compiler it gates is the lesson of #2365; this is the first change written under it.

Five gaps, each verified by the error moving

# gap shape
1 anonymous braced literal .{}, .{ a, b }, .{ .f = a }
2 anonymous struct type in return position fn f(…) struct { u64, []const u8 }
3 Zig slice spellings x[a..b] and the open-ended x[a..]
4 keyword-named enum values return .const;, .use, .data
5 struct-method skip mistaking a type's brace for the body's the one that mattered

Gap 5 is the reason the campaign was worth running. Struct methods are skipped brace-balanced, and the skip stopped at the first {. With an anonymous struct return type that brace belongs to the type, so the balance closed on the type, the struct ended at its first such method, and the module ended at the struct's own }. compiler/parser/lexer.t27 lost everything after line 362 while reporting a parse error 144 lines further on.

The code already carried a comment about exactly this hazard — W577, which cost specs/jit/jit.t27 797 of its 875 lines the same way. The hazard was documented and recurred on a construct the comment did not anticipate.

Two corrections to my own earlier reports

  • I wrote in The anonymous braced literal .{ } is one of several parser gaps in lexer.t27, not the gap #2368 that read_number at line 362 "parses" while the two-parameter form at 424 does not. Neither parsed. The one-parameter form was consumed by the skip and looked like success — which is precisely the failure mode being fixed here. The bisect that found it tested prefixes of the struct, method by method, rather than trusting the reported line.
  • Gap 3's fix was applied to the slice parser first and changed nothing, because .. is in the binary-operator chain and was eaten before the slice code ever saw it. The working fix leaves the .. alone when the next token is ] or ).

Ceremony

frozen-digest re-run and the seal line replaced, per FROZEN.md §5. cargo build --release green. Resealing invalidates the NMSE certification in repro/numerics/nmse_manifest.json; nmse_gf16.py --seal needs re-running after refreezing stage0/FROZEN_HASH, and that is deliberately left to the owner rather than done here.

What this does not fix

Six specs remain, each for its own reason, exactly as #2372 triaged them: union(enum), func.?(), inline for, keywords as enum members, closures, and one corrupted-source case already repaired separately. This campaign closed the family that lexer.t27 needed and no more.

Refs #2368, #2372.

🤖 Generated with Claude Code

The seal move here is made on the Architect's explicit authorisation, given
in answer to #2368's last section, which asked for exactly that before
touching bootstrap/src/compiler.rs.

12 specs move from the compile-debt ledger to generating, with zero
regressions:

  before:  712 specs, 541 generate, 171 known-broken
  after:   712 specs, 553 generate, 159 known-broken

The ledger is updated in the same commit and by the same compiler that
changed, so the gate now HOLDS those twelve compiling rather than excusing
them. Measuring a ledger with the compiler it gates is the lesson of #2365;
this is the first change written under it.

Five gaps, each verified by the error moving rather than by hope:

  1. the anonymous braced literal .{} / .{ a, b } / .{ .f = a }
  2. an anonymous struct TYPE in return position
  3. Zig's slice spellings x[a..b] and the open-ended x[a..]
  4. keyword-named enum values, `return .const;`
  5. the struct-method skip mistaking a return type's brace for the body's

Gap 5 is why the campaign was worth running. Struct methods are skipped
brace-balanced and the skip stopped at the first brace; with an anonymous
struct return type that brace belongs to the TYPE, so the balance closed on
the type, the struct ended at its first such method, and the module ended at
the struct's own brace. compiler/parser/lexer.t27 lost everything after line
362 while reporting a parse error 144 lines further on. The code already
carried a comment about this exact hazard -- W577, which cost specs/jit/jit.t27
797 of its 875 lines the same way. The hazard was documented and recurred on
a construct the comment did not anticipate.

Two corrections to my own earlier reports. I wrote in #2368 that line 362
parses while line 424 does not; NEITHER parsed -- the first was consumed by
the skip and looked like success, which is the failure mode being fixed. And
gap 3's fix went into the slice parser first and changed nothing, because
`..` sits in the binary-operator chain and was eaten before the slice code
saw it.

Ceremony per FROZEN.md section 5: frozen-digest re-run, seal line replaced,
cargo build --release green. Resealing invalidates the NMSE certification in
repro/numerics/nmse_manifest.json -- nmse_gf16.py --seal needs re-running
after refreezing, and that is left to the owner.

Six specs remain, each for its own reason, exactly as #2372 triaged them.
This campaign closed the family lexer.t27 needed and no more.
# Conflicts:
#	bootstrap/stage0/FROZEN_HASH
@gHashTag

Copy link
Copy Markdown
Collaborator Author

Rebased onto current master (seal conflict only — recomputed from the merged compiler.rs). Numbers re-verified after the merge, because master touched the compiler too: OK: 712 specs, 553 generate, 159 known-broken, exit 0. Unchanged from before the merge.

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-22 15:08:49 UTC

Summary

Status Count
Total Open PRs 4
PRs with Failing Checks 1
PRs with All Checks Green 3
READY 2
FAILING 1
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=c1c6522ef38b != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-22 15:27:23 UTC

Summary

Status Count
Total Open PRs 4
PRs with Failing Checks 1
PRs with All Checks Green 3
READY 2
FAILING 1
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=c1c6522ef38b != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

…2426)

Wave 2 of the campaign authorised in #2426.

  541 generate -> 558, ledger 171 -> 154, zero regressions.

Four gaps, each verified by the error moving:

  6. keyword-named enum MEMBERS -- then, and, assert, expect are real
     members of ClauseKind and requiring Ident dropped them
  7. a nested type declaration inside a struct body, const X = enum {...};
  8. payload capture in if -- if (opt) |value| { ... }
  9. Zig's while continue expression -- while (i < n) : (i += 1) { ... }

Gap 7 is the THIRD appearance of one shape: a nested brace group ending the
construct that contains it. W577 found it for methods, wave 1 found it for an
anonymous struct return type, and this is the same failure for a nested type
-- the struct body loop stops at any RBrace, so the enum's closing brace ended
the struct and the module ended at the struct's own.

Gap 9 attaches the continue expression rather than discarding it. Dropping it
would leave a loop whose counter never advances, which compiles and never
terminates; refusing the file is better than emitting that silence.

compiler/codegen/verilog/codegen.t27 now generates, joining lexer.t27.
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-22 15:59:07 UTC

Summary

Status Count
Total Open PRs 6
PRs with Failing Checks 3
PRs with All Checks Green 3
READY 3
FAILING 3
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=d0fc6be841df != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@gHashTag

Copy link
Copy Markdown
Collaborator Author

Wave 2 — four more gaps, and the totals grow.

                generate    ledger
wave 1            553         159
wave 2            558         154
from master's 541 / 171

Seventeen specs leave the compile-debt ledger, zero regressions, and compiler/codegen/verilog/codegen.t27 joins compiler/parser/lexer.t27 as generating.

# gap shape
6 keyword-named enum members then, and, assert, expect are real members of ClauseKind
7 a nested type inside a struct body const Kind = enum { … };
8 payload capture in if if (opt) |value| { … }
9 while continue expression while (i < n) : (i += 1) { … }

Gap 7 is the third appearance of one shape. A nested brace group ending the construct that contains it: W577 found it for methods, wave 1 found it for an anonymous struct return type, and this is the same failure for a nested type — the struct-body loop stops at any RBrace, so the enum's closing brace ended the struct and the module ended at the struct's own. Three constructs, one hazard, and each time the reported error was tens of lines past the cause.

Gap 9 attaches the continue expression rather than discarding it. Dropping it would leave a loop whose counter never advances — code that compiles and never terminates. Refusing the file is better than emitting that silence, which is the same reasoning as the open-ended slice getting its own marker in wave 1.

Still six specs short of the file being clean, and each remaining one has its own distinct cause: union(enum), func.?(), inline for, an error-union return !void, and closures — which the compiler refuses deliberately, not for want of grammar.

#2426)

Wave 3 of the campaign authorised in #2426.

  541 generate -> 562, ledger 171 -> 150 across three waves, zero regressions.

  10. tagged unions: union(enum) { A: struct {...}, ... }
  11. a call on an ARBITRARY expression: func.?(a, b)
  12. the inline unroll hint before for / while

union is not a keyword in this lexer, so union(enum) parsed as a CALL whose
argument is the keyword enum. Captured verbatim rather than lowered to a
struct: its variants carry payload types, and emitting a union as a struct
would give every variant its own storage and silently change the layout.

The expression-call branch replaces a comment reading 'shouldn't normally
happen here'. specs/jit/jit.t27 is exactly it -- the optional unwrap parsed,
and the call on its result was left unconsumed, so the statement parser met a
bare '(' and pointed past the construct rather than at it.

inline is likewise not a keyword, so 'inline for' parsed as the expression
'inline' followed by a loop keyword. Skipped, not recorded: a parser that does
not unroll loses nothing by ignoring the hint.

compiler/codegen/zig/codegen.t27, specs/jit/jit.t27 and
specs/ternary/bigint.t27 now generate. Five of the seven target specs are
done; main_zig_handwritten needs an error-union return type, and cap_test is
the compiler deliberately refusing closures, not a gap in the grammar.
@gHashTag

Copy link
Copy Markdown
Collaborator Author

Wave 3 — the campaign's totals, and five of seven target specs done.

                 generate   ledger
master's start     541       171
wave 1             553       159
wave 2             558       154
wave 3             562       150

Twenty-one specs leave the compile-debt ledger, zero regressions at every step.

# gap
10 tagged unions — union(enum) { A: struct {…}, … }
11 a call on an arbitrary expression — func.?(a, b)
12 the inline unroll hint before for / while

union is not a keyword in this lexer, so union(enum) parsed as a call whose argument is the keyword enum — reported as "Unexpected token in expression: KwEnum" pointing into the middle of a type. It is captured verbatim rather than lowered to a struct: its variants carry payload types (RegOperand: struct { reg_num: u8 }), and emitting a union as a struct would give every variant its own storage and silently change the layout. A backend that cannot read the text refuses the declaration, which is the outcome to want.

Gap 11 replaces a comment that read "shouldn't normally happen here." specs/jit/jit.t27 is exactly it: the optional unwrap .? parsed fine — it has since W581 — and the call on its result was left unconsumed, so the statement parser met a bare ( and reported an error past the construct rather than at it. Third time in this campaign that the reported line was not the cause.

Where the seven stand

spec
compiler/parser/lexer.t27 generates
compiler/codegen/verilog/codegen.t27 generates
compiler/codegen/zig/codegen.t27 generates
specs/jit/jit.t27 generates
specs/ternary/bigint.t27 generates
contrib/backend/zig/legacy/main_zig_handwritten.t27 needs an error-union return !void
bootstrap/tests/goldring/cap_test.t27 not a grammar gap — the compiler refuses closures deliberately, naming the remedy in its own message

The last one is worth stating plainly: it is a wanted feature written as a passing test, and it belongs in the ledger until closures exist. No amount of parser work will move it.

FROZEN_HASH was the only conflict, and it is the one kind of conflict
that must not be resolved by choosing a side. Each candidate hashes its
own branch's compiler.rs; after the merge neither describes the file
that exists:

  ours   8e62cacb81c6e84d
  theirs 4f003654a44a4348
  merged 6e2bad56817414a6

Recomputed from the merged bytes. `bootstrap/build.rs` verifies it on
every build, so a picked side would have failed loudly -- but the
reflex to pick one is wrong and worth naming.

The corpus ratchet then failed for the good reason: this branch FIXES
parse for 17 specs the ledger still lists as broken. Five of those
seventeen reach the next check and fail `parse-no-discard` instead --
the same specs, one stage deeper, not new damage. Ledger 220 -> 208:
seventeen removed, five re-entered at the phase they now reach, with
the reason saying which.

check_specs_generate goes 541 -> 562. Elaboration 176, baseline 176,
clean. Verilog widths 2 known, 0 new.

Refs #2427
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-23 16:45:28 UTC

Summary

Status Count
Total Open PRs 4
PRs with Failing Checks 1
PRs with All Checks Green 3
READY 2
FAILING 1
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=6e2bad568174 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@gHashTag
gHashTag merged commit d646071 into master Aug 23, 2026
35 of 36 checks passed
gHashTag added a commit that referenced this pull request Aug 23, 2026
`bootstrap/stage0/FROZEN_HASH` holds sha256 of the file it seals. When
two branches both change `compiler.rs`, git conflicts the seal and
offers two hashes, and BOTH are wrong -- each describes its own side's
file, and the merged file is a third thing:

  ours   8e62cacb81c6e84d
  theirs 4f003654a44a4348
  merged 6e2bad56817414a6

Every reflex a conflict trains -- take ours, take theirs, pick the
newer -- writes a seal for a file that does not exist. `build.rs`
catches it, so the cost is confusion rather than corruption, but the
command exists to remove the temptation.

`tri reseal check` reports three states, not two: matching, mismatched,
and CONFLICTED. The third needs its own message because a conflicted
file trimmed into a comparison reports a plain mismatch against
`<<<<<<< HEAD` and sends the reader to fix the wrong thing.

`tri reseal write` discards both candidates and recomputes. Written
through a temporary and renamed: a seal truncated by an interrupted
write agrees with nothing, and reads exactly like a seal that was never
updated.

Verified in all four states on a real repository -- matching, deliberately
mismatched, conflicted, and conflicted-then-written -- with the seal
returned to its original value afterwards. Three unit tests cover the
conflict recognition, including what `trim()` would have produced.

Skill sections 78 and 79: a derived value in conflict has no correct
side, and a ratchet that fails because you fixed something.

Refs #2427
gHashTag added a commit that referenced this pull request Aug 29, 2026
The gate that would have caught #2427.

    $ tri mods orphan --gate            # at origin/master
    ::error::cli/tri: orphaned rose 0 -> 1. A file left the build and
      nothing else would have said so.
    exit 1

    $ tri mods orphan --gate            # after the fix in this branch
    ORPHAN CEILING: CLEAN
    exit 0

Exact match in both directions, following MAX_SORRY: a count above its ceiling
is a regression, and a count below it is slack the next orphan can hide in.
Both were exercised -- a planted file in cli/tri/src exits 1, and a ceiling
raised to 3 against an actual 0 exits 1 with the instruction to lower it.

`--self-check` is the negative control. It builds a two-file crate in a temp
directory, one declared and one not, and exits non-zero unless exactly the
undeclared file comes back -- so a green run from this gate means the walk was
working, not that it was skipped.

The workflow carries NO `paths:` filter, and the comment says why: the change
that strands a file is a change to the crate ROOT, not to the file that goes
missing. A filter on the stranded path is silent for exactly this defect.

bootstrap's 7 orphans are recorded, not repaired. `tooling.rs` alone holds 7
gate bodies (#2905), and connecting any of them is a decision about what goes
red rather than a repair. The ledger says so in the file, beside the number.

Refs #2900
Refs #2905
gHashTag added a commit that referenced this pull request Aug 29, 2026
…ilent (#2911)

* fix(tri): restore `mod elab;` -- the command has not existed since #2427

d646071 ("[GOLD-RING] parser: the Zig subset lexer.t27 needed", a change to
bootstrap/src/compiler.rs) removed three lines from cli/tri/src/main.rs in one
hunk: the `mod` declaration, the `Elab` variant and the dispatch arm. The
319-line file stayed. Since then:

    $ tri elab classify
    error: unrecognized subcommand 'elab'

Nothing said so. `cargo build` cannot fail on a file it does not compile, and
the test suite went 358 -> 354 without a gate that reads the number. Four tests
left the suite in silence.

`tools/check_elab_ratchet.py` still instructs its reader to run this command, in
its docstring and in its failure output, and two NOW documents describe it as
working. `tri elab` was written because that same script once counted iverilog's
per-file summary line `N error(s) during elaboration.` as a diagnostic -- 25
phantoms in a reported 186, quoted onward before anyone read the rows.

Restored, it reads:

    real diagnostics: 176
    summary lines NOT counted: 28

Nothing else changes: elab.rs is untouched, and its four tests pass as written.

Closes #2900

* feat(tri): `mods orphan` -- source files no crate root reaches

The general form of the defect in the parent commit. Rust compiles a file only
if a `mod` declaration leads to it from the crate root; a file nothing declares
is not unused, it is a file the compiler never opens. It cannot warn, cannot
fail to build, and its `#[test]` functions do not exist for `cargo test`.

    bootstrap     97 files under src/,  90 compiled,   7 orphaned
        bootstrap/src/_enrich/mod.rs                441 lines
        bootstrap/src/math_pslq.rs                  589 lines
        bootstrap/src/neural/attention.rs             4 lines
        bootstrap/src/neural/hslm.rs                  4 lines
        bootstrap/src/runtime_minimal.rs             19 lines
        bootstrap/src/runtime_minimal_test.rs        18 lines
        bootstrap/src/tooling.rs                    409 lines
    cli/tri       33 files under src/,  33 compiled,   0 orphaned

Historical control: the same binary run against origin/master, where `mod elab;`
is still missing, prints

    cli/tri       32 files under src/,  31 compiled,   1 orphaned
        cli/tri/src/elab.rs        319 lines   4 #[test] that cargo cannot see

It fires on the defect and is silent after the fix.

Resolution follows the language rather than a heuristic. `mod c;` in `a/b.rs`
means `a/b/c.rs` or `a/b/c/mod.rs`, never `a/c.rs`; `#[path]` and `include!` are
edges; inline `mod c { }` needs no file. The loose rule -- is this stem named by
any `mod` anywhere -- hides a real orphan behind a same-named module in an
unrelated directory, and its mutation fails the first test here.

`reached` counts files outside `src/` too, because `#[path = "../../gen/..."]`
is a real edge, so the report does not subtract the two sets and call the
difference orphans; it prints the two comparable ones and says where the rest
went.

Refs #2900

* feat(ci): ratchet the orphaned-module count, with a negative control

The gate that would have caught #2427.

    $ tri mods orphan --gate            # at origin/master
    ::error::cli/tri: orphaned rose 0 -> 1. A file left the build and
      nothing else would have said so.
    exit 1

    $ tri mods orphan --gate            # after the fix two commits back
    ORPHAN CEILING: CLEAN
    exit 0

Exact match in both directions, following MAX_SORRY: a count above its ceiling
is a regression, and a count below it is slack the next orphan can hide in.
Both were exercised -- a planted file in cli/tri/src exits 1, and a ceiling
raised to 3 against an actual 0 exits 1 with the instruction to lower it.

`--self-check` is the negative control. It builds a two-file crate in a temp
directory, one declared and one not, and exits non-zero unless exactly the
undeclared file comes back -- so a green run means the walk was working, not
that it was skipped.

The workflow carries NO `paths:` filter, and the comment says why: the change
that strands a file is a change to the crate ROOT, not to the file that goes
missing. A filter on the stranded path is silent for exactly this defect.

bootstrap's 7 orphans are recorded, not repaired. `tooling.rs` alone holds 7
gate bodies (#2905), and connecting any of them is a decision about what goes
red rather than a repair. The ledger says so in the file, beside the number.

Refs #2900
Refs #2905
@gHashTag
gHashTag deleted the feat/zig-subset branch October 26, 2026 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant