Skip to content

Round 5: paren-less/destructuring def params, for loops, command-arg disambiguation (Rails+Puppet parse 98.66% → 99.96%) - #8

Merged
tannevaled merged 1 commit into
mainfrom
parser-100pct-round5
Jun 26, 2026
Merged

Round 5: paren-less/destructuring def params, for loops, command-arg disambiguation (Rails+Puppet parse 98.66% → 99.96%)#8
tannevaled merged 1 commit into
mainfrom
parser-100pct-round5

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

Summary

Round-5 straggler pass for go-embedded-ruby parse conformance. Rails+Puppet
parse rate rises 98.66% → 99.96% (5577/5579 files). The remaining 2
files are intentional syntax-error test fixtures that MRI itself rejects,
so this is a literal 100% of all valid Ruby in the corpus.

Every feature was verified against MRI 4.0.x (ruby -c) and is pinned by an
MRI-parity table test in round5_test.go.

Features landed

Feature Example
Paren-less def params (+ leading */**, &block) def initialize *names, def g *a, &b
Destructuring def params (incl. nesting / anon splat) def f((a, b), c), def k((a, *), c)
Char-literal escapes hex/unicode/octal/control/meta ?\x00, ?\u{1f600}, ?\C-a, ?\M-\C-a
Empty `
Nested regexp in interpolation in regexp / %r"…" /^#{x[/\A\s*/]}/, %r"#{Regexp.escape 'a "b"'}"
yield(...) as a paren-less command argument raise yield('x')
RSpec it { … } / it do … end (call, not implicit param) it { is_expected.to eq 1 }
Local/const promoted to a call by a command arg type = 1; type 'X', raise ArgumentError urn.inspect
Local-name do block fork = false; pid = fork do … end
Keyword/assignment arg value not swallowing the next arg f a: x = 1, b: y = 2, assert_equal a, tag = b
key: value on the next line (vs shorthand) f(\n k:\n v)
Paren-less setter def x= with next-line body; keyword setters def ensure=(v)
for VAR[, …] in ITER [do] … end loop (new ast.For) for i in 1..100 … end
super/yield / as division (vs regexp arg) super / @step_size
One-line pattern match in a condition; multiline bracket patterns if node in Foo[a: 1]
Assignment/op-assignment as a range endpoint @i...@i += n

Rails + Puppet parse: before / after

Before After
OK 5504 5577
Fail 75 2
Rate 98.66% 99.96%

(5579 .rb files across /tmp/conf-repos/rails + /tmp/conf-repos/puppet.)

Irreducibility report — the final 2 files

All remaining failures classified into the requested buckets:

  • (a) clean grammar feature still addable: 0. Every clean feature found in
    the corpus was implemented this round.

  • (b) multi-issue: 0. The multi-issue files this round each reduced to a
    single shared root cause once fixed (e.g. every Puppet pops/types/*.rb
    failure was the one "known-local followed by a command argument" rule; the
    super / x division rule unblocked another).

  • (c) MRI-rejects-in-isolation / intentional fixtures: 2.

    • puppet/spec/fixtures/unit/.../func_with_syntax_error.rb — contains
      1+ + + + under the comment # this syntax error is here on purpose!;
      MRI: SyntaxError.
    • puppet/spec/fixtures/faulty_face/puppet/face/syntax.rb — a deliberately
      omitted end (# This 'end' is deliberately omitted, to induce a syntax error. Please don't fix that); MRI: SyntaxError.

    Our parser correctly rejects both, so 99.96% is the literal ceiling on this
    corpus; the gap to 100% is unreachable by design, not asymptotic.

rbgo compiler follow-up

The parser is complete for all features. rbgo rebuilds against this tree and
runs the new constructs end-to-end (paren-less/destructuring params,
label-on-next-line, empty ||, char escapes all execute). The one follow-up:
the new ast.For node needs a compiler case in go-embedded-ruby
(cannot compile *ast.For) — parsing is done, code generation for for is not
yet wired.

Quality

  • 100% coverage maintained (CI gate >= 100.0).
  • go test -race ./... green; gofmt + go vet clean; CGO=0.
  • New MRI-parity tests in round5_test.go.

🤖 Generated with Claude Code

…arg disambiguation

Pushes Rails+Puppet parse conformance from 98.66% to 99.96% (5577/5579),
the residual two files being intentional syntax-error test fixtures that
MRI itself rejects — i.e. a literal 100% of all valid Ruby in the corpus.

Parser/lexer additions, each verified against MRI 4.0.x (`ruby -c`) with
MRI-parity table tests:

- Paren-less method-def params, including leading `*`/`**` and destructuring
  groups (`def f *a, &b`, `def f((a, b), c)`, anonymous splat in a group),
  expanded to a leading multiple-assignment prepended to the body.
- Character-literal escapes `?\xHH`, `?\uHHHH`, `?\u{…}`, `?\NNN`, `?\C-x`,
  `?\M-x`, `?\M-\C-x` (the consumers swallow the full escape).
- Empty `||` and block-local variables after `;` in a block parameter list.
- A nested regexp inside an interpolation inside a regexp / %r-literal, via a
  verbatim brace-balanced interpolation copy that skips strings.
- `yield(...)` as a paren-less command argument.
- RSpec `it { … }` / `it do … end`: a bare `it` taking a block is a method
  call, not the Ruby-3.4 implicit block parameter.
- A bareword that is a known local, or a constant, promoted back to a method
  call by an unambiguous following command argument (`type = 1; type 'X'`),
  and a local-name `do` block (`fork = false; pid = fork do … end`).
- Keyword/assignment argument values that no longer swallow the following
  argument; command-argument masgn suppression (`assert_equal a, tag = b`).
- A `key:` value on the next line inside a call or hash, distinct from the
  value-omitted shorthand.
- A paren-less setter `def x=` whose body starts on the next line (the setter
  `=` no longer continues the line), and keyword-named setters (`def ensure=`).
- The `for VAR[, …] in ITER [do] … end` loop (new ast.For node).
- `super`/`yield` followed by a space-flanked `/` parsed as division, not a
  regexp argument.
- One-line pattern match in a condition (`if node in Foo[...]`) and bracket
  patterns spanning newlines.
- An assignment / op-assignment as a range endpoint (`@i...@i += n`).

100% coverage maintained (CI gate), gofmt + go vet clean, CGO=0, all suites
race-clean. End-to-end verified by rebuilding rbgo against this tree.

rbgo follow-up: the new ast.For node needs a compiler case in
go-embedded-ruby (parsing is complete; compilation of `for` is not yet wired).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 1bbe8b4 into main Jun 26, 2026
9 checks passed
@tannevaled
tannevaled deleted the parser-100pct-round5 branch June 26, 2026 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant