Round 5: paren-less/destructuring def params, for loops, command-arg disambiguation (Rails+Puppet parse 98.66% → 99.96%) - #8
Merged
Conversation
…arg disambiguation
Pushes Rails+Puppet parse conformance from 98.66% to 99.96% (5577/5579),
the residual two files being intentional syntax-error test fixtures that
MRI itself rejects — i.e. a literal 100% of all valid Ruby in the corpus.
Parser/lexer additions, each verified against MRI 4.0.x (`ruby -c`) with
MRI-parity table tests:
- Paren-less method-def params, including leading `*`/`**` and destructuring
groups (`def f *a, &b`, `def f((a, b), c)`, anonymous splat in a group),
expanded to a leading multiple-assignment prepended to the body.
- Character-literal escapes `?\xHH`, `?\uHHHH`, `?\u{…}`, `?\NNN`, `?\C-x`,
`?\M-x`, `?\M-\C-x` (the consumers swallow the full escape).
- Empty `||` and block-local variables after `;` in a block parameter list.
- A nested regexp inside an interpolation inside a regexp / %r-literal, via a
verbatim brace-balanced interpolation copy that skips strings.
- `yield(...)` as a paren-less command argument.
- RSpec `it { … }` / `it do … end`: a bare `it` taking a block is a method
call, not the Ruby-3.4 implicit block parameter.
- A bareword that is a known local, or a constant, promoted back to a method
call by an unambiguous following command argument (`type = 1; type 'X'`),
and a local-name `do` block (`fork = false; pid = fork do … end`).
- Keyword/assignment argument values that no longer swallow the following
argument; command-argument masgn suppression (`assert_equal a, tag = b`).
- A `key:` value on the next line inside a call or hash, distinct from the
value-omitted shorthand.
- A paren-less setter `def x=` whose body starts on the next line (the setter
`=` no longer continues the line), and keyword-named setters (`def ensure=`).
- The `for VAR[, …] in ITER [do] … end` loop (new ast.For node).
- `super`/`yield` followed by a space-flanked `/` parsed as division, not a
regexp argument.
- One-line pattern match in a condition (`if node in Foo[...]`) and bracket
patterns spanning newlines.
- An assignment / op-assignment as a range endpoint (`@i...@i += n`).
100% coverage maintained (CI gate), gofmt + go vet clean, CGO=0, all suites
race-clean. End-to-end verified by rebuilding rbgo against this tree.
rbgo follow-up: the new ast.For node needs a compiler case in
go-embedded-ruby (parsing is complete; compilation of `for` is not yet wired).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Round-5 straggler pass for go-embedded-ruby parse conformance. Rails+Puppet
parse rate rises 98.66% → 99.96% (5577/5579 files). The remaining 2
files are intentional syntax-error test fixtures that MRI itself rejects,
so this is a literal 100% of all valid Ruby in the corpus.
Every feature was verified against MRI 4.0.x (
ruby -c) and is pinned by anMRI-parity table test in
round5_test.go.Features landed
*/**,&block)def initialize *names,def g *a, &bdef f((a, b), c),def k((a, *), c)?\x00,?\u{1f600},?\C-a,?\M-\C-a%r"…"/^#{x[/\A\s*/]}/,%r"#{Regexp.escape 'a "b"'}"yield(...)as a paren-less command argumentraise yield('x')it { … }/it do … end(call, not implicit param)it { is_expected.to eq 1 }type = 1; type 'X',raise ArgumentError urn.inspectdoblockfork = false; pid = fork do … endf a: x = 1, b: y = 2,assert_equal a, tag = bkey:value on the next line (vs shorthand)f(\n k:\n v)def x=with next-line body; keyword settersdef ensure=(v)for VAR[, …] in ITER [do] … endloop (newast.For)for i in 1..100 … endsuper/yield/as division (vs regexp arg)super / @step_sizeif node in Foo[a: 1]@i...@i += nRails + Puppet parse: before / after
(5579
.rbfiles across/tmp/conf-repos/rails+/tmp/conf-repos/puppet.)Irreducibility report — the final 2 files
All remaining failures classified into the requested buckets:
(a) clean grammar feature still addable: 0. Every clean feature found in
the corpus was implemented this round.
(b) multi-issue: 0. The multi-issue files this round each reduced to a
single shared root cause once fixed (e.g. every Puppet
pops/types/*.rbfailure was the one "known-local followed by a command argument" rule; the
super / xdivision rule unblocked another).(c) MRI-rejects-in-isolation / intentional fixtures: 2.
puppet/spec/fixtures/unit/.../func_with_syntax_error.rb— contains1+ + + +under the comment# this syntax error is here on purpose!;MRI:
SyntaxError.puppet/spec/fixtures/faulty_face/puppet/face/syntax.rb— a deliberatelyomitted
end(# This 'end' is deliberately omitted, to induce a syntax error. Please don't fix that); MRI:SyntaxError.Our parser correctly rejects both, so 99.96% is the literal ceiling on this
corpus; the gap to 100% is unreachable by design, not asymptotic.
rbgo compiler follow-up
The parser is complete for all features. rbgo rebuilds against this tree and
runs the new constructs end-to-end (paren-less/destructuring params,
label-on-next-line, empty
||, char escapes all execute). The one follow-up:the new
ast.Fornode needs a compiler case in go-embedded-ruby(
cannot compile *ast.For) — parsing is done, code generation forforis notyet wired.
Quality
>= 100.0).go test -race ./...green; gofmt +go vetclean; CGO=0.round5_test.go.🤖 Generated with Claude Code