ctx.ast(): support { comments: true } for Ruby

[rhuanbarreto]

Summary

ctx.ast(path, "ruby", { comments: true }) currently throws unconditionally — commentsUnsupportedError (src/engine/ast-support.ts:35) is raised from a guardrail check in runner.ts before the Ruby serializer is ever invoked (runner.ts:241-243). TypeScript/JavaScript and Python already support { comments: true }; Ruby is the one AstLanguage left out, closing off any comment-governance rule (comment-length/style policies, "explanatory comment exempts this construct" patterns) for Ruby codebases specifically.

Background

The current per-language comment story:

• TypeScript/JavaScript (src/engine/js-parser.ts) — a hand-rolled scanner extracts // line and /* … */ block comments directly (collectComments, js-parser.ts:46-127), attached as a comments array on the returned tree.
• Python (src/engine/ast-support.ts:113-139) — ast.parse() carries no comment nodes, so PYTHON_AST_WITH_COMMENTS_PROGRAM runs the tokenize module as a second, independent pass over the same source and emits {"_tree": ..., "comments": [...]}. Tokenizer errors on otherwise-parseable source degrade to an empty comment list rather than failing the whole parse.
• Ruby (src/engine/ast-support.ts:142-155) — RUBY_AST_PROGRAM runs Ripper.sexp(source), which — like Python's ast — carries no comment nodes at all. No second pass exists. runner.ts short-circuits with commentsUnsupportedError before even reaching the interpreter.

This was a deliberate, explicit scope cut when comments support first shipped (issue #479 named it directly: "Ruby — Ripper.lex (not Ripper.sexp) surfaces comment tokens... [this] warrants its own design pass") rather than an oversight — but it leaves Ruby as the only language where ctx.ast() cannot back a comment-aware rule at all.

The gap in practice

A rule that needs "is there a human-authored comment near this construct that exempts it" (the same shape used by Python-based rules exempting an otherwise-bare except: pass when a comment explains why) has no path to that information for Ruby source — { comments: true } fails immediately, and there's no fallback except a hand-rolled regex/line-scan over raw source, the exact kind of fragile heuristic ctx.ast() exists to replace.

Proposed design

Mirror Python's two-pass approach, using Ripper.lex (not Ripper.sexp) as the second, comment-only pass:

require "ripper"
require "json"

source = File.read(ARGV[0], mode: "r:bom|utf-8")
sexp = Ripper.sexp(source)
if sexp.nil?
  warn "Ruby syntax error"
  exit 1
end

comments = []
begin
  Ripper.lex(source).each do |(line, col), event, tok, _state|
    next unless event == :on_comment
    comments << {
      type: "line",
      value: tok.sub(/\A#/, "").chomp,
      loc: {
        start: { line: line, column: col },
        end: { line: line, column: col + tok.length },
      },
    }
  end
rescue StandardError
  # Degrade to an empty comment list, matching Python's tokenizer-error
  # fallback -- a comment-extraction failure must not fail the whole parse.
end

puts JSON.generate({ tree: sexp, comments: comments }, max_nesting: false)

Notes:

• Ripper.lex returns [[line, col], event, token, state] tuples; :on_comment events carry the full #... token including trailing newline (needs chomp), matching Python's value semantics (leading #/marker stripped, per commentsUnsupportedError's stated parity goal).
• Ruby has line comments only (#), same as Python — no block-comment variant needed (unlike TS/JS), so the CommentToken.type is always "line", same as Python.
• Multi-line =begin/=end block comments are a distinct Ripper lex event (:on_embdoc/:on_embdoc_beg/:on_embdoc_end, not :on_comment) — worth deciding explicitly whether to include them (as type: "block") or scope them out for a first pass, rather than silently missing them.
• Remove the runner.ts:241-243 early throw for language === "ruby" once the serializer supports it; commentsUnsupportedError itself can likely be deleted if this closes the last unsupported case (check its other call sites first).

Acceptance criteria

• ctx.ast(path, "ruby", { comments: true }) returns { ...tree, comments: CommentToken[] } instead of throwing.
• Comment value has the leading # stripped, matching Python/TS-JS convention.
• A lex failure on otherwise-parseable source degrades to an empty comment list rather than failing the parse (matches Python's tokenizer-error handling).
• Explicit decision (not silent omission) on =begin/=end block comments — included with type: "block", or documented as out of scope.
• runner.ts's early commentsUnsupportedError throw for Ruby is removed; the error helper is deleted or updated if Ruby was its last remaining case.
• Documented in docs/*/reference/rule-api.mdx (all three locales) as a supported language for { comments: true }.
• Test coverage: a Ruby fixture with line comments (and, per the block-comment decision above, =begin/=end if included) confirms correct loc/value extraction.

Out of scope

• Unifying comment shape guarantees beyond CommentToken (already shared across languages).
• Any change to Ripper.sexp's tree output — this only adds a parallel comment-collection pass, same as Python's approach.

References

• ctx.ast(): add base-revision access so sandbox hardening (#477) doesn't strand legitimate rules #479 — original base-revision + comments proposal, explicitly scoping Ruby comments out as a follow-up ("Gap 2" design B)
• src/engine/ast-support.ts — commentsUnsupportedError, RUBY_AST_PROGRAM, PYTHON_AST_WITH_COMMENTS_PROGRAM (the pattern to mirror)
• src/engine/runner.ts:241-243 — the guardrail throw to remove
• src/engine/js-parser.ts:46-127 — collectComments, the TS/JS analog