fix(lexer): strip backslash-newline line continuations in interpolating strings - #13
Merged
Merged
Conversation
…ng strings Inside a double-quoted / interpolating string literal, a backslash placed immediately before a newline is a line continuation: MRI removes BOTH the backslash and the newline. The escape decoder previously fell through to its default branch and emitted the raw newline byte, so "a\<LF>b" decoded to "a\nb" instead of "ab". Add the continuation branch to scanStringSegment (the interpolating-string decoder, which the double-quoted, %Q/%(), and interpolating-heredoc paths all re-lex through). Handle both the LF and CR-LF forms; a lone \<CR> stays a dropped-backslash + literal CR, matching MRI. The non-interpolating flavours (single-quoted, %q, <<'TERM' heredocs) never reach this decoder and keep the backslash+newline verbatim, also matching MRI. Verified byte-for-byte against ruby 4.0.x across the full string-flavour matrix. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
Inside a double-quoted / interpolating string literal, a backslash placed immediately before a newline is a line continuation — MRI removes both the backslash and the newline. The escape decoder fell through to its
defaultbranch and emitted the raw newline byte, so:Fix
Add the continuation branch to
scanStringSegment(the interpolating-string escape decoder that the double-quoted,%Q/%(), and interpolating-heredoc paths all re-lex through). Both\<LF>and\<CR><LF>are stripped entirely; a lone\<CR>stays dropped-backslash + literal CR, matching MRI. The non-interpolating flavours (single-quoted,%q,<<'TERM'heredocs) never reach this decoder and keep the backslash+newline verbatim — also matching MRI.MRI-match matrix (verified byte-for-byte vs
ruby4.0.5)\+newline"..."double-quoted%Q()/%()<<~/<<-/<<\<CR>(no LF)'...'single-quoted%q()<<'TERM'non-interpolating heredocCovers continuation-then-escape (
\<LF>\t), at end-of-literal, and immediately before#{…}.Gate
go vetclean; builds on all 6 64-bit arches (amd64/arm64/riscv64/loong64/ppc64le/s390x) + js/wasm + wasip1/wasm.🤖 Generated with Claude Code