Fix parsing of raw f-strings with backslash before brace#236
Closed
worksbyfriday wants to merge 1 commit intodavidhalter:masterfrom
Closed
Fix parsing of raw f-strings with backslash before brace#236worksbyfriday wants to merge 1 commit intodavidhalter:masterfrom
worksbyfriday wants to merge 1 commit intodavidhalter:masterfrom
Conversation
In raw f-strings like rf"\{x}", the backslash is a literal character
and should not prevent { from opening an f-string expression. The
tokenizer was using the same regex for both raw and non-raw f-strings,
where \{ was consumed as a backslash escape sequence.
The fix:
- Track whether an f-string has a raw prefix in FStringNode
- Use separate regex patterns for raw f-strings that treat backslash
as an ordinary character
- Store the raw flag in fstring_pattern_map alongside the quote
Fixes davidhalter#207.
Owner
|
The linter does not pass. The coveralls error is obviously fine. Something is wrong with the service. Since this is likely generated/assisted with AI, please make sure to think about this problem well enough so I don't have to. It really really wastes my time and makes me mad if people don't. Also I would personally recommend to not use AI for contributions, they are in my experience just sub-par and you don't learn anything yourself doing it that way. |
Author
|
Understood and respected. Thank you for explaining your position clearly. Closing this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #207. Raw f-strings like
rf"\{x}"fail to parse because the tokenizer treats\{as a backslash escape sequence, consuming the{and preventing it from opening an f-string expression.Root Cause
The f-string content regex (
fstring_string_single_line) includes\\[^\r\nN]which matches a backslash followed by any non-newline character. This is correct for non-raw f-strings where\n,\t, etc. are escape sequences. But in raw f-strings, backslash is a literal character --\{should be tokenized as literal\followed by expression-opening{.Parso used the same regex for both raw and non-raw f-strings because
FStringNodeonly tracked the quote character, not the prefix.Fix
is_rawflag toFStringNodefstring_pattern_mapalongside the quotefstring_string_single_line_raw,fstring_string_multi_line_raw) that treat backslash as an ordinary character_find_fstring_stringbased ontos.is_rawTests
Added 8 test cases for raw f-strings with all prefix case variations (
rf,Rf,rF,fR,Fr) and both single and triple-quoted forms. All 1812 tests pass.