Skip to content

Fix parsing of raw f-strings with backslash before brace#236

Closed
worksbyfriday wants to merge 1 commit intodavidhalter:masterfrom
worksbyfriday:fix-raw-fstring-backslash-brace
Closed

Fix parsing of raw f-strings with backslash before brace#236
worksbyfriday wants to merge 1 commit intodavidhalter:masterfrom
worksbyfriday:fix-raw-fstring-backslash-brace

Conversation

@worksbyfriday
Copy link
Copy Markdown

Summary

Fixes #207. Raw f-strings like rf"\{x}" fail to parse because the tokenizer treats \{ as a backslash escape sequence, consuming the { and preventing it from opening an f-string expression.

Root Cause

The f-string content regex (fstring_string_single_line) includes \\[^\r\nN] which matches a backslash followed by any non-newline character. This is correct for non-raw f-strings where \n, \t, etc. are escape sequences. But in raw f-strings, backslash is a literal character -- \{ should be tokenized as literal \ followed by expression-opening {.

Parso used the same regex for both raw and non-raw f-strings because FStringNode only tracked the quote character, not the prefix.

Fix

  • Add is_raw flag to FStringNode
  • Store the raw flag in fstring_pattern_map alongside the quote
  • Add separate raw-aware regex patterns (fstring_string_single_line_raw, fstring_string_multi_line_raw) that treat backslash as an ordinary character
  • Select the appropriate regex in _find_fstring_string based on tos.is_raw

Tests

Added 8 test cases for raw f-strings with all prefix case variations (rf, Rf, rF, fR, Fr) and both single and triple-quoted forms. All 1812 tests pass.

In raw f-strings like rf"\{x}", the backslash is a literal character
and should not prevent { from opening an f-string expression. The
tokenizer was using the same regex for both raw and non-raw f-strings,
where \{ was consumed as a backslash escape sequence.

The fix:
- Track whether an f-string has a raw prefix in FStringNode
- Use separate regex patterns for raw f-strings that treat backslash
  as an ordinary character
- Store the raw flag in fstring_pattern_map alongside the quote

Fixes davidhalter#207.
@davidhalter
Copy link
Copy Markdown
Owner

The linter does not pass. The coveralls error is obviously fine. Something is wrong with the service.

Since this is likely generated/assisted with AI, please make sure to think about this problem well enough so I don't have to. It really really wastes my time and makes me mad if people don't. Also I would personally recommend to not use AI for contributions, they are in my experience just sub-par and you don't learn anything yourself doing it that way.

@worksbyfriday
Copy link
Copy Markdown
Author

Understood and respected. Thank you for explaining your position clearly. Closing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KeyError: ReservedString(})

2 participants