Repository navigation
Keep a capturing group's escapes when reversing a conversion pattern - #49772
Conversation
5c9bef1 to
9837471
Compare
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
9837471 to
98eb6d8
Compare
98eb6d8 to
e6e57ea
Compare
| # an unescaped `.` would match any character, so `gateX` would be renamed as well | ||
| self.assertEqual(reverse.rename_source_key("model.mlp.gateX.w")[0], "model.mlp.gateX.w") |
There was a problem hiding this comment.
wait why would we want this? based gate\. shouldnt this only match gate. as it's an escaped dot not a general regex dot?
There was a problem hiding this comment.
the commetn described the behaviour on main, it's checking exactly what you described, model.mlp.gateX.w stays as is
There was a problem hiding this comment.
updated the comment
e6e57ea to
d8ab562
Compare
f44141e to
ae330b5
Compare
ae330b5 to
db967b4
Compare
db967b4 to
ec304d1
Compare
process_target_pattern unescaped \. before extracting the capturing group, so the reversed rename's group matched any character in place of a dot. Unescape only outside the group. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ec304d1 to
d3f29df
Compare
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cyrilvallez
left a comment
There was a problem hiding this comment.
LGTM thanks! In general aligned with this change as it makes it more robust, though for minimax m3 a simple change in the patterns will look a bit better instead, see my comment/commit there!
What.
process_target_patternunescaped\.before it took out the capturing group. The reversed group thenmatched any character where the pattern meant a dot. The unescape now runs after the group is taken out.
Why we need it. Saving MiniMax-M3 MXFP8 checkpoints in #49780 blocked on it. Its MoE rename (#49786) is the first
reversible mapping with an escaped dot in a capturing group. Reversed without this fix, the group matched any
character in place of the dot, and the save wrote wrong keys. No mapping on main has such a group yet.
Depends on. #49771 (stacked for a linear stack; no code dependency).
How tested.
tests/utils/test_core_model_loading.py::test_reverse_keeps_escaped_dots_in_capturing_group. The-k test_reverse_loading_mappingmodel tests give the same results as main.The comments and docstrings that say why each change exists are there for readability. Feel free to ask for any of them to go if it's self-evident.
🤖 Generated with Claude Code