Skip to content

Keep a capturing group's escapes when reversing a conversion pattern - #49772

Merged
IlyasMoutawwakil merged 5 commits into
mainfrom
fg-split/fix-reverse-escaping
Oct 27, 2026
Merged

IlyasMoutawwakil merged 5 commits into
mainfrom
fg-split/fix-reverse-escaping

Conversation

@IlyasMoutawwakil

@IlyasMoutawwakil IlyasMoutawwakil commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

🤗 Transformers CI

What. process_target_pattern unescaped \. before it took out the capturing group. The reversed group then
matched any character where the pattern meant a dot. The unescape now runs after the group is taken out.

Why we need it. Saving MiniMax-M3 MXFP8 checkpoints in #49780 blocked on it. Its MoE rename (#49786) is the first
reversible mapping with an escaped dot in a capturing group. Reversed without this fix, the group matched any
character in place of the dot, and the save wrote wrong keys. No mapping on main has such a group yet.

Depends on. #49771 (stacked for a linear stack; no code dependency).

How tested. tests/utils/test_core_model_loading.py::test_reverse_keeps_escaped_dots_in_capturing_group. The
-k test_reverse_loading_mapping model tests give the same results as main.

The comments and docstrings that say why each change exists are there for readability. Feel free to ask for any of them to go if it's self-evident.

🤖 Generated with Claude Code

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Comment thread tests/utils/test_core_model_loading.py Outdated
Comment on lines +183 to +184
# an unescaped `.` would match any character, so `gateX` would be renamed as well
self.assertEqual(reverse.rename_source_key("model.mlp.gateX.w")[0], "model.mlp.gateX.w")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait why would we want this? based gate\. shouldnt this only match gate. as it's an escaped dot not a general regex dot?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the commetn described the behaviour on main, it's checking exactly what you described, model.mlp.gateX.w stays as is

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the comment

@IlyasMoutawwakil
IlyasMoutawwakil force-pushed the fg-split/fix-reverse-escaping branch from e6e57ea to d8ab562 Compare October 20, 2026 16:04
@IlyasMoutawwakil
IlyasMoutawwakil removed this pull request from stack #49788 October 20, 2026 16:25
@IlyasMoutawwakil
IlyasMoutawwakil force-pushed the fg-split/fix-reverse-escaping branch 2 times, most recently from f44141e to ae330b5 Compare October 21, 2026 07:24
@IlyasMoutawwakil
IlyasMoutawwakil marked this pull request as ready for review October 21, 2026 07:30
@IlyasMoutawwakil
IlyasMoutawwakil force-pushed the fg-split/fix-reverse-escaping branch from ae330b5 to db967b4 Compare October 21, 2026 07:52
@IlyasMoutawwakil
IlyasMoutawwakil removed this pull request from stack #49798 October 21, 2026 08:59
@IlyasMoutawwakil
IlyasMoutawwakil force-pushed the fg-split/fix-reverse-escaping branch from db967b4 to ec304d1 Compare October 23, 2026 13:03
Base automatically changed from fg-split/fix-skip-list-index to main October 23, 2026 13:34
IlyasMoutawwakil and others added 2 commits October 23, 2026 13:34
process_target_pattern unescaped \. before extracting the capturing group,
so the reversed rename's group matched any character in place of a dot.
Unescape only outside the group.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@IlyasMoutawwakil
IlyasMoutawwakil force-pushed the fg-split/fix-reverse-escaping branch from ec304d1 to d3f29df Compare October 23, 2026 13:35
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@Cyrilvallez Cyrilvallez left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks! In general aligned with this change as it makes it more robust, though for minimax m3 a simple change in the patterns will look a bit better instead, see my comment/commit there!

@IlyasMoutawwakil
IlyasMoutawwakil added this pull request to the merge queue Oct 27, 2026
Merged via the queue into main with commit 9ed172c Oct 27, 2026
122 checks passed
@IlyasMoutawwakil
IlyasMoutawwakil deleted the fg-split/fix-reverse-escaping branch October 27, 2026 10:38
IlyasMoutawwakil added a commit that referenced this pull request Oct 27, 2026
… small loader fixes

Rebased onto the merged reverse-escaping fix (#49772) and the rebased
MiniMax-M3-VL save-mapping PR. The reverse loading path stays main's (#49928);
this keeps the PR's other loader fixes and their tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IlyasMoutawwakil added a commit that referenced this pull request Oct 27, 2026
… small loader fixes

Rebased onto the merged reverse-escaping fix (#49772) and the rebased
MiniMax-M3-VL save-mapping PR. The reverse loading path stays main's (#49928);
this keeps the PR's other loader fixes and their tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants