Repository navigation

[Model] Lfm2Moe - #41401

Merged

Cyrilvallez merged 30 commits into

huggingface:mainfrom

paulpak58:lfm2_moe

Oct 7, 2025

Commits on Oct 6, 2025

[new-models] LFM2-MoE
paulpak58
committed
d3e71a9
[docs] add in template lfm2_moe doc files
paulpak58
committed
ee096db
[configuration] update configuration class
paulpak58
committed
1ef7c59
[modular][lfm] minor: fix rotary_emb typo
paulpak58
committed
d42252a
[modeling] modular/modeling files for Lfm2Moe
paulpak58
committed
79b3f20
[modeling][lfm2_moe] fix Lfm2Moe modular/modeling
paulpak58
committed
5bc8e0d
[configuration][lfm2_moe] update configuration keys with latest config changes
paulpak58
committed
bfd559a
[misc] make fixup
paulpak58
committed
e410997
[modular][lfm2_moe] address comments: dtype, mlp, buffers
paulpak58
committed
7888413
[configuration][lfm2_moe] add initializer_range
paulpak58
committed
d048b12
[modular][lfm2_moe] include init_weights to pass test_initialization
paulpak58
committed
c048622
[tests][causal_lm] include pos_emb as possible rope attribute
paulpak58
committed
d10a8c9
[modeling][lfm2_moe] remove load_balancing_loss_func due to lack of support for hooking expert biases
paulpak58
committed
7114e62
[misc] make style
paulpak58
committed
1b74cee
[modeling][lfm2_moe] MoE refactor PR update in LFM2Moe
paulpak58
committed
fb65ff5

Commits on Oct 7, 2025

[tests] lfm2_moe: unit tests
paulpak58
committed
f4785d9
[misc] update LFM2-8B-A1B repo id
paulpak58
committed
dd63210
[tests] lfm2: update ModelTests for lfm2
paulpak58
committed
fc7d2ea
Update LFM2 documentation
mlabonne
authored
daa35aa
Add Lfm2Moe documentation
mlabonne
authored
96fbd87
[misc] fix ci
paulpak58
committed
c7bac38
[docs] remove trust_remote_code
paulpak58
committed
37d2fff
[misc] ci: fix modular
paulpak58
committed
23c2c99
Merge branch 'main' into lfm2_moe
Cyrilvallez
authored
1427f48
reapply modular
Cyrilvallez
committed
8b6aa83
simplify
Cyrilvallez
committed
cd873d4
remove static address and inplace op
Cyrilvallez
committed
0c67e26
simplify
Cyrilvallez
committed
44ff5d2
simplify a bit more the modular
Cyrilvallez
committed
38130f5
imports
Cyrilvallez
committed
b3a0924