You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Proposal] Hybrid architectures warn on unresolved optional 'hook_attn' aliases when they are skipped #1578
When setup_submodules skips an optional submodule, prune that block’s hook_aliases whose target path starts at the skipped key (e.g. drop hook_attn_out → attn.hook_out when attn was skipped). Copy the aliases dict per layer so templates don’t share mutations; leave residual/MLP aliases alone. Full-attention siblings keep HT aliases; GDN native aliases are unchanged. Cover with unit tests for prune + no warning on register.
Motivation
When loading models like Qwen-3.5 Hybrid, a warning is printed per hybrid layer, cluttering the terminal quickly (as noted in #1547). The proposed solution would remove unused aliases since it is expected behavior for GDN layers to use their native hook_linear_attn_out instead.
UserWarning: Hook alias 'hook_attn_out' -> 'attn.hook_out' on BlockBridge(name='model.language_model.layers.2') did not resolve; this hook will not be accessible.
getattr(module, "_register_aliases")()
Pitch
Loading hybrids like Qwen3.5 spam harmless hook_attn_out did not resolve warnings on every GDN/Mamba-2 layer. Those layers never had full attn; they expose hook_linear_attn_out (and similar) instead. Treat that as expected, not a fault, so boot stays quiet without hiding real alias bugs on full-attention layers.
Alternatives
The warning could be left as-is, as this is not a functional error and the user might want to be notified at build-time that the standard hook_attn_out alias will be missing, though it could be caught and reworded to reflect its non-breaking nature.
Checklist
I have checked that there is no similar issue in the repo (required)
Activity