Skip to content

[Proposal] Hybrid architectures warn on unresolved optional 'hook_attn' aliases when they are skipped #1578

Closed

Description

@KYinXu

Proposal

When setup_submodules skips an optional submodule, prune that block’s hook_aliases whose target path starts at the skipped key (e.g. drop hook_attn_out → attn.hook_out when attn was skipped). Copy the aliases dict per layer so templates don’t share mutations; leave residual/MLP aliases alone. Full-attention siblings keep HT aliases; GDN native aliases are unchanged. Cover with unit tests for prune + no warning on register.

Motivation

When loading models like Qwen-3.5 Hybrid, a warning is printed per hybrid layer, cluttering the terminal quickly (as noted in #1547). The proposed solution would remove unused aliases since it is expected behavior for GDN layers to use their native hook_linear_attn_out instead.

UserWarning: Hook alias 'hook_attn_out' -> 'attn.hook_out' on BlockBridge(name='model.language_model.layers.2') did not resolve; this hook will not be accessible.
  getattr(module, "_register_aliases")()

Pitch

Loading hybrids like Qwen3.5 spam harmless hook_attn_out did not resolve warnings on every GDN/Mamba-2 layer. Those layers never had full attn; they expose hook_linear_attn_out (and similar) instead. Treat that as expected, not a fault, so boot stays quiet without hiding real alias bugs on full-attention layers.

Alternatives

The warning could be left as-is, as this is not a functional error and the user might want to be notified at build-time that the standard hook_attn_out alias will be missing, though it could be caught and reworded to reflect its non-breaking nature.

Checklist

  • I have checked that there is no similar issue in the repo (required)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

TransformerBridgeBug specific to the new TransformerBridge systembugSomething isn't workingcomplexity-simpleSimple issues, which may be good for beginners

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions