Describe the bug
I pretrained a ViT using GMML method(https://github.com/Sara-Ahmed/GMML). I would like to be able to use this pretrained model as the ViT for UNETr. For every layer I am unable to load checkpoints forvit.blocks.1.attn.qkv.bias.
This largely due to on line
|
self.qkv = nn.Linear(hidden_size, hidden_size * 3, bias=False) |
The bias term has been set
False and their no way to access it from the higher level UnetR.
Describe the bug
I pretrained a ViT using GMML method(https://github.com/Sara-Ahmed/GMML). I would like to be able to use this pretrained model as the ViT for UNETr. For every layer I am unable to load checkpoints for
vit.blocks.1.attn.qkv.bias.This largely due to on line
MONAI/monai/networks/blocks/selfattention.py
Line 45 in 4a38e11
The bias term has been set
Falseand their no way to access it from the higher level UnetR.