Repository navigation
Question regarding the aggregation strategy in _get_attn_k() #16
Description
This is a very insightful piece of work! It's impressive to see robust motion segmentation and quality enhancement for dynamic point clouds achieved in a training-free manner.
I have a question regarding the implementation details. I noticed that when aggregating the cross-attention maps for a single image (referring to the _get_attn_k() function in model.py), the aggregation is performed along the query dimension rather than the key dimension.
Is there a specific reason or intuition behind this design choice? I would really appreciate it if you could share your insights. Thanks!
Activity
rover-xingyu commentedon Dec 2, 2025
Hi @zhangshuoneu, thanks for your interest.
Intuitively, since the attention map governs the information flow from
keytoquery, averaging along the query dimension allows us to assess how much each token in thekey viewcontributes to thequery viewas a whole. This effectively highlights the spatial significance of each token in terms of its contribution.zhangshuoneu commentedon Dec 2, 2025
Thanks for your reply.
If I understand correctly, doing so produces an image coordinate system aligned with the key view. Given that cross-attention tokens are typically aligned with the query view, wouldn't it be more reasonable to extract features that align with the query view instead?
rover-xingyu commentedon Dec 2, 2025
You are right. Averaging along the query dimension indeed produces a map aligned with the Key view.
Our specific goal here is to evaluate the importance of each token in terms of contribution (
Key) rather than gain (Query).zhangshuoneu commentedon Dec 3, 2025
Thanks for the reply!
Following up on that, have you experimented with aggregating along the key dimension instead? I'm curious if that yields different performance. If so, would it be possible to share any comparisons?