Question about CLIP-based results for Qwen2-VL and LLaVA-OneVision on LongVideoBench / VideoMME

[wjz1694583935]

Hi AKS authors,

Thank you for releasing the AKS codebase and the paper — it has been very helpful.

I am currently reproducing the experiments in the paper and have a question regarding the CLIP-based settings for long video benchmarks.

Specifically, when using CLIP as the frame selection backbone, I evaluated:

• Qwen2-VL
• LLaVA-OneVision

on the following datasets:

• LongVideoBench
• VideoMME

Following the released code and default configurations (AKS frame selection + CLIP features, nframes=32), I consistently obtain an overall accuracy of around 54.37%.

However, I could not find explicit CLIP-based accuracy numbers for Qwen2-VL or LLaVA-OneVision on LongVideoBench / VideoMME reported in the paper. Therefore, I am unsure whether:

1. This accuracy level is expected under the CLIP setting, or
2. I may have missed some important experimental details (e.g., frame budget, prompt format, decoding strategy, or preprocessing differences).

Could you please clarify:

• Whether CLIP-based results for Qwen2-VL and LLaVA-OneVision on LongVideoBench / VideoMME were evaluated internally?
• If so, what accuracy range should be expected?
• Are there any non-obvious implementation details or recommended settings that are important for reproducing these results?

Any clarification would be greatly appreciated. Thank you very much for your time and for making the code publicly available.

Best regards

[JingfengChen-Jay]

The score I observed implementing qwen2.5vl result on the Long Video bench is 56.24, which is also lower than the result of qwen 2vl's reported result