Fix problem- llava_vid

[amitshomer]

Hi,
Thanks for the great work!
Inside you custom llava_vid.py under load_video_index function you have the following code:

        if len(frame_idx) < max_frames_num:
            sample_fps = max_frames_num
            uniform_sampled_frames = np.linspace(0, total_frame_num - 1, sample_fps, dtype=int)
            frame_idx = uniform_sampled_frames.tolist()
            frame_time = [i / vr.get_avg_fps() for i in frame_idx]

Which mean any selected frame smaller than the maximal value will change to uniform sample, while the logic should be vise versa. For example we created 32 selected frames using yours logic, but while the max_frames_num=64 it will load always 64 uniform frames.

[ncTimTang]

Hi @amitshomer ,

Thank you for your interest in our work!

You are right! This do will cause any selected frame smaller than the maximal value will change to uniform sample.

We do this, because, for some videos (e.g. the length is 62s) in the test set, when sampling is performed according to the set sampling rate (1fps), the number of sampled frames in some videos (e.g. 62 frames) is less than the set number of frames (e.g. 64 frames). In this case, it is actually uniform sampling, and no frame selection occurs. However, the number of sampled frames is still less than that of the uniform sampling (e.g. 64 frames) used for comparison. Therefore, in such cases, we use uniform sampling to ensure that the number of sampled frames is consistent across all videos, thereby ensuring fairness in the comparison.

Hope these can help you.

Best,

Tim