[Bug]: Linux screen capture time-compresses under frame drops, desyncing webcam/audio/cursor #511

Describe the bug

On Linux (PipeWire capture), the recorded screen video silently time-compresses when the capture pipeline drops frames under load, so the screen track ends up shorter than real elapsed time and plays fast, drifting ahead of the webcam, audio, and the cursor overlay.

The screen encoder writes constant frame rate with PTS = frame index (electron/native/pipewire-capture/src/encoder.rs, time_base = 1/fps). The pacing loop (capture.rs::advance + current_index) is meant to hold the last picture across gaps and backfill missed 60 fps ticks, but the backfill is bounded (MAX_CATCHUP_FRAMES = 8 per advance() call) and advance() only runs from two event-loop arms (FrameReady and the idle Timeout heartbeat). Under sustained GPU/CPU load the loop is starved and the catch-up cannot keep up, so next_index — which is simultaneously the PTS and the frame counter — permanently lags the wall clock. The lost real-time interval simply disappears: file duration == frames_encoded / fps, with no dependence on elapsed time, and nothing reconciles the deficit at stop.

Because every other track is wall-clock based (audio sample-count PTS, webcam WebM, cursor timeMs), the screen is the only track on a "frame-index" clock, so the moment any frame drops it drifts against everything else and cameraTrack.offsetMs (a small constant nudge) cannot compensate a proportional stretch.

Field-diagnosed measurements from one real recording (61.06 s session):

~350 video frames (~5.9 s) were silently lost and the survivors stamped at a uniform 60 fps → 61 s of activity compressed into a 55.2 s file that plays ~10% fast. Three other same-session recordings that didn't hit the load had screen/webcam deltas of only ~80–90 ms and looked fine.

Expected behavior

The recorded screen video's duration should equal real elapsed wall-clock time even when frames are dropped, so it stays in sync with audio, webcam, and the cursor overlay. Dropped frames should be represented as timing (a longer-held frame), never as removed time.

To Reproduce

1. On Linux/PipeWire, record the screen under enough GPU/CPU load to starve the capture pipeline (an active app while the machine is busy).
2. Stop and ffprobe the result:

   ffprobe -select_streams v:0 -count_frames \
     -show_entries stream=nb_read_frames,avg_frame_rate,duration <file>.mp4
   

3. Compare nb_read_frames / fps against the session's real wall-clock length (and the sibling -webcam.webm duration). A shortfall = dropped frames not reflected in the timeline; the screen plays ahead of the webcam.

OS

Linux

OS Version

Any Linux/Wayland with PipeWire ScreenCast (observed on GNOME/mutter + AMD).

Additional context

• Root cause is pre-existing in the original constant-frame-rate pacing design (capture.rs/encoder.rs), independent of the dmabuf/VAAPI zero-copy work ([Bug]: Whole-screen recordings freeze/judder on GNOME/Wayland (AMD) — capture forced onto shm, mutter under-delivers vs dmabuf #507/fix(capture): zero-copy dmabuf→VAAPI capture — fix whole-screen recording freeze on GNOME/AMD (#507) #508). It affects both the shm and dmabuf paths — any Linux recording that drops frames. The dmabuf path makes drops (and heartbeat starvation) more likely, which is how it surfaced.
• Proposed fix: stamp each encoded frame with its real capture wall-clock time (rescaled into time_base) and mux VFR, so file duration == real elapsed time even under drops; keep the clock heartbeat so a static screen still extends duration. The editor/compositor already seek/play by decoded PTS (same path as the already-VFR webcam), so VFR is tolerated. Add capture-side telemetry that warns when duration diverges from measured wall-clock beyond a small epsilon, so this can't regress silently.
• Full engineering write-up lives in technical-documentation/engineering/linux-capture-frame-drop-desync-bug.md.