Repository navigation
macOS: is software decode still the right pick for 10-bit and HEVC? (4K now measured — it is) #584
Description
Narrowed after measurement. This issue originally asked where the VideoToolbox/software decode crossover sits for 4K, 10-bit and HEVC. 4K is now answered and is no longer part of the question.
4K — measured, software wins
Decode only, 1200 frames, best of three passes, Mac mini M1, with the 1080p case kept as a control against the end-to-end figure in #583:
| source | software | VideoToolbox | ratio |
|---|---|---|---|
| 1080p (control) | 2586 fps | 212 fps | 12.19× |
| 4K | 849 fps | 71 fps | 11.88× |
The ratio barely moves with resolution: VideoToolbox's cost is a fixed per-frame latency, so four times the pixels does not rescue it. 4K is in fact where hardware decode hurts most — 71 fps is below real time for a 4K60 timeline.
Recorded in Decoder::open_with; no resolution threshold exists in the condition, deliberately.
What is still open
codec_id == H264 && format == YUV420P excludes both of these by construction, so they keep VideoToolbox untested:
- HEVC 8-bit. iPhone footage and some screen captures. The software HEVC decoder is materially more expensive than H.264's, so the ratio could genuinely invert here — this is the case most likely to have a real crossover.
- H.264 / HEVC 10-bit. Rarer, and it needs work beyond the predicate:
mac_frames::CpuFramesconverts to 8-bit NV12 (AV_PIX_FMT_NV12,kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange), so routing 10-bit through the software path would silently truncate to 8 bits. The predicate is what currently prevents that, which is worth knowing before anyone widens it.
How to answer them
No code change is needed to measure either: OPENSCREEN_MAC_DECODE=videotoolbox|software forces both sides on the same binary. The cheap decode-only method above was validated against the full export by its 1080p control, so it is enough to find the direction; confirm anything surprising with a full export A/B against a per-cycle ffmpeg floor.
Also worth a thought while doing it: the 4K figures were taken on an upscaled screen recording, which is easy to decode for its pixel count. A denser 4K bitstream is where the software decoder's cost actually scales and VideoToolbox's does not, and it is the fair attempt at overturning the result above.
Activity
[-]macOS: where is the crossover between VideoToolbox and software decode? (4K, 10-bit and HEVC unmeasured)[/-][+]macOS: is software decode still the right pick for 10-bit and HEVC? (4K now measured — it is)[/+]EtienneLescot commentedon Sep 15, 2026
HEVC 8-bit is measured — it keeps VideoToolbox, and the method witness only half-reproduced.
Recorded in
pipeline_macos.rswith the harness committed undercrates/compositor/examples/(#673, merged as2f642f22): productionDecoder(OPENSCREEN_MAC_DECODE, NV12/CVPixelBuffer present included), 1200 frames, best of three, Parsec disconnected, same Mac mini M1 8 GiB / macOS 26.5. HEVC sources arehevc_videotoolboxMainyuv420p(no libx265 on this box — that is also what iPhones emit) encoded from the densest real capture available (11.5 Mbps); the 4K one is an upscaled render:Software wins at both sizes but by a fraction of the H.264 margin, VideoToolbox holds real time everywhere (303 fps on a 1080p60 timeline, 102 on 4K60), and the export walk is encoder-bound anyway. Per the #583 decision rule (tight → no change): HEVC keeps VideoToolbox; the predicate is untouched.
The witness only half-reproduced, and that matters for the ×12 on record. The software side of the 1080p H.264 control reproduces exactly — 2721 fps in pure decode vs the 2586 on record — but only once Parsec was disconnected (with it, and the swap pressure it brings on an 8 GiB machine, the same harness measured 1192 fps). The 212 fps VideoToolbox figure reproduces under no harness variant (294–330 across production decoder, raw decode, and the ffmpeg CLI), so the historical ×12.2 should be read as an upper bound from a specific day, not a stable ratio.
10-bit remains unmeasured — tracked in #676 (
CpuFramesrenders NV12 only, so routing it through software would silently truncate to 8 bits).