Skip to content

Add host-backed hardware video decoding for the ARM64 guest #167

Description

@nocell

The README documents CPU-only video decoding. The ARM64 guest has accelerated VirGL graphics, but applications cannot use the Mac's media decoder through the current virtual devices.

Proposed implementation within the existing Apple Silicon / QEMU / HVF architecture:

  • A guest VA-API driver and a broker restricted to the video group send compressed HEVC, AV1 and VP9 to a supervised host helper.
  • VideoToolbox must confirm a hardware decoder before the driver exposes the session.
  • A private Mach capability carries IOSurfaces to QEMU for GPU copies into the guest's existing VirGL textures. CPU readback uses bounded per-client buffers through a dedicated PCI aperture.
  • Application-scoped, pinned FFmpeg libraries support mpv and Firefox without replacing Arch's FFmpeg libraries. Firefox keeps its RDD sandbox; Vivaldi provides the verified VP9 YouTube path.

A local M5 Pro installation completed the requested 2160p60 YouTube clip without playback stalls or dropped frames during a 7,529-frame measurement interval. HEVC Main10 also works in mpv and sandboxed Firefox. Initial FFmpeg validation matched all 120 decoded 4K Main10 frames and timestamps against software decoding; the accompanying PR will include reproducible checks and separate decode-only timings from CPU readback costs.

Opening this issue before the implementation PR as requested by CONTRIBUTING.md, so the new guest/host transport and application integration can be reviewed explicitly.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions