feat(movie): MovieDecoder seam + 2 backends (pl_mpeg + Theora) — proven generic (ADR-0041)#40
Merged
Merged
Conversation
MovieTexture frames are now decoded and played onto geometry. Previously
TextureRef::Source::Movie was surfaced but had no consumer decode path, so every
movie-textured conformance scene rendered blank.
Seam (core stays codec-free):
- runtime/extract/MovieDecoder.hpp — frozen consumer callback
(url, mediaTimeSeconds) -> FrameResult{Ready|Pending|Failed, VideoFrame}, the
same shape/threading contract as TextureResolver/FontMetrics. Bottom-left RGBA8.
makeNullMovieDecoder() is the always-Failed default.
Backend A (flag-gated isolated target x3d_plmpeg, the x3d_stbtt pattern):
- runtime/io/plmpeg/ — pl_mpeg (MIT, MPEG-1: royalty-free, patents expired),
PL_MPEG_IMPLEMENTATION confined to one TU; decoder-free public header. The
factory takes an AssetResolver to fetch bytes and caches one decoder per URL.
- The NIST .mpg corpus is RAW ELEMENTARY video (0x000001B3 sequence header, no
MPEG-PS system layer) which pl_mpeg's plm_t demuxer reports as zero streams, so
the backend detects the stream shape and drives the low-level plm_video_t
directly (sequential decode + rewind-on-backward); plm_t + seek for program
streams. Alpha is forced opaque (plm_frame_to_rgba leaves the A byte untouched).
PoC wiring:
- resolveTexRef() routes Source::Movie through the seam, (re)uploading the decoded
frame each tick (create-once + glTexSubImage2D). Media time = the render clock;
--screenshot captures ~t=0 (the clip's first frame). Covers the lit/unlit/pbr
base-color + emissive slots.
Tests/docs:
- x3d_movie_tests: per-backend semantics-contract test (Ready dims + tight RGBA,
EOF-holds-last, backward-seek rewind, Pending/Failed parity) against a committed
686-byte self-made MPEG-1 fixture. ADR-0041 -> Accepted; seam-status row ->
NOT-YET-PROVEN (contract test green, awaiting a 2nd backend); VIS-MOVIE-DECODE ->
fixed; NOTICE adds pl_mpeg (MIT, flag-gated).
Verified: build-ci (111), validate-examples (both consumers), golden,
conformance-gate, strict docs-build, and a render of Appearance/movietexture.x3d
(VTS card plays on box/sphere/cone/cylinder).
Fixes #38
…-0041) The MovieDecoder seam now has TWO independent backends behind the frozen runtime/extract/MovieDecoder.hpp, so it is proven generic (one interface, two format-partitioned codecs). Backend B — libtheora/libogg (runtime/io/theora/, flag-gated x3d_theora): - Ogg/Theora decode: feed bytes into an ogg_sync, demux pages by serial number to find the Theora logical stream, parse its 3 setup headers, decode data packets to YCbCr, convert to bottom-left opaque RGBA8 (BT.601, matching the pl_mpeg backend; handles 4:2:0/4:2:2/4:4:4 subsampling + the pic crop region). Sequential decode to media time; backward seek rebuilds the session from the retained bytes. - libtheora/libogg are NOT single-header — found via pkg-config, linked PRIVATE to the one TU (the FreeType backend-B pattern); the public header is codec-free. Theora is a Xiph BSD, royalty-free codec, so it joins the default-shippable tier. Genericity proof (the point of a 2nd backend): - movie_decoder_tests refactored to a shared runContract() run against EACH backend's own reference clip (pl_mpeg redsquare.m1v, theora redsquare.ogv): same observable seam behavior — Ready dims + tight opaque RGBA, EOF-holds-last, backward-seek rewind, Pending/Failed/garbage parity. Two backends passing the identical contract is what flips the seam to STABLE. Codecs partition by format (no shared input), so the proof is contract-parity, not bit-swap (ADR-0041). Also fixes a pl_mpeg raw-path bug surfaced by the stronger contract: a backward seek after EOF failed because plm_video_rewind does not recover post-EOF. The raw path now rebuilds the decoder from retained bytes (each decoder owns its own byte copy — no shared-memory aliasing), the same robust rewind the Theora path uses. ADR-0041 -> seam PROVEN GENERIC; seam-status MovieDecoder row -> STABLE; NOTICE adds libtheora/libogg (BSD, flag-gated). Verified: x3d_movie_tests (both backends, 37 assertions) with -DX3D_CPP_BUILD_THEORA=ON, build-ci (111), validate-examples, golden, conformance-gate, strict docs-build, and a visual decode of a Theora color-bars clip (crop/subsample/conversion correct).
delta9000
changed the title
Churn the seam to enforce pluggability, not just assert it. runContract now runs the IDENTICAL semantics-contract against FOUR plugs: - pl_mpeg / MPEG-1 (Backend A) - libtheora / Ogg-Theora (Backend B) - a FROM-SCRATCH std-only backend written against ONLY the public header — no codec, no external dep. Passing the same contract as the real codecs proves the interface carries zero codec coupling: any downstream can implement it. - the multi-format COMPOSER, which is itself a conforming MovieDecoder. New shipped artifact — runtime/extract/MultiFormatMovieDecoder.hpp (header-only, std-only, the MultiFormatTextureResolver analog): sniffMovieFormat() classifies the container by magic bytes (Ogg "OggS", EBML/WebM, MPEG-1 start codes) and makeMultiFormatMovieDecoder() dispatches per URL to the registered backend (sniffed once, cached). Backends COMPOSE behind one seam; a downstream registers WebM / H.264 / OS-native without touching the core. Follows the decode-seam RULE (ADR-0024 §7): route by a cheap key up front, never by Failed. Pluggability stress added: - composer routes each container to the right backend by magic (m1v -> pl_mpeg, ogv -> theora); unreadable -> Failed, not a routing crash. - each backend rejects a FOREIGN container as Failed (pl_mpeg given Ogg, theora given MPEG) — so the magic route and any consumer can trust Failed. - truncated input (1/4/16/64/half bytes) yields a defined Failed/Ready, never a crash/UB. x3d_movie_tests: 8 cases / 85 assertions (both real backends built). ADR-0041 documents the pluggability enforcement. build-ci (111, incl. per-header isolation of the new composer header), golden, conformance-gate, docs-build all green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the MovieDecoder seam with TWO independent backends per ADR-0041 — so the seam is proven generic, not just wired. MovieTexture scenes that rendered blank now play video on geometry.
Seam (core stays codec-free)
runtime/extract/MovieDecoder.hpp— frozen(url, mediaTime) → FrameResult{Ready|Pending|Failed, VideoFrame}, same shape/threading as TextureResolver/FontMetrics; bottom-left opaque RGBA8.makeNullMovieDecoder()is the always-Failed default.Backend A —
x3d_plmpeg(pl_mpeg, MPEG-1)Single-header MIT codec, patents expired. Wired into the PoC renderer (the whole NIST corpus is MPEG-1). The corpus
.mpgare raw elementary streams (0x000001B3, no MPEG-PS system layer) which pl_mpeg'splm_tdemuxer can't read — so the backend detects the stream shape and drives the low-levelplm_video_tdirectly;plm_t+seek for program streams.Backend B —
x3d_theora(libtheora/libogg, Ogg-Theora)Xiph BSD, royalty-free. System libs via pkg-config, PRIVATE-linked to one TU (the FreeType backend-B pattern); codec-free public header. Hand-written Ogg demux + 3-header setup + YCbCr→RGB (BT.601 matching Backend A; 4:2:0/4:2:2/4:4:4 + pic-crop) + sequential decode-to-time with rebuild-on-rewind.
Genericity proof
x3d_movie_testsruns ONE sharedrunContract()against each backend's own reference clip (codecs partition by format — no shared input, so the proof is contract-parity, not bit-swap): Ready dims + tight opaque RGBA, EOF-holds-last, backward-seek rewind, Pending/Failed/garbage parity. Both pass (37 assertions) → seam-status MovieDecoder row → STABLE.PoC wiring
resolveTexRef()routesSource::Moviethrough the seam (Backend A), re-uploading the decoded frame each tick across the lit/unlit/pbr base-color + emissive slots.--screenshotcaptures ~t=0.Notes
VIS-MOVIE-DECODE→ fixed; NOTICE adds pl_mpeg (MIT) + libtheora/libogg (BSD), both flag-gated.Verified:
x3d_movie_tests(both backends,-DX3D_CPP_BUILD_THEORA=ON),build-ci(111),validate-examples,golden,conformance-gate, strictdocs-build, plus visual decodes (VTS card on box/sphere/cone/cylinder; Theora color-bars conversion).Fixes #38
Pluggability churn (enforced, not asserted)
To be confident any downstream plug drops in,
x3d_movie_testsruns the identicalrunContractagainst four plugs and adds robustness stress (8 cases / 85 assertions):MultiFormatMovieDecoder(new shipped header, theMultiFormatTextureResolveranalog):sniffMovieFormat()classifies by magic bytes (Ogg/EBML/MPEG start codes) and dispatches per-URL to the registered backend — backends compose behind one seam; a downstream registers WebM/H.264/OS-native without touching the core. The composer is itself a conformingMovieDecoder(runs the full contract).Failed, no crash) so the magic route and any consumer can trustFailed.Failed/Ready, never UB.