conformance: visual sweep tooling, NIST findings, and the MovieDecoder ADR#34
Merged
Conversation
This was referenced Jun 28, 2026
Closed
…ST) set A diagnostic that runs the OpenGL PoC over the full 763-scene X3D Conformance (NIST) example set to surface parser/extractor gaps the unit conformance suite doesn't cover. - scripts/fetch_conformance_nist.sh: download the official web3d.org bundle and extract the .x3d (+ --textures) into .x3d-corpus/x3d-code/ (gitignored). - scripts/poc_conformance_sweep.py: per-scene headless parse+extract probe (OK / OK_EMPTY / PARSE_FAIL / CRASH / TIMEOUT / ERROR), opt-in --gl pass that drives the real OpenGL pipeline via poc --screenshot under xvfb + mesa (llvmpipe). Bucketed by section; failures collapsed by normalized signature. - mise tasks: conformance-nist-fetch, poc-conformance-sweep. First run: 737/763 (96.6%) extract OK; GL pass 15/15 on a Geometry sample. The 26 zero-item scenes split into legit-empty (bindable/script/interpolator-only, EXTERNPROTO decls, newScene) and real gaps — notably <Extrusion/> with default crossSection/spine extracts no geometry (reproduces in cpuraster too, so it is an SDK extraction gap, not poc-specific).
…Decoder ADR Adds the vision-model visual conformance sweep and records what it surfaced: * scripts/visual_conformance_sweep.py — renders each ConformanceNist scene through the PoC (--front, xvfb+mesa), fetches the matching NIST *-front.jpg reference, and scores "same subject?" with a local OpenAI-compatible vision model. Resumable JSONL; --no-front for bindable-Viewpoint scenes whose reference is the bound view, not a canonical front. Always exits 0 (diagnostic). * findings.yaml — VIS-EXTRUSION-EMPTY (Extrusion default/simple -> 0 geometry), VIS-ELEV-SPACING-EMPTY (ElevationGrid x/zSpacing -> 0), VIS-SPHERE-NOUV (Sphere ImageTexture not applied — no texcoords), VIS-MOVIE-DECODE (deferred -> ADR-0041); each item count verified by poc --headless. SEAM-TEXT-METRICS confirmed (length compression renders as bars now that a real atlas is wired). * ADR-0041 — MovieDecoder seam: ship royalty-free + permissive codecs (pl_mpeg MPEG-1, then Theora, WebM/VP8-9), leave patent-encumbered / copyleft (H.264/265, FFmpeg, GStreamer) to implementors behind the same frozen seam. Added to the nav + seam-status matrix.
… unlit textures
Three conformance gaps surfaced by the visual NIST sweep, each root-caused to a
distinct layer:
- Extrusion with unauthored crossSection/spine extracted no geometry
(default_extrusion.x3d, test_simpleorient.x3d). Root cause was the GENERATOR,
not the extractor: the MF-struct default emitter collapsed any multi-element
array default to its first element, so the Extrusion binding defaulted
crossSection to {1,1} and spine to {0,0,0} instead of the 5-point unit square
and the 2-point segment. Fixed by chunking the scalar list into element-sized
groups (emit/defaults.py) + regenerating Extrusion.hpp; an authored 1-point
spine still culls (the "< 2 spine points => empty" contract holds).
- ElevationGrid test_xSpacing/test_zSpacing authored only one dimension and left
the other at its 0 default, so the xd>=2 && zd>=2 guard zeroed the output.
Infer the missing dimension from height.size() when it divides evenly by the
authored dimension (conservative, documented leniency matching the NIST refs).
- Sphere with an ImageTexture and NO Material rendered flat white. The extractor
was already spec-correct (no Material => Unlit with the image on the Emissive
slot, §12.2.5); the PoC's unlit shader simply had no texcoord/sampler. Added
aTexCoord + sampler2D to unlit.vert/frag and bound the texture in the unlit
draw path.
Findings VIS-EXTRUSION-EMPTY, VIS-ELEV-SPACING-EMPTY, VIS-SPHERE-NOUV flipped to
fixed; conformance view regenerated. All 48 C++ tests, golden, conformance-gate,
and both example consumers pass.
delta9000
force-pushed
the
chore/poc-conformance-sweep
branch
from
422a1a1 to
6b65dc9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Web3D/NIST visual conformance sweep, what it found, and the fixes for the three concrete gaps it surfaced. Pairs with #33 (the sweep drives the PoC's
--front).Tooling
scripts/poc_conformance_sweep.py— headless parse+extract probe over the NIST set (bucketed, resumable).scripts/visual_conformance_sweep.py— renders each scene through the PoC (--front, xvfb+mesa), fetches the matching NIST*-front.jpg, and scores "same subject?" with a local OpenAI-compatible vision model. Resumable JSONL;--no-frontfor bindable-Viewpoint scenes whose reference is the bound view. Always exits 0 (diagnostic, not a gate).scripts/fetch_conformance_nist.sh+ mise tasks.Fixes (each root-caused to a distinct layer)
crossSection→{1,1}/spine→{0,0,0}instead of the unit square / 2-point segment. Fixed inemit/defaults.py(chunk by element arity) + regeneratedExtrusion.hpp; authored 1-point spine still culls.test_{x,z}Spacingauthor one dimension and omit the other; infer the missing dimension fromheight.size()when it divides evenly (conservative, documented leniency matching the NIST refs).Material⇒ Unlit with the image on the Emissive slot, §12.2.5); the PoC's unlit shader had no texcoord/sampler. AddedaTexCoord+sampler2Dtounlit.vert/fragand bound the texture in the unlit draw path.Other findings
SEAM-TEXT-METRICSconfirmed.ADR-0041 — MovieDecoder seam
Ship royalty-free + permissive codecs (pl_mpeg/MPEG-1, then Theora, WebM/VP8-9); leave patent-encumbered / copyleft (H.264/265, FFmpeg, GStreamer) to implementors behind the same frozen seam. Added to nav + seam-status matrix.
Verified: all 48 C++ tests,
golden,conformance-gate, and both example consumers (validate-examples) pass; strictdocs-buildclean.