[chaofengw-nv]

Background

MiniMax-H3 on main generates video with audio from text. Reference-to-video keeps the people and places of a story consistent across shots: the caller passes reference images and the model renders them into the clip. The follow-up short-drama app PR depends on this path.

This work was first posted as several drafts. #1844 (two-window Ref2VA) failed Community CPU: test_family_production_does_not_hash_sources_plans_or_bundle_data flagged its content digests. #1847 (streamed plan inputs) passed Community CPU and internal premerge on its own. Both were closed so the work could land as one reviewable change. This PR rebases the complete stack onto current main, takes #1847 in its final reviewed form, and fixes what the repository's checks reject.

Exit Criteria

• An explicit build produces the original MiniMax-H3 image-reference bundle (one 124-frame window, or two windows for 226 frames) and the FastVideo/FastVideo-FastH3-Omni-8-Step-V1 OmniRef student bundle; python -m tensorrt_model_connect minimax_h3 build-omniref builds both.
• trtmc-server serves a reference-to-video bundle through /v1/videos with reference images and an optional continuation prompt, after registering the family's VSA kernel through the generic BYOK bridge.
• MiniMax-H3 plans stream into TensorRT from the bundle instead of being read whole into host memory.
• Family code has no content hashing and no environment side channels, and the Community CPU checks pass.
• Non-goals: automatic discovery (support.py is unchanged); any claim of trained-output quality, numerical equivalence of the complete pipeline with upstream, or real-time generation.

Implementation

• families/minimax_h3: the original image-reference path. This covers the Qwen vision and reference-aware text encoders, the VAE encoder and reference posterior, the reference AdaLN and DiT builders, and the native pipeline (factory, ImageReferenceDenoiser, two-window run_windows with an optional continuation_prompt). New runtime schemas minimax_h3.image_ref2va/v1 and minimax_h3.image_ref2va_continuation/v1.
• OmniRef student: reference_pdd.py (sampling contract and PDD output-head math), reference_pdd_builder.py, reference_pdd_dit_builder.py, reference_vsa.py and kernels/omniref_vsa.cu. The kernel is a TVM-FFI BYOK binding with a generic SM80+ route and optional sm100a/sm103a cubins. Schemas minimax_h3.omniref_pdd/v1 and minimax_h3.omniref_pdd_continuation/v1.
• Builds: reference_build.build builds the original bundle with single-file direct staging in core/builder BundleWriter. reference_continuation_build.build and reference_pdd_build.build rebuild only their changed plans and copy the others from an existing original bundle. build-omniref (cli.json, cli.py) runs the original build and then the student build.
• core/runtime: streamed plan loading, identical to the last head of perf(runtime): stream MiniMax-H3 engine inputs #1847 (its three commits).
• apps/server: --byok-library, --byok-function and --byok-name register a kernel before bundles load. /v1/videos accepts input_reference images and continuation_prompt for reference tasks and returns the clip with its audio.
• Cleanup in separate commits on top of the original work:
  • plan reuse checks the source's header and section layout, its runtime profile, its pinned checkpoint and BF16 provenance, its tokenizer bytes, and that the file stays unchanged while plans are copied; it no longer checks content digests;
  • the oracle and localization scripts under tests/, and the digest pins in the remaining tests, are removed;
  • the VSA route follows only the compiled library and the GPU (TRTMC_MINIMAX_H3_VSA_BACKEND and TRTMC_MINIMAX_H3_VSA_TRACE are removed);
  • clang-format 22.1.8 formatting;
  • an explicit optimization level for the PDD projection builder;
  • a README without run records.
• Compatibility: runtime.json no longer carries config_sha256, and the runtime ignores the key, so earlier bundles still load. Reusing plans from such a bundle is refused because its runtime.json differs; rebuild the source bundle first.

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Validation

Commands and Results

The CPU checks below ran in a fresh copy of this head with the GPU hidden (CUDA_VISIBLE_DEVICES=):

• python tools/legal_headers.py --check, python -m tools.model_ci validate, python tools/check_cyclomatic_complexity.py core/runtime --max-ccn 10 --top 20: passed.
• ruff check --config ruff.toml (0.16.4) and clang-format --dry-run --Werror (22.1.8) on every changed file; git diff --check: passed.
• The architecture-contract pytest set and the Community CPU unit pytest set: everything passed except the CI-tooling tests in tools/tests/test_community_ci.py and tools/tests/test_merge_ready_slack_alert.py, which need jq; it was not installed in that container. This PR does not touch those tools.
• pytest families/minimax_h3/tests -m "not gpu and not trt": 404 passed, 1 skipped.
• pytest apps/server/python/tests: 238 passed.
• cmake -DTRTMC_BUILD_TESTS=ON + ctest --label-exclude gpu: every minimax_h3 and core runtime test passed (18 family tests, plus bundle_format_v1, bundle_stream_reader, trt_plan_stream_reader_cpu, family_loader and server_video_worker). Six tests of other families failed at "create CUDA stream" because the GPU was hidden. sana_wm did not compile because that container's torch adds compute_50.
• GPU, one GB300: one OmniRef bundle, built before this cleanup (minimal, six_portrait, two windows), was served by trtmc-server built from the original stack and then from this head. Each server rendered the same reference-to-video request: four reference images, a prompt plus a continuation_prompt, and seed 42. The output is a 9.42 s clip of 226 frames at 768x1344 and 24 fps, with stereo 32-kHz audio. The two MP4 files are byte-identical (259,544,471 bytes); generation took 721 s and 711 s. This also shows that a bundle whose runtime.json still carries config_sha256 loads.

Hardware, Environment, and Revisions

Head based on main at 2c2256cc5. One NVIDIA GB300 in an aarch64 Ubuntu 24.04 container with TensorRT 11.2.0.113, CUDA 13.0 and torch 2.12.0+cu130. Checkpoints: MiniMaxAI/MiniMax-H3 at 48d93ede732756e404a3b1b2f3b3a9b5a22f6cfc, FastVideo/FastVideo-FastH3-Omni-8-Step-V1 at 53e0d26a1691aa473b6262d8196a8feb2c297df0.

Not Run / Remaining Gaps

• The GPU-marked tests in families/minimax_h3/tests (component and trained-model comparisons with upstream) were not run on this head.
• build-omniref was not rerun from scratch on this head: the two bundles need about 250 GB of free disk. The changed reuse code is covered by the CPU build tests above. The runtime check above used a bundle built before the cleanup.
• Internal premerge has not run yet.
• Known open gap: the trained 256x256 one-frame FP32 VAE encoder comparison fails its unchanged rtol=1.3e-6, atol=1e-5 on 54 of 12,288 moments. Thresholds were not changed.

Contributor Self-Review

• I have completed a self-review of this change.

Notes For Future Readers

• Suggested review order:
  1. core/runtime and core/builder (perf(runtime): stream MiniMax-H3 engine inputs #1847 and direct staging);
  2. apps/server;
  3. the original reference path in families/minimax_h3;
  4. continuation;
  5. the OmniRef student and kernels/omniref_vsa.cu;
  6. the cleanup commits at the end.
• Plan reuse trusts the source bundle the caller selects. It checks that bundle's metadata, provenance, tokenizer and file identity. It does not detect a same-size, same-mtime corruption of a reused plan, because family code may not hash bundle data.
• families/minimax_h3/README.md documents build commands, presets, capacity, bundle layout, the request contract and the known gaps.

Risk level

• Low
• Medium
• High

Large and experimental. The new MiniMax-H3 paths are opt-in and not selected by discovery. The shared changes are #1847's streamed plan loading, which passed internal premerge on its own; additive BundleWriter staging; and opt-in server options.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

_{Comment @coderabbitai help to get the list of available commands.}