Repository navigation
Prep #24: Add reproducible Blackwell experiment run pack #37
Copy link
Copy link
Open
Labels
consistencyRollout-training consistencyRollout-training consistencyenhancementNew feature or requestNew feature or requestrl-kernelRL-Kernel integrationRL-Kernel integrationtelemetryStructured telemetry and observabilityStructured telemetry and observability
Description
Background
Prerequisite for #24, assuming the Phase 1-7 roadmap foundation in #6 is already available.
#24 should run Blackwell/B-series post-training baseline and candidate A/B experiments. Before those runs, vime needs a small, reproducible experiment pack that fixes workload, hardware target, launch mode, artifact locations, and mismatch-knob order while reusing the existing Phase 1-7 audit, provenance, profiling, CI, and report mechanisms.
This issue owns #24-specific run preparation only. It does not create new framework telemetry, provenance, operator-comparison, distributed-contract, CI, or release-report machinery.
Scope
- Add a checked-in [Experiment] Build Blackwell train-inference mismatch baseline for VIME post-training #24 experiment/runbook entry for P0 smoke, P1 stable baseline, and P2 production-like split.
- Provide config templates or launch snippets for the recommended Qwen3-30B-A3B Megatron + vLLM setup.
- Specify required run inputs: checkpoint paths, data path, model family, Docker/image ID, commit hash, hardware type, node/GPU count, colocate vs non-colocate mode, and RL-Kernel candidate toggle when present.
- Specify where Phase 1-7 artifacts should be collected for each run: config snapshot, logs, wandb links, audit/report outputs, profiler outputs, runtime provenance, and summary tables.
- Add a dry-run/config-validation path where possible so command generation can be tested without owning B-card hardware.
- Document the exact [Experiment] Build Blackwell train-inference mismatch baseline for VIME post-training #24 mismatch knob order: near-aligned sanity, FP8 rollout, sampling filters, combined mismatch, colocate/non-colocate, then candidate operator path.
Non-Goals
- Do not implement B-card CUDA kernels here.
- Do not add new audit metrics, result-cube, provenance, profiling, CI, or report schema; reuse Phase 1-7 outputs.
- Do not duplicate Phase 3 audit/replay/result-cube implementation from Phase 3: Build consistency audit harness and replay diagnostics #29.
- Do not duplicate Phase 4/5 operator comparison and distributed contract harnesses from Phase 4: Add single-card operator comparisons and drift attribution #30/Phase 5: Add distributed operator contracts and A0-A5 profiles #31.
- Do not duplicate Phase 6/7 profiling, benchmark, CI, or release rules from Phase 6: Add profiling-guided fast-path expansion telemetry #35/Phase 7: Add RL-Kernel CI, benchmark matrix, and release rules #36.
- Do not claim a valid baseline until [Experiment] Build Blackwell train-inference mismatch baseline for VIME post-training #24 actually runs on target hardware.
Acceptance Criteria
- A user can identify the exact command/config set for P0/P1/P2 [Experiment] Build Blackwell train-inference mismatch baseline for VIME post-training #24 runs.
- The run pack records where existing Phase 1-7 artifacts are expected, without introducing a second report format.
- The dry-run/config-validation path can run on a developer machine or clearly reports which hardware-backed fields are unavailable.
- The issue explicitly separates setup artifacts from final Blackwell/B-card experimental results.
Hardware Notes
- Most of this issue can be developed on CPU or a single-card machine.
- Final validation of P0/P1/P2 configs requires B200/GB200 hardware because the target workload and FP8 rollout path are hardware-specific.
Dependencies
- Parent experiment: [Experiment] Build Blackwell train-inference mismatch baseline for VIME post-training #24.
- Roadmap foundation: RL-Kernel integration roadmap for vime #6 Phase 1-7.
- Phase 6/7 reporting/profiling outputs should be consumed rather than reimplemented: Phase 6: Add profiling-guided fast-path expansion telemetry #35, Phase 7: Add RL-Kernel CI, benchmark matrix, and release rules #36.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
consistencyRollout-training consistencyRollout-training consistencyenhancementNew feature or requestNew feature or requestrl-kernelRL-Kernel integrationRL-Kernel integrationtelemetryStructured telemetry and observabilityStructured telemetry and observability
Activity
[-]Prep #24: Add reproducible Blackwell baseline experiment pack[/-][+]Prep #24: Add reproducible Blackwell experiment run pack[/+]