[dorodo0721]

Required human contributor answers

The existing human request and contributor claim define this documentation scope:
issue #124 and
the contributor's claim.

1. What outcome are you trying to achieve?

Issue #124 asks: “Please add a documentation section linking the available validation scripts, benchmark commands, execution logs and raw results for Ouro and Nanbeige, together with the corresponding code commits, model revisions and runtime configurations.” This PR adds that index for Ouro-1.4B and Nanbeige4.2-3B.

2. What is the highest-leverage variable for that outcome?

The requester identifies the evidence-discovery gap: “I found the performance summaries in #30/#31 and the integration tests in #77/#119, but could not locate a complete public validation package for these models.” The contributor's claim focuses on linking existing material and identifying missing or unpublished evidence.

3. How will this work move that variable rather than create busywork?

The contributor's stated approach is a “validation and reproducibility index,” linking reference/native checks, accuracy tools, benchmark commands and public artifacts, recording versions/configurations/scope, and “clearly identify[ing] missing or unpublished evidence.” The index provides those entry points and separates source code, historical reports, verified downloadable artifacts and unavailable raw files.

4. Who is the customer or user, and what do they actually need?

The requester of #124 needs the four evidence categories listed in the issue: HF/reference-vs-native scripts/results; accuracy scripts and per-example outputs; benchmarks, logs and raw results; and matching code/model/configuration identities. They also ask for “a minimal reproducible validation recipe.” The index addresses that stated need without treating missing artifacts as available evidence.

PR necessity and bugfix evidence

A PR is needed to add the documentation section requested in #124 to the maintained repository. The contributor's linked claim explicitly limits the work to documentation and an audit of existing material, with no new GPU experiments or accuracy/performance claims.

The existing accuracy example also omitted two required arguments. Running its original prepare --model /path/to/ouro --output ... command locally exits 2 before any preparation or inference:

gsm8k.py prepare: error: the following arguments are required: --model-repo, --model-revision

The example now supplies both. This is a documentation correction; inference and evaluation code are unchanged.

Purpose

Closes #124.

• Add docs/validation.md, linked from the documentation index, model-support table and accuracy guide.
• Index tiny-model oracles, official-code and real-checkpoint checks, historical HF/native GSM8K results, PD reports, benchmark recipes and source/model/configuration identities.
• Record the unavailable GSM8K raw-evidence release (release-tag API returned 404), zero upstream Actions artifacts at the audit date, missing Nanbeige raw output/log packages and inconsistencies in the historical parity report.
• Link PR [Kernel] Triton packed paged prefill and split-KV decode attention #128's public profiling traces; both were downloaded and matched the published SHA256 digests and gzip integrity checks. They remain diagnostic profiles, not unprofiled benchmark samples or accuracy outputs.
• Distinguish current commands from historical [MODEL][PD] Integrate Nanbeige4.2 with PD serving #77 and unmerged draft Add a real-checkpoint P->D agreement test for the RFC #111 handoff contract #119 commands. Label all checkpoint/GPU recipes unexecuted by this audit.

No runtime, model, measurement or test implementation changes. No weights were downloaded and no checkpoint inference or GPU/NPU experiments were run.

Test Plan

vllm-rlt Version / Commit: runtime/source audit at 989921ac1d3897c5c43e27ec8cddbc91fa953194; documentation commit 6b03a295f8ec1ab5cf4b7c5a2f479d66f01aa4ec.

• pre-commit run --all-files --verbose: Ruff lint, Ruff format and the CPU regression suite.
• Check relative paths/anchors, Bash snippet syntax, required GSM8K arguments against the CLI/help, and git diff --check.
• Run preemption --plan-only; verify the two existing public profiling downloads without running inference.
• Apply the repository performance/refactoring review proportionally: performance measurement is inapplicable to a documentation-only diff; review evidence boundaries and reproduction contracts (R24/R25), with no applicable runtime refactoring required.

Test Result

• Lint and format passed. CPU suite: 911 passed, 584 skipped, on macOS ARM64, Python 3.12.14, torch 2.13.0, Transformers 4.55.0, Ruff 0.13.2. Hub access was offline; GPU tests were disabled.
• Skips: 549 GPU cases, 28 requiring optional lm_eval, and seven requiring official tokenizer/code/weight assets. These do not establish real-checkpoint or device qualification.
• The initial sandboxed suite had 21 localhost socket/shared-memory permission failures (890 passed, 584 skipped). The same suite passed after permitting those local operations; both logs are retained locally.
• Documentation links, shell syntax, CLI requirements and whitespace checks passed. Preemption plan-only produced eight cases without inference.
• Both public PR [Kernel] Triton packed paged prefill and split-KV decode attention #128 traces matched GitHub's SHA256 asset digests and decompressed successfully. The historical GSM8K archive could not be retrieved or checksum-verified.
• Historical checkpoint correctness, accuracy, PD and performance experiments were not rerun. Missing evidence remains explicitly documented.