[git-jxj]

Fixes #1697.

This fixes four scoring correctness issues: model text can expand judge-template placeholders, arena confidence intervals depend on unrelated global RNG activity, metric exceptions count as successful zero scores, and inclusion accuracy uses different matching rules for strings and lists.

• Render judge placeholders in one pass over the original template. Inserted questions, predictions, and references remain literal, including {gold}, JSON, and backslashes.
• Give general_arena bootstrap its own RNG, seeded from TaskConfig.seed. Fixed seeds reproduce confidence intervals regardless of global RNG consumption; explicit None retains nondeterministic sampling without modifying global RNG state.
• Omit failed metric keys and mark fully failed samples EXCLUDED or partially failed samples DEGRADED. Preserve valid metrics and their denominators through judge fallback and LLM recall, including a missing primary metric and equivalent metric names. When the judge recovers a completely failed rule, align equivalent metric names with the rule declaration so acc and accuracy do not split the cross-sample aggregation denominator.
• Match one complete, normalized acceptable answer for allow_inclusion. Lists remain alternatives (including MMLU-Redux single-choice answers); string references become a single alternative, so York no longer matches New York. Case and surrounding whitespace follow the existing normalization rules, and blank predictions do not receive credit.

The bootstrap issue is global-RNG coupling, not an absence of seeding in the standard entry point: run_task already defaults to seed 42. This does not change arena_hard, whose current aggregation does not call bootstrap. The AIME/AIR-Bench parsing defects in #1578 were already fixed by #1588 and subsequently migrated to JSON contracts in #1601; those adapters are unchanged here.

Compatibility

general_arena, mmlu_redux, and trivia_qa move to evaluation version v1.1. A shared scoring version is also included in native cache fingerprints so old scores cannot bypass the common judge and metric fixes. This invalidates existing native score-cache identities, including benchmarks outside these three: use rerun_review=True to reuse predictions and recompute reviews. Legacy fingerprints remain calculated with their original algorithm for provenance tracking.

Validation

Four of the 10 new cross-sample aggregation cases fail against e0400d03: equivalent judge and rule names produce duplicate accuracy aggregates with one sample each instead of one aggregate with two samples.

The results below are from the current revision unless marked as earlier. Regression tests were run against the old behavior and rerun after each fix. All judge responses use offline doubles.

• Python 3.13: 281 passed across judge contracts/execution, adapter scoring, evaluation versioning, metrics/aggregators, and arena bootstrap:

  python -m pytest tests/api/judge tests/api/test_metric_failure_status.py tests/api/test_default_data_adapter.py tests/api/test_scoring_policy.py tests/api/test_evaluation_versioning.py tests/metrics tests/benchmark/test_arena_bootstrap.py -q --tb=short

• Earlier Python 3.12 focused regressions: 64 passed, 3 skipped; the three real-ELO adapter cases were skipped because scikit-learn is unavailable in that environment, and passed under Python 3.13.
• python -m pytest tests/cli/test_all.py::TestRun::test_ci_lite -v -s -p no:warnings: 1 passed, using the mock model.
• make lint: passed.

[Yunnglin]

LGTM. Renaming the parametrized argument avoids the pytest-benchmark fixture collision; the full targeted regression suite (89 passed), pre-commit, and CI smoke test all pass locally.