CUDA: B1 indexer misses WMMA and 1M HCA falls off the 7936-row fast path

[wangshang23]

Summary

On current main (84cc882), DeepSeek-V4-Flash CUDA B1 decode has two
long-context cliffs on DGX Spark / GB10:

1. The ratio-4 CSA indexer always takes indexer_score_one_direct_kernel for
   n_tokens == 1, even when the existing WMMA128 scorer is faster.
2. At 1M context, ratio-128 HCA has 8192 compressed rows. The shared score cap is
   8192 total rows but reserves 256 raw rows, so only 7936 compressed rows fit.
   The last 256 compressed rows force all 20 HCA layers onto
   attention_decode_mixed_heads8_online_kernel instead of the existing exact
   score-split path.

Together these account for most of the 512K -> 1M decode collapse.

Reproduction environment

• DS4: 84cc882
• Hardware: DGX Spark, NVIDIA GB10 (sm_121)
• Model: DeepSeek-V4-Flash IQ2XXS / Q2_K / Q8 projections and output
• Model residency reported by DS4: 80.76 GiB
• DS4_CUDA_Q8_F16_CACHE_MB=0
• 1M session plan: 13.46 GiB KV + 0.06 GiB buffers + 80.76 GiB model =
  94.28 GiB

The benchmark allocates and touches the real CUDA F32-slot KV/indexer caches,
places the session at context - 1, and measures a complete target-only decode
token. It does not include a synthetic 1M prefill in TPOT. Legacy and optimized
paths alternate in the same process/session; each has 3 warmups and 30 samples.

End-to-end A/B

Context	Current TPOT	Patched TPOT	Current tok/s	Patched tok/s	Speedup
128K	82.182 ms	80.208 ms	12.168	12.468	1.025x
256K	93.963 ms	88.571 ms	10.642	11.290	1.061x
512K	115.649 ms	104.450 ms	8.647	9.574	1.107x
1M	228.729 ms	143.250 ms	4.372	6.981	1.597x

Kernel evidence

Microbenchmarks:

Context	CSA direct	CSA WMMA	HCA current	HCA score-split
128K	0.574 ms	0.438 ms	0.125 ms	0.126 ms
256K	1.050 ms	0.792 ms	0.230 ms	0.232 ms
512K	1.919 ms	1.359 ms	0.428 ms	0.428 ms
1M	3.713 ms	2.502 ms	2.925 ms	0.849 ms

Nsight Systems capture of one complete 1M decode (capture excludes startup,
cache initialization, and warmup):

Current

• attention_decode_mixed_heads8_online_kernel: 97.286 ms / 20 instances
• indexer_score_one_direct_kernel: 78.036 ms / 21 instances
• Captured wall TPOT: 261.230 ms

Patched

• indexer_scores_wmma128_kernel: 48.565 ms / 21 instances
• attention_decode_score_split_finalize_kernel: 16.158 ms / 22 instances
• attention_decode_score_split_scores_tile512_kernel: 9.242 ms / 22 instances
• Captured wall TPOT: 148.228 ms

The profiled wall times include Nsight overhead and are not the primary
throughput numbers, but the kernel attribution is unambiguous.

Proposed fix

• Increase DS4_CUDA_ATTENTION_SCORE_CAP from 8192 to 8448. This is the exact
  8192 compressed + 256 raw capacity needed by the 1M HCA shape and keeps the
  already-existing exact score-split implementation active.
• For non-quality B1 indexer calls, use the existing WMMA path once
  n_comp >= 8192; keep the direct scorer for short contexts, quality mode,
  and DS4_CUDA_NO_INDEXER_WMMA.

Correctness checks

• QAT-format B1 score/top-k at n_comp=8192: max score delta
  6.10352e-05, top-512 index differences 0/512.
• Isolated HCA output at 8192 compressed rows: max delta 0 on the regression
  vector.
• Full synthetic-frontier logits at 128K/256K/512K: bit-identical.
• Full synthetic-frontier logits at 1M: max abs 0.0469046, RMSE 0.00876944,
  same argmax. This is the expected reduction-order change from the long online
  fallback to DS4's existing exact score-split path.
• Real 30,474-token story fact-recall regression: passes.
• make cuda-regression CUDA_ARCH=sm_121: passes.

Existing issue/PR audit

• Add HISA hierarchical indexer for long-context decode #258 targets the same long-context indexer cost with approximate HISA
  pruning. It is based on an older revision, currently conflicts with main,
  and reports +1.9% at 256K. The change proposed here accelerates the current
  flat full-top-512 scorer and is complementary to a future HISA path.
• Metal pre-M5: decode collapses ~10x once context crosses n_indexer_top_k (GPU 95% idle) #586 is a Metal/SSD shared-event serialization problem.
• Metal: FP8-packed compressed-KV cache + long-context memory optimizations #416 and Metal: FP8-packed compressed-KV cache + long-context memory optimizations #418 are Metal packed-KV / prefill-memory work.
• [Discussion] Tile indexed-attention scoring independently of the global prefill chunk #644 is about token-tiling prefill scratch, not B1 decode.
• Add guarded long-context benchmark tooling #588 was benchmark tooling only and closed without merge.

I did not find an existing issue or PR for the CUDA HCA 7936-row capacity
fallback.