Skip to content

cuda: sanitize non-finite router scores - #740

Open
exochard wants to merge 1 commit into
antirez:mainfrom
exochard:fix/router-nonfinite-selection
Open

cuda: sanitize non-finite router scores#740
exochard wants to merge 1 commit into
antirez:mainfrom
exochard:fix/router-nonfinite-selection

Conversation

@exochard

@exochard exochard commented Aug 7, 2026

Copy link
Copy Markdown

Problem

The CUDA router computes probabilities and scores used by fixed-size top-k selection. A NaN or Inf in a router logit or bias can poison comparisons and leave an invalid expert selection, which later becomes an unsafe expert-tensor index.

Change

  • Centralize router probability conversion and clamp non-finite results to zero.
  • Treat non-finite router bias entries as zero during score comparison.
  • Apply the same guards to scalar, parallel, and warp top-k kernels.

This keeps selection deterministic and prevents malformed numeric input from becoming an invalid routed-expert address. It does not change finite-input behavior.

Validation

Builds completed with no compiler warnings:

make -B ds4 CUDA_HOME=/opt/cuda CUDA_ARCH=sm_86
make -B ds4 CUDA_HOME=/opt/cuda CUDA_ARCH=sm_89

git diff --check is clean. Target hardware is an RTX 4060 Ti 16 GiB (sm_89); the guard is independent of model quantization and applies to the CUDA DeepSeek router.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant