Skip to content

ggml : dedup expert cache for MoE CPU offload - #21620

Closed
e1n00r wants to merge 1 commit into
ggml-org:masterfrom
e1n00r:feat/expert-cache-dedup
Closed

e1n00r wants to merge 1 commit into
ggml-org:masterfrom
e1n00r:feat/expert-cache-dedup

Conversation

@e1n00r

@e1n00r e1n00r commented Apr 8, 2026 •

Copy link
Copy Markdown

PR withdrawn by author.

@ggml-gh-bot

ggml-gh-bot Bot commented Apr 8, 2026

Copy link
Copy Markdown

Hi @e1n00r, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

  • AI-generated content: This project does not accept PRs, descriptions or commit messages that are fully or predominantly AI-generated. If you have used AI to assist you in writing code, please make sure to disclose that explicitly.

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

@am17an

am17an commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

@ggerganov we probably need to ban this user

@ggerganov

Copy link
Copy Markdown
Member

Put them on 1 week timeout

@ggerganov ggerganov closed this Apr 8, 2026
@e1n00r
e1n00r deleted the feat/expert-cache-dedup branch April 8, 2026 12:23
simonyos added a commit to simonyos/llama.cpp that referenced this pull request Oct 16, 2026
- credit ggml-org#29949: its LRU_FILL kernel already fills the pool by reading
  pinned memory, so the kernel-copy idea is not ours; what we add is the
  measurement of the DMA stall on a side stream
- remove the unmeasured claim that ggml-org#29949's GPU-fill mode is slower on
  PCIe 3.0; same-machine numbers are being measured
- list earlier llama.cpp work (ggml-org#20757, RFC ggml-org#28248, closed PRs ggml-org#21609,
  ggml-org#21614, ggml-org#21620, ggml-org#24524)
- note that our speed-ups use the best --n-cpu-moe split as baseline,
  not all experts on the CPU, and that the CPU-miss vs GPU-fill trade-off
  depends on PCIe speed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants