speed-bench: add MacBook Pro M4 Max q2 sweep

[datanerdie]

Adds an M4 Max row measured with the documented command, so the M5 Max and GB10 columns have a current Apple-silicon point to sit next to.

Setup

ds4f-q2 (DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf, 80.76 GiB, fully resident), b030961, MacBook Pro M4 Max 128 GB, macOS 26, Metal.

./ds4-bench -m <ds4f-q2> --prompt-file speed-bench/promessi_sposi.txt \
  --ctx-start 2048 --ctx-max 65536 --step-incr 2048 --gen-tokens 128

ctx	prefill	generation
2048	331.58 t/s	29.87 t/s
16384	280.53 t/s	28.75 t/s
32768	229.33 t/s	25.65 t/s
65536	190.79 t/s	23.28 t/s

Why a new file rather than updating m4_max.csv

The existing speed-bench/m4_max.csv predates the 0731 checkpoint and uses the older 6-column format, so it is not directly comparable to the refreshed rows — putting this in the same file would blur two different measurements. Happy to fold it in instead if you would rather have one file per chip.

One suggestion while you are here: the CSV filenames record only the chip, not the quant, build or chassis, which makes rows from different passes easy to compare by accident. A short header comment or a companion note in speed-bench/README.md would make provenance explicit.

Reading this against the M5 Max column

This machine logs Metal 4 tensor API disabled for pre-M5/pre-A19 devices, so the accelerated prefill path from 532ec8b and 222b2cb never dispatches here. The gap works out at 2.0–2.4× on prefill but only 1.19–1.34× on generation, which is what you would expect if the prefill difference is partly the TensorOps path rather than raw silicon. I cannot separate the two from this side — that would need DS4_METAL_DISABLE_METAL4 on an M5.

🤖 Generated with Claude Code

[datanerdie]

Follow-up: I said above that the silicon-versus-TensorOps split could not be separated without DS4_METAL_DISABLE_METAL4 on an M5. It can be answered from the M4 side instead, by going the other way and enabling the path here. Result: your gate is right, and the M5 prefill lead is silicon.

Method

Added "M4" to the default_enable device-name list in ds4_metal.m and re-ran the identical sweep — same machine, same ds4f-q2 file, same b030961, same command. The gate was the only variable.

First finding: the probe passes on M4 Max. The patched build prints

ds4: Metal device Apple M4 Max, 128.00 GiB RAM
ds4: Metal 4 tensor API enabled for Tensor kernels

so on macOS 26 the tensor API and mpp::tensor_ops::matmul2d compile fine on M4 — the restriction is policy, not capability.

But enabling it loses

ctx	prefill off → on	generation off → on
2048	331.6 → 312.2 (−5.8%)	29.87 → 30.44 (+1.9%)
8192	287.8 → 251.9 (−12.4%)	29.13 → 29.41 (+1.0%)
16384	280.5 → 273.0 (−2.7%)	28.75 → 28.42 (−1.1%)
32768	229.3 → 210.2 (−8.3%)	25.65 → 24.28 (−5.3%)
49152	199.2 → 200.8 (+0.8%)	23.76 → 23.79 (+0.1%)
65536	190.8 → 194.2 (+1.8%)	23.28 → 23.01 (−1.2%)

Aggregated over all 32 frontiers: prefill −4.2%, generation −0.9%, with prefill improving at only 10 of 32. That is exactly what the comment above default_enable predicts — on pre-M5 the TensorOps path maps to ordinary shader fallbacks, and here they are slower than the established kernels.

Consequence for the M5 comparison in this PR

Enabling the path moves M4 backwards, so the M5/M4 prefill ratio widens rather than closes: 2.38× → 2.53× at ctx 2048 and 2.43× → 2.65× at 32768 (roughly unchanged at 64K, 2.09× → 2.05×). So the caveat in the PR description should be read as settled: the prefill gap is the M5's per-GPU-core neural accelerators, not the gating decision holding M4 back. The generation gap stays 1.19–1.34× either way, which fits — decode is bandwidth-bound and TensorOps does not touch it.

Not proposing a change. The gate does the right thing, and this is only evidence for it. Reverted locally.

Caveats: one run per frontier and no thermal control between the two sweeps on a laptop, so a few percent of the aggregate could be drift; the per-cell signs are mixed at the top of the range. I did not check whether TensorOps changes numerics on M4, only throughput.

[OPS-NeoRetro]

@datanerdie, the file name of your benchmark submission is totally misleading. Please rename your submission file.

[OPS-NeoRetro]

Please rename this to m4_max_flash_0731.csv