Repository navigation
Community benchmark: RTX Pro 4000 Blackwell, Swift 1.5 IQ3_XXS 1 #2169
Description
Community benchmark on NVIDIA RTX PRO 4000 Blackwell
Measured on 2026-10-12 with tools/strata_bench.py (suite 1, 3 runs per test, medians with ranges). Speeds are the engine's own timings; first-token times are measured by the client over HTTP. Speed only: no correctness check was run.
Hardware and software
- GPU and VRAM: NVIDIA RTX PRO 4000 Blackwell, 24 GiB VRAM
- CPU: Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz (60 cores, 60 threads); installed RAM: 220.5 GiB; storage: not measured
- OS: Linux 6.12.0-211.49.1.el10_2.x86_64; GPU driver: 580.105.08; CUDA: 13.0
- Strata: project 0.1.43.1, commit a0116b2; engine 0.1.43.1
- Background workloads and power limits: not recorded
Model and configuration
- Model and quantization: swift-1.5-iq3_xxs
- Context: 131072 tokens; parallel slots: 1
- Settings: the install's own configuration, unchanged; thinking off for the suite; temperature 0
strata --pack swift-iq3_xxs --native Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf --ple-gguf Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf --expert-profile expert-profile.bin --expert-cache auto --prefill auto --spec 4 --mtp rt --max-context 131072 --kv int8 --ple-io ram --kv-resident 32768 --vision --vram-reserve-mib 700 --spec-min-p 0.5 --pool-workers 15 --adapt-every 1 --adapt-swaps 80 --adapt-decay 0.97
Method
Run by python tools/strata_bench.py against the HTTP API of a Strata server (started by the benchmark from the install configuration); model loading is not timed. One warm-up chat and a 1K prompt are discarded first. Every prompt starts with a unique tag, so the conversation cache cannot help the fresh-prompt tests.
- short chat: the three questions of
tools/ab_engine.py, up to 100 new tokens, 3 runs - 4K / 16K prompts: generated word lists sized by a calibration request, 24 new tokens, 3 fresh prompts each
- agent session: system prompt and four tools, then 5 turns that each add about 1500 words of a project file and continue from the model's own earlier answers (the way agent clients resend a conversation), 64 new tokens per turn, 3 sessions; the cache numbers come from the answer's usage (
cached_tokens) and timings (prompt_n) - concurrency (when run): N requests sent at the same moment, total tokens over the wall time of the burst, latency over all requests
Results
| Configuration | Actual prompt tokens | Reused tokens | Generated tokens | Runs | Prompt tok/s median and range | Decode tok/s median and range | TTFT seconds median and range |
|---|---|---|---|---|---|---|---|
| short chat | 46 (43-51) | 0 | 100 (100-100) | 9 | not measured | 70.1 (51.9-83.0) | 0.46 (0.38-0.60) |
| 4K prompt, fresh | 4019 (4019-4019) | 0 (0-0) | 24 cap | 3 | 1512.6 (1463.3-1516.4) | not measured | 2.69 (2.68-2.78) |
| 16K prompt, fresh | 16058 (16058-16058) | 0 (0-0) | 24 cap | 3 | 2315.0 (2301.7-2323.2) | not measured | 7.01 (6.99-7.05) |
| agent session, follow-up turns | 14474 (14430-14491) at the last turn | see below | up to 64 | 3 | see below | 96.0 (78.9-132.8) | 2.79 (2.52-2.96) |
Ranges are the lowest and highest of the runs; Runs counts requests that returned a token.
Agent session (conversation cache)
Cache hit over the follow-up turns: 68.5 % (68.5-68.6); earlier-turn tokens read again per session: 144 (75-144) (earlier turns that the cache did not hold; the 13635 (13633-13636) new tokens of the files are read once); first turn (nothing cached) first token: 1.38 s (1.37-1.44).
| Turn (run 1) | Prompt tokens | Reused | Read | Prompt tok/s | First token (s) | Decode tok/s |
|---|---|---|---|---|---|---|
| 0 | 609 | 0 | 609 | 430.3 | 1.444 | 66.9 |
| 1 | 3703 | 620 | 3083 | 1095.4 | 2.862 | 79.1 |
| 2 | 5236 | 3766 | 1470 | 584.2 | 2.561 | 88.7 |
| 3 | 8680 | 5229 | 3451 | 1191.8 | 2.955 | 83.8 |
| 4 | 11562 | 8702 | 2860 | 1027.7 | 2.85 | 96.0 |
| 5 | 14430 | 11584 | 2846 | 1043.2 | 2.793 | 78.9 |
Per-request data: results.json. Engine log lines from the start: engine-log-excerpt.txt (when this run started the server).
Correctness and limitations
Not measured: this is a speed suite. The conversation uses the model's own replies, so token counts differ a little between runs. Machine details come from the server, nvidia-smi/rocm-smi and the OS; background load and power limits are not recorded. Total time of the suite: 108 s.
Activity