Repository navigation
[Bug]: Windows HIP gfx1031: prompt reading ~30 % slower on the 0.1.43 engine than 0.1.40 (same model and server) #2210
Description
What happened
After updating from 0.1.40 to 0.1.43, prompt reading (prefill) is about 30 % slower on this card. Decode is unchanged. Running the 0.1.40 engine under the 0.1.43 server, same model, same prompts, brings the speed back, so the change is in the engine.
Method. One server start per setting, a short warm-up, then three fresh prompts (~16k, ~19k, ~16k tokens; non-overlapping slices of Markdown text, a unique first line so nothing is reused, max_tokens: 1). Speed is taken from the engine's prompt N tokens = 0 reused + N read in T ms line. The same slices in the same order for every setting. E–H ran back to back in alternating order, so file-cache warmth or background load can't favour one engine.
| Run | Model | Engine | Setting | Read tok/s |
|---|---|---|---|---|
| A | Swift 1.5 IQ2_XS | 0.1.43 | as set up | 393, 303, 381 |
| B | Swift 1.5 IQ2_XS | 0.1.43 | --ple-io direct, STRATA_WATCHDOG_S=900 |
396, 386, 388 |
| C1 | IQ3_XXS | 0.1.43 | as set up | 384, 380, 394 |
| C2 | IQ3_XXS | 0.1.40 | as set up | 560, 541, 592 |
| D | Swift 1.5 IQ2_XS | 0.1.40 | as set up | 530, 525, 554 |
| E | Swift 1.5 IQ2_XS | 0.1.43 | rematch | 397, 386, 396 |
| F | Swift 1.5 IQ2_XS | 0.1.40 | rematch | 525, 524, 553 |
| G | Swift 1.5 IQ2_XS | 0.1.43 | rematch | 398, 383, 395 |
| H | Swift 1.5 IQ2_XS | 0.1.40 | rematch | 524, 492, 533 |
| I | Swift 1.5 IQ2_XS | 0.1.43 | --prefill 8192 |
421, 393, 444 |
Request history on this PC agrees: fresh prompts of 4k+ tokens read at a median 540–578 tok/s on 0.1.40 in every size bucket (160 requests), and 303–395 on 0.1.43 (42 requests).
Ruled out
- PLE reads: 0.1.43 usually measures the drive at ~18–34 MB/s random 4 KiB and switches to
--ple-io mmap. Forcingdirect(run B, watchdog raised so a slow first read can't trip it) changes nothing, and in the rematch (E, G) its probe keptdirecton its own: still ~395. - Mostly not the Windows HIP prompt-chunk cap (Windows HIP (RX 7900 XTX): a long prompt with
--prefill 8192/automakes the commit charge jump ~70 GB in one second, the engine dies at the commit limit (0xC0000409); 7500 and below stay flat #1630): 0.1.40 reads in 8192-token chunks, 0.1.43 in 6144. Giving 0.1.43 8192 chunks (run I) recovers a little (~420 against ~395), far short of 0.1.40's ~525;--prefill 7168earlier measured about the same as 6144. - The Stager's sleeping waits (bc089d5): Linux-only since 2a8cded.
- ROCm: both engines built against the same ROCm, identical DLLs.
Same model on both engines: decode 39.2 vs 39.3 tok/s, and the same answers on a small set of tool-use, coding and log tasks.
Settings (from setup): --expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp ... --max-context 131072 --kv int8 --kv-resident 32768, env STRATA_HIP_PROMPT_F16=1.
I can bisect v0.1.40..v0.1.43 on this card, or run a build with diagnostics, if that helps.
Strata version, GPU, OS
Server 0.1.43 (7695859); engines 0.1.43 and 0.1.40, both self-built with tools\hip\build_windows.bat, STRATA_HIP_ARCHS=gfx1031, ROCm 10.2.0a20260930 · RX 6700 XT 12 GB (gfx1031) · Ryzen 5 5500, 64 GB RAM · Windows 11 Pro (26200)
Engine log
0.1.43 (run E), start and prompt lines (generation part trimmed):
strata generate: the SSD is kept awake while rows are read: one page of the table after 100 ms without a read, until 60 s after the last request (STRATA_SSD_KEEPALIVE=0 turns it off)
strata generate: CPU pool tasks/phase: 18 (automatic), participating threads: 6
strata generate: session is up (engine 0.1.43)
strata serve: prompt chunk auto: 6144 tokens, a 384-slot ring
strata serve: prompt 16408 tokens = 0 reused + 16408 read in 41324 ms (397.1 tok/s)
strata serve: prompt 18690 tokens = 0 reused + 18690 read in 48407 ms (386.1 tok/s)
0.1.40 (run F), the same prompts:
strata generate: the SSD is kept awake while rows are read: one page of the table after 100 ms without a read, until 60 s after the last request (STRATA_SSD_KEEPALIVE=0 turns it off)
strata generate: session is up (engine 0.1.40)
strata serve: prompt chunk auto: 8192 tokens, a 384-slot ring
strata serve: prompt 16408 tokens = 0 reused + 16408 read in 31251 ms (525.0 tok/s)
strata serve: prompt 18690 tokens = 0 reused + 18690 read in 35677 ms (523.9 tok/s)
strata serve: prompt 16234 tokens = 0 reused + 16234 read in 29373 ms (552.7 tok/s)
Activity