Skip to content

[Bug]: Windows HIP gfx1031: prompt reading ~30 % slower on the 0.1.43 engine than 0.1.40 (same model and server) #2210

Open

Description

@huy-sec

What happened

After updating from 0.1.40 to 0.1.43, prompt reading (prefill) is about 30 % slower on this card. Decode is unchanged. Running the 0.1.40 engine under the 0.1.43 server, same model, same prompts, brings the speed back, so the change is in the engine.

Method. One server start per setting, a short warm-up, then three fresh prompts (~16k, ~19k, ~16k tokens; non-overlapping slices of Markdown text, a unique first line so nothing is reused, max_tokens: 1). Speed is taken from the engine's prompt N tokens = 0 reused + N read in T ms line. The same slices in the same order for every setting. E–H ran back to back in alternating order, so file-cache warmth or background load can't favour one engine.

Run Model Engine Setting Read tok/s
A Swift 1.5 IQ2_XS 0.1.43 as set up 393, 303, 381
B Swift 1.5 IQ2_XS 0.1.43 --ple-io direct, STRATA_WATCHDOG_S=900 396, 386, 388
C1 IQ3_XXS 0.1.43 as set up 384, 380, 394
C2 IQ3_XXS 0.1.40 as set up 560, 541, 592
D Swift 1.5 IQ2_XS 0.1.40 as set up 530, 525, 554
E Swift 1.5 IQ2_XS 0.1.43 rematch 397, 386, 396
F Swift 1.5 IQ2_XS 0.1.40 rematch 525, 524, 553
G Swift 1.5 IQ2_XS 0.1.43 rematch 398, 383, 395
H Swift 1.5 IQ2_XS 0.1.40 rematch 524, 492, 533
I Swift 1.5 IQ2_XS 0.1.43 --prefill 8192 421, 393, 444

Request history on this PC agrees: fresh prompts of 4k+ tokens read at a median 540–578 tok/s on 0.1.40 in every size bucket (160 requests), and 303–395 on 0.1.43 (42 requests).

Ruled out

Same model on both engines: decode 39.2 vs 39.3 tok/s, and the same answers on a small set of tool-use, coding and log tasks.

Settings (from setup): --expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp ... --max-context 131072 --kv int8 --kv-resident 32768, env STRATA_HIP_PROMPT_F16=1.

I can bisect v0.1.40..v0.1.43 on this card, or run a build with diagnostics, if that helps.

Strata version, GPU, OS

Server 0.1.43 (7695859); engines 0.1.43 and 0.1.40, both self-built with tools\hip\build_windows.bat, STRATA_HIP_ARCHS=gfx1031, ROCm 10.2.0a20260930 · RX 6700 XT 12 GB (gfx1031) · Ryzen 5 5500, 64 GB RAM · Windows 11 Pro (26200)

Engine log

0.1.43 (run E), start and prompt lines (generation part trimmed):

strata generate: the SSD is kept awake while rows are read: one page of the table after 100 ms without a read, until 60 s after the last request (STRATA_SSD_KEEPALIVE=0 turns it off)
strata generate: CPU pool tasks/phase: 18 (automatic), participating threads: 6
strata generate: session is up (engine 0.1.43)
strata serve: prompt chunk auto: 6144 tokens, a 384-slot ring
strata serve: prompt 16408 tokens = 0 reused + 16408 read in 41324 ms (397.1 tok/s)
strata serve: prompt 18690 tokens = 0 reused + 18690 read in 48407 ms (386.1 tok/s)

0.1.40 (run F), the same prompts:

strata generate: the SSD is kept awake while rows are read: one page of the table after 100 ms without a read, until 60 s after the last request (STRATA_SSD_KEEPALIVE=0 turns it off)
strata generate: session is up (engine 0.1.40)
strata serve: prompt chunk auto: 8192 tokens, a 384-slot ring
strata serve: prompt 16408 tokens = 0 reused + 16408 read in 31251 ms (525.0 tok/s)
strata serve: prompt 18690 tokens = 0 reused + 18690 read in 35677 ms (523.9 tok/s)
strata serve: prompt 16234 tokens = 0 reused + 16234 read in 29373 ms (552.7 tok/s)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions