Laya scorer: 8.75 s -> 0.38 s per decision, byte-identical output

[bong-water-water-bong]

The encoder ran its projections, GeGLU and attention on one thread. Only the small decision head used the threaded linear(). This PR changes three things:

• Wqkv, Wo and both GeGLU projections go through linear().
• Encoder attention runs one thread per (sequence, head), each with its own score buffer.
• linear() reads each weight row once per four input rows. The four sums are independent, so no sum changes order.

Measured on Strix Halo with 1bit route --laya-model ~/models/laya-pinned --devices vulkan,hrx,zinc:

	before	after
decision	8.07-8.17 s	0.375-0.391 s
load (once)	2.7 s	2.7 s
laya_gate run (each checkpoint)	53 s	4.5 s

Eight rows per weight load gave no further gain (0.38 s).

Correctness: laya_gate output is byte-identical to the single-threaded scorer on the root and typed-decisions/ checkpoints (diffed against baseline runs, both PASS). ctest -R laya: 3/3 pass, including laya_route_e2e.

Docs: laya.md gains a cost section, and README and PORTING quote 0.38 s.

🤖 Generated with Claude Code

[context7]

Docs7 for 1bit-monster/engine

Result	Status	Action
Deployment	➖ Not used	—
Content review	➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan.	—

Commit 9ed1e5d

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Thread Safety in `worker_threads()` The worker_threads() function uses a static local variable to cache the number of worker threads. While this is generally safe in C++11 and later due to thread-safe initialization of static local variables, it's worth noting that this approach relies on the compiler and standard library implementation details. If the code is ever compiled with a non-conforming compiler or in a context where static initialization is not thread-safe, this could lead to race conditions. However, given the context and standard compliance, this is likely acceptable. int worker_threads() { static const int n = int(std::clamp(std::thread::hardware_concurrency() / 2, 1u, 16u)); return n; } Potential Integer Overflow in `parallel_ranges` In the parallel_ranges function, the calculation int((size_t)n * p / t) and int((size_t)n * (p + 1) / t) could potentially lead to integer overflow if n is very large and p or p + 1 are close to the maximum value of int. While this is unlikely in practice given the typical sizes of the parameters involved, it's a potential issue that should be considered for robustness. for (int p = 0; p < t; ++p) pool.emplace_back(fn, int((size_t)n * p / t), int((size_t)n * (p + 1) / t)); Memory Access Pattern in `linear()` The linear() function accesses memory in a pattern where each weight row is read once per four input rows. This is intended to improve cache locality and reduce memory bandwidth requirements. However, if the input matrix in or the weight matrix W are not properly aligned or if the memory layout does not align well with the access pattern, it could lead to performance degradation or even incorrect behavior due to memory access violations. This is particularly relevant in a hardware-accelerated environment where memory access patterns are critical. for (; i + 4 <= m; i += 4) { const float *x0 = in + (size_t)i * n, *x1 = x0 + n, *x2 = x1 + n, *x3 = x2 + n; float a0 = b, a1 = b, a2 = b, a3 = b; for (int c = 0; c < n; ++c) { const float w = wr[c]; a0 += x0[c] * w; a1 += x1[c] * w; a2 += x2[c] * w; a3 += x3[c] * w; } out[(size_t)i * k + j] = a0; out[(size_t)(i + 1) * k + j] = a1; out[(size_t)(i + 2) * k + j] = a2; out[(size_t)(i + 3) * k + j] = a3; } for (; i < m; ++i) { const float* x = in + (size_t)i * n; float acc = b; for (int c = 0; c < n; ++c) acc += x[c] * wr[c]; out[(size_t)i * k + j] = acc; }