Fuse unnormalized and expression-weighted pipelines on special values with SQL's results #116

🪭 Claude (einfold)

Fuse unnormalized pipelines and expression weights on special values with SQL's results, rather than declining.

#107 lets a pipeline's weights be any row expression of its scores (Weight::Of), and lets it go unnormalized (Normalizer::None). On longer runs of pipeline_vs_sql.rs (EINFOLD_CASES=4000), the fused algorithm gave results that weren't SQL's where a weight or value wasn't finite:

• seed 2810052966: identity weights (s AS w), unnormalized: einfold inf, SQL NaN;
• seed 2810054457: exp(s) weights, unnormalized: einfold -inf, SQL NaN.

SQL computes SUM(w * v) term by term, so a single inf · 0 or +inf + -inf makes the sum NaN. einfold computes the weighted sum as a matrix product (gemm::multiply_add, through matrixmultiply), and the special values come out otherwise. I haven't yet found where: in the library's kernel, in how tiles split the sum, or in how weights are placed. Normalized softmax pipelines match SQL on the same tests, since there a special value reaches the division by the normalizer.

For now, #107's fused algorithm declines in these cases, so the host's plan gives SQL's answer:

• a weight or a value that isn't finite, under expression weights or with no normalizer;
• a row whose sums of finite terms overflowed before a larger score rescaled them (CodeRabbit's finding on Let a pipeline's weights be any row expression of its producer's values #107).

Only queries that meet such values lose fusion. To fuse them again: find the mechanism, then handle special values exactly, for example by counting non-finite terms per row beside the product.

🤖 Generated with Claude Code

https://claude.ai/code/session_01M96xSez6PZwTmfaGh2eJgS