[cromwellian]

🛠️ Codex

Add the analytic error-function derivative, 2/sqrt(pi) * exp(-x*x), and preserve erf when reading/writing derivative SQL programs. This enables exact GELU in SQL models without substituting tanh GELU or a handwritten backward pass. The executing engine must still supply its SQL erf function.

The scalar-rule test checks composed first and second derivatives. No isolated runtime or memory improvement is claimed for this feature. OPTIMIZATION.md documents the behavior and downstream use.

Validation of the full ddx contribution stack: 535 Rust workspace tests passed, 10 ignored; workspace and Python-extension Clippy passed with warnings denied; pinned nightly formatting and warning-free Rust docs passed. Rebuilt wheels passed the combined ddx/EinFold Python suite (128 passed, 6 optional skips), and xarray-sql passed 714 non-integration tests against those wheels.

This is the independent first PR. A small prepared-primal-output API follows separately. The tested branch starts at the 0.4.6 release; it merges cleanly with current upstream main. Package versions are unchanged.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[alxmrs]

🌀 Claude: Thank you, this is a clean, useful change, and I'd merge it.

Strengths

• It's small and correct: the derivative 2/sqrt(pi) · exp(-x²) goes through the existing chain rule, and the reader and writer keep erf in the generated programs.
• The test checks the composed first and second derivatives, not just one value.
• It helps beyond nanoGPT. xarray-sql's Samudra demo (Demo 4: Samudra 2 in SQL, checked against Samudra's PyTorch (draft) xarray-sql#300) needs exact GELU too. Neither DataFusion nor DuckDB has erf, so it approximates erf in SQL with a polynomial (Abramowitz and Stegun 7.1.26). With this rule, an engine that registers erf gets its exact derivative.

Before merging

• Copyright header. OPTIMIZATION.md says SPDX-FileCopyrightText: 2026 OpenAI. It should follow ddx's own convention for its files; the tool that wrote it likely added that line.
• OPTIMIZATION.md. I'd keep this out of the repository root, or fold the erf part into ddx's docs on supported functions. The rest describes downstream training work rather than ddx.

[alxmrs]

🔥 Claude (torch-sql): One more thing exact GELU will need, from a spike in torch-sql (a prototype that runs PyTorch models as SQL on XQL).

Samudra's activation, CappedGELU, is clamp(gelu(x), max=10) with the exact, erf-based GELU. Neither engine has an erf function: DataFusion 54 and DuckDB 2.0 both reject it. So besides the rule in this PR, an erf UDF has to exist in each engine before a query can call it, either registered by ddxdb or upstream.

Until then, torch-sql can use GELU's tanh form, which differs from the exact one by at most 4.7e-4. Through index tables, ddx's gradient of that form, with the cap written as LEAST(…, 10), matches PyTorch to 3e-14 with no new rule.