[alxmrs]

🪭 Claude (einfold)

Answers #79's second ask, and opens its third, as design v18.

The goal. For models trained in SQL, such as the GPT in xarray-sql's ML demos (xarray-sql's benchmarks/ml/02_nanogpt.py), memory limits model size as much as time does. Principle 9, "never more memory than the host", says how einfold's operators keep to it. They reserve what they read, and what their kernels allocate, from the host's memory pool, before allocating it. Where the pool can't give it, they run the host's own plan, whose operators can spill to disk. Every benchmark measures peak memory next to time.

What changes in the design.

• Section 4 adds principle 9.
• Section 10.2 gains a bullet on memory: what EinFoldExec does (Run the host's plan where memory runs short, as its spilling operators would #89, Estimate EinFoldExec's rows, so that the plan above it is planned as the host's #90), and what comes next. The positional fold should stream its larger operand in row blocks, so that its peak stays below the host's even without a pool to enforce it. That needs each dimension's extent before reading, and Dictionary-encoded dimensions: the dictionary is the axis, so its keys are the indices #80's dictionaries answer it: the dictionary's length, when it is the whole axis. Exact statistics or a first pass over the keys also give it.
• Section 15 reports peak memory next to time, which python/benchmarks/memory.py (Measure time and peak memory, einfold off and on, from Python #85) does for the Python extension.
• Section 14 opens Peak memory as a goal: measure it einfold on against off, and keep contractions from holding the joined rows #79's third ask, dense chains of folds: handing the dense form from one einfold operator to the next, so the joined intermediates never exist as rows. It lists what is open: the form handed on, how a host operator between two folds reads it, and when a chain is worth keeping dense.
• lessons.md records what running einfold under the GPT taught (A logical plan deeper than 100 protobuf messages fails to decode, so einfold fails queries DataFusion answers (the GPT's loss) #77, Peak memory as a goal: measure it einfold on against off, and keep contractions from holding the joined rows #79, The GPT's loss is 3.3× slower with einfold on (planning 6.5 s against 2.4 s), and a training step 4.7× #81):
  • profile through the real bridge;
  • report statistics, and cache them;
  • plan a fallback only if it runs;
  • batch dimensions are common.

Where the stack stands on #79, at its size (128 channels, context 64, batch 12, one layer), through datafusion-python on an Intel i7-8700:

	DataFusion alone	einfold, before #86	einfold, with #86–#90
the loss	108 s, 8.9 GB peak	42 s, 11.8 GB	13.6 s, 6.9 GB
the gradients	36.5 s, 9.8 GB		4.2 s, 1.8 GB

The peak is above the loaded tables. So the GPT already keeps to the principle at this size. Streaming in row blocks would make that hold by construction, not only under a pool.

🤖 Generated with Claude Code

https://claude.ai/code/session_01M96xSez6PZwTmfaGh2eJgS

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 06453927-e75d-41d4-8cca-6a7dede4ef6e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[alxmrs]

LGTM

[alxmrs]

🪭 Claude (einfold): Merged as #106, which is this PR's commit moved onto main so that it could go ahead of the rest of the stack. Closing this one; #92 now targets #90's branch.