docs/research(storage): inventory construction machinery and record comparison protocol (#1504)

[DecisionNerd]

Purpose

Produce the inventory and version-grounded comparison protocol required by #1504 before any replacement prototype is measured. This issue is the sequencing gate for the mechanism spikes.

Scope

• Map every custom sorting, partitioning, spilling, scheduling, buffering, cancellation, and memory-accounting mechanism in the construction path to callers, invariants, tests, and maintenance burden. Separate existing library reuse from candidate new reuse.
• Record current-main baseline revision and exact dependency versions (arrow, datafusion, tokio, Rayon, related crates). Link version-matched upstream docs for capability and limitation claims.
• Build a mechanism-by-candidate matrix: current implementation, applicable DataFusion facilities, Arrow kernels/batch facilities, Tokio coordination facilities, and meaningful hybrids. Include Rayon as the scheduling baseline. Do not assume each library replaces every mechanism.
• Separate execution partitioning from durable layout, generic mechanisms from graph semantics, transient spill from recovery authority, and library-accounted memory from total admitted memory.
• Before any spike measures results, record workloads, resource envelopes, repetitions, uncertainty treatment, regression thresholds, correctness gates, and how maintenance benefits will be weighed against performance costs. Do not invent a universal speedup threshold. Explicitly note that any production tradeoff changing an existing requirement needs maintainer acceptance.

Surfaces

Likely: crates/graphforge-storage/src/graph_construction*, construction_directory.rs, construction determinism/lifecycle tests, API import/resource-policy code. Related evidence from #1456 / #1465 may be cited with stated limitations.

Acceptance criteria

• Inventory covers sorting, partitioning, spilling, and scheduling, plus memory/cancellation interactions.
• Candidate matrix is version-grounded and distinguishes applicable vs inapplicable facilities per mechanism.
• Comparison protocol is recorded in-repo before spike benchmark results land; later protocol changes are explained, not retrofitted.
• Workloads (empty/tiny, normal, duplicate-heavy, skewed/hub, variable-width/nested properties, spill-forcing) and S18–S20 quiet-host measurement expectations are named without claiming S26 qualification.
• Correctness gates (identities, duplicate policy, surrogate/endpoint mappings, properties, ADR 0038 publication where required, CAS/receipts, structured errors, atomic publication) are listed as hard gates for later spikes.
• This issue blocks the epic(storage): evaluate DataFusion, Tokio, and Arrow replacements for custom construction machinery #1504 mechanism spikes until closed.

Non-goals

Runnable replacement prototypes; performance claims; ADR conclusions; production migration.

Parent

Native sub-issue and blocker of #1504.

[DecisionNerd]

Inventory and comparison protocol drafted in PR #1510:

• docs/development/construction-reuse-inventory-protocol-1505.md
• Cross-link from docs/development/evidence/encode-datafusion-seam-1465.md

Baseline revision: eeda1467 (arrow/parquet 58.4.0, datafusion 54.1.0, tokio 1.53.1, rayon 1.12.0 via exec). Mechanism spikes #1506–#1508 remain blocked until this lands.