perf(exec): hydrate only requested variable-length path nodes

[DecisionNerd]

Problem

Confirmed on current main at 6dc51fa47a9658d26a3f6e51e8a8261aaf9b4b86.

Each variable-length path-node UDF invocation calls hydrate_path_node_children at crates/graphforge-rel/src/expr.rs:11998-12000. The helper reads every node batch and builds a graph-wide UUID map at crates/graphforge-rel/src/expr.rs:12057-12083, then reads and concatenates every property stem and indexes every row at crates/graphforge-rel/src/expr.rs:12111-12166.

Work is proportional to total graph/property volume rather than requested nodes and repeats across UDF/output batches. A small nodes(p) result can scan and materialize the whole graph and recreate contiguous Arrow limits. Closed M4 issue #341 addressed analyst-output shaping, not this Cypher UDF.

Objective

Hydrate variable-length path nodes demand-first and batchwise, with work and peak memory bounded by requested UUIDs plus configured resource limits.

Requirements

• Deduplicate requested UUIDs before reads.
• Push identity selection into batchwise node/property reads or an indexed gather path; do not concatenate complete tables.
• Avoid rebuilding graph-wide UUID maps per UDF/output batch.
• Preserve full labels, properties, ordering, nullability, repeated-node semantics, cancellation, and structured resource errors.
• Consume the established embedded resource/batch policy; no independent pool or unbounded cache.
• Keep Rust authoritative and bindings thin.

Acceptance Criteria

• A small path in a graph with many irrelevant nodes/properties does not scan or materialize all property rows.
• No complete property stem passes through concat_batches solely for path hydration.
• Structural counters/test instrumentation show work bounded by selected UUIDs/batches rather than total rows.
• Repeated UUIDs hydrate once internally while preserving repeated public positions.
• Results are identical across accepted batch boundaries and reopen.
• Cancellation/resource exhaustion returns structured errors without partial results.
• Peak-memory evidence improves on a representative sparse-selection workload without correctness regression.

BDD Completion Scenarios

Scenario: A small path does not materialize a large graph

Given a graph with many nodes and property stems
And a path containing a small selected subset
When nodes(p) is evaluated
Then GraphForge gathers only needed UUIDs within the resource policy
And returns complete canonical node values.

Scenario: Batch boundaries do not alter hydration

Given the same graph/path under different accepted batch sizes
When path nodes are hydrated
Then labels, properties, ordering, repeats, nulls, and errors are identical
And only bounded I/O/allocation observations differ.

Implementation Notes

Likely surfaces: crates/graphforge-rel/src/expr.rs, storage/catalog readers, UUID lookup/index facilities, resource policy, and path-expression tests.

Issue #705 must land first so the optimization preserves the authoritative full-label contract. A native blocked-by relationship will encode this.

Observability

Use aggregate test/benchmark counters for rows/batches scanned, rows gathered, bytes materialized, and peak memory. Never log values, UUIDs, properties, or paths.

Security And Privacy

Any cache/index must be generation-safe, bounded, instance-scoped, and free of user-data logging.

Testing

Use a sparse-selection fixture with irrelevant nodes and multiple property stems. Cover repeated/missing UUIDs, cancellation, reopen, rollover, and deterministic results. Compare structural counters and peak memory, not timing alone. Preserve #705's full-label regression.

Documentation

Update execution/scale docs if hydration gains a documented resource/indexing contract.

Non-Goals

• Reworking analyst output shaping already closed by perf(exec): shape analyst outputs directly into bounded Arrow batches #341.
• Changing path schemas or semantics.
• Foreign runtimes or binding-side hydration.

Related Issues

• perf(exec): shape analyst outputs directly into bounded Arrow batches #341 — related closed M4 analyst-output work; does not cover this UDF.
• Blocked by fix(exec): preserve all node labels in variable-length path values #705.
• Parent tracker will be attached through GitHub's native sub-issue relationship.

[coderabbitai]

🔗 Related PRs

#382 - test(storage-api): close remaining coverage floors for #369 [merged]
#413 - fix(bdd): bulk add_nodes operation_uuid readback (#355) [merged]
#489 - perf(storage): stream partitioned Parquet execution (#339) [merged]
#607 - perf(exec): parallelize resource_allocation on ComputePool (#513) [merged]
#635 - perf(exec): serial disposition for Leiden (#527) [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[DecisionNerd]

Implementation PR (stacked on #718): #726

Merge order: land #718 first, then #726 (rebase onto main if needed). Auto-close keyword may not bind while base is fix/705-path-node-full-labels.