Purpose
Evaluate DataFusion/Arrow sorting and partitioning facilities (and hybrids) against GraphForge's custom construction sorting/partitioning, with runnable evidence or a specific independently reviewed incompatibility proof.
Prerequisite
Blocked by the #1504 inventory/protocol child. Use that protocol; do not invent thresholds after measuring.
Scope
- Prototype representative construction/shaping operations using suitable library sorting and partitioning facilities, including selective kernel reuse and integrated execution alternatives.
- Compare allocation/copy costs, ordering semantics, global merges, repartitioning, skew behavior, and memory admission against the current implementation.
- Cover power-law hubs and partitions exceeding the current fail-closed budget; distinguish safe refusal from successfully processing a larger input. Evaluate bounded processing/spill alternatives without assuming adaptive partitioning alone solves skew.
- Explain interaction with recorded UUID splitters, surrogate assignment, endpoint resolution, and canonical publication. Do not require byte-identical transient intermediates unless the publication contract demands them.
Acceptance criteria
Non-goals
Production migration; rewriting graph semantics/public APIs; closing #1504 alone.
Parent
Native sub-issue and blocker of #1504. Blocked by inventory/protocol child.
Purpose
Evaluate DataFusion/Arrow sorting and partitioning facilities (and hybrids) against GraphForge's custom construction sorting/partitioning, with runnable evidence or a specific independently reviewed incompatibility proof.
Prerequisite
Blocked by the #1504 inventory/protocol child. Use that protocol; do not invent thresholds after measuring.
Scope
Acceptance criteria
Non-goals
Production migration; rewriting graph semantics/public APIs; closing #1504 alone.
Parent
Native sub-issue and blocker of #1504. Blocked by inventory/protocol child.