Skip to content

feat(io): export deterministic graph data subsets with referential closure #786

Description

@DecisionNerd

Problem

Component-level selection cannot express a specific graph or tabular data subset. Ad hoc filtering risks nondeterministic results, dangling edges/references, ontology mismatch, leaked properties, and packages whose claimed identity cannot be reproduced.

Objective

Export deterministic graph/data subset packages from one pinned generation with explicit referential-closure, schema/ontology, provenance, and privacy semantics.

Requirements

  • Define typed selectors over stable UUIDs, named datasets/participants, and supported deterministic public query/projection results. Never expose runtime catalog IDs as portable identities.
  • Define at least induced-node and edge-with-endpoints closure modes; reject dangling endpoints or references unless the selected package class explicitly models them.
  • Pin one source generation/checkpoint and record the canonical selector, source fingerprint, closure mode, ontology/schema requirements, property projection/redaction, counts, and result fingerprint.
  • Require deterministic ordering and reject selectors whose semantics/order cannot be reproduced under the portable-v2 contract.
  • Stream selection and package writing with bounded memory, backpressure, cancellation, progress, finite output/count limits, and no full subgraph materialization requirement.
  • Preview exact selector semantics, estimated counts where safe, dependency closure, projected/redacted fields, and authorization/privacy implications before writing.
  • Verify that every emitted edge endpoint/reference and every declared ontology/schema dependency is present or explicitly externalized by a supported contract.
  • Keep consume/import semantics explicit; creating a new subset project is allowed by the format, but merging into an existing graph is not introduced here.

Acceptance criteria

  • Repeating a supported selector against the same pinned source produces the same selection fingerprint, counts, ordered content, and package digest.
  • Induced-node and edge-with-endpoints fixtures prove referential closure and stable UUID preservation.
  • Property projection/redaction tests prove excluded fields and unrelated rows do not leak through payloads, manifests, errors, or provenance.
  • Ambiguous, nondeterministic, dangling, unsupported, or resource-exceeding selectors fail before a final package is published.
  • Large subset export remains bounded by execution/writer configuration rather than total selected bytes.
  • The resulting subset verifies through feat(io): verify expanded and bundled portable projects #784 and can be published/pulled without changing identity.

BDD completion scenarios

Scenario: Export an induced subgraph

Given stable selected node UUIDs in a pinned generation
When induced-node closure is exported
Then the package contains exactly those nodes, permitted properties, and edges whose endpoints are both selected
And its receipt records deterministic counts, ontology/schema dependencies, and selection fingerprint.

Scenario: Export selected edges safely

Given stable selected edge UUIDs
When edge-with-endpoints closure is exported
Then both endpoint nodes are included with the declared property projection
And no dangling edge or silent extra neighborhood is emitted.

Scenario: Reject nondeterministic selection

Given an unordered or time-dependent selector without a canonical snapshot/order contract
When export is planned
Then it fails with a typed determinism error
And no final package is published.

Observability and security

Report aggregate source/scanned/selected/rejected/redacted counts, phase timing, spill/progress, and failure codes. Authorization remains caller-owned; selectors, property values, UUID inventories, and graph content must not enter logs.

Testing and documentation

Cover closure modes, isolated nodes, self-loops, multiedges, missing endpoints, ontology constraints, property redaction, deterministic reruns, cancellation/spill/resource limits, corruption verification, and clean subset-project reopen. Document reproducible subset recipes and privacy review.

Non-goals

Graph merge/upsert, arbitrary nondeterministic query export, inference, anonymization guarantees, policy-engine design, or foreign execution engines.

Related issues

Portable-v2 epic: #740. Component selection prerequisite: #785. Streaming result export remains #743.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    coreCore source code changesenhancementNew feature or requesttestingTest coverage and testing infrastructure

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions