epic: datasets as the first peer extension (gf-datasets) — v0.5.x #20

Vision

Core owns capabilities ("how graphs work"). Datasets own knowledge ("what graphs exist").
Datasets depend on Core; Core never depends on Datasets.

Datasets become the first of several peer extensions (dataset / ontology / algorithm /
epistemology / UI) — peers over the public API rather than special cases inside core. This epic
moves datasets out of core (greenfield: there is zero dataset code in any crate today — the old
registry lived only in the deleted Python shim, CurateLabs/graphforge-legecy#862) and establishes the peer-extension pattern.

Decision (see ADR 0008)

• Mechanism: a native Rust crate gf-datasets depending on gf-api (the public SDK),
  never on gf-core or the graph layer. This is the template the future Rust peers
  (algorithm, epistemology) reuse — a pure language-package approach would not generalize to them.
• Scope: both — generic loader/registry infrastructure and curated domain-content
  packages on top of it.
• Distribution: separate language packages — graphforge-datasets (PyO3/maturin) and
  @curatelabs/graphforge-datasets (napi-rs) — so pip install graphforge / npm i @curatelabs/graphforge stay
  dataset-free.

Boundary tests (the definition of done for the pattern)

1. Delete test: remove crates/gf-datasets from the workspace and cargo build/test the rest
   still passes — enforced by a CI gate.
2. Third-party test: a team could publish an unrelated dataset package (pharma, aerospace,
   intel) depending only on the published gf-api, without touching gf-core. A documented
   template proves it.

Architecture (realistic stack today)

Tokio → DataFusion → gf-core / graph layer → gf-api (public SDK) → gf-datasets (peer extension)

Note: there is no "GF Service" layer in the repo yet; extensions depend on gf-api directly.
A service/session layer can slot in later without changing this boundary.

Phases (sub-issues)

• docs: ADR 0008 — peer-extension boundary (datasets first) #19 — ADR 0008 — peer-extension boundary (datasets first; Core-never-depends-on-extensions; CI gate)
• feat(gf-datasets): scaffold the peer-extension crate + CI boundary/delete gate #18 — gf-datasets crate scaffold — new workspace member depending on gf-api; empty loader/registry
  surface; CI boundary/delete gate (graph-layer crates must not depend on gf-datasets; workspace
  builds with the crate removed) (not blocked by the write API)
• feat(gf-datasets): generic loader + registry infrastructure #17 — Generic loader + registry infrastructure — DatasetInfo / DatasetLoader / registry + cache;
  CSV / Cypher / GraphML / JSON-graph / LDBC loaders over the write API; restores the SNAP/LDBC/
  NetworkRepository catalog. (Uses the public v0.5.0 write API.)
• feat(gf-datasets): domain-content packages (genealogy, investigations, PM, software-eng, demo) #16 — Domain-content packages — genealogy, investigations, product management, software engineering,
  demo/sample — each a curated graph + its ontology YAML, as the first reference extensions.
• feat(bindings): graphforge-datasets (PyO3) + @curatelabs/graphforge-datasets (napi) packages #15 — Bindings packaging — graphforge-datasets (PyO3) + @curatelabs/graphforge-datasets (napi) crates +
  publish.yaml jobs; optional extras/peer-dep wiring so core installs stay dataset-free.
• docs: third-party peer-extension template + docs/extensions guide #14 — Third-party extension template + docs — docs/extensions/ guide + a template proving Test 2.

Dependencies / sequencing

• Uses the public write APIs shipped in v0.5.0. Extensions must not bypass gf-api or edit project storage directly.
• Post-v0.5.0 work in M15: Peer Extensions for v0.5.x, independent of M2 Swift/Kotlin bindings.

Reconciliation with CurateLabs/graphforge-legecy#860

CurateLabs/graphforge-legecy#860 (M19) previously bundled datasets and dev-tooling/benchmarks/examples migration, and proposed
placing datasets inside the Python binding. The dataset portion moves here (peer extension); CurateLabs/graphforge-legecy#860
is re-scoped to the non-dataset native-tooling migration only.