Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

Post-v0.5.0 work lands here. The v0.5.0 publication cut is under `[0.5.0]` below.

- Define the `.graphforge/` repository integration and deployment configuration
boundary, including closed versioned contracts, ignored data surfaces,
secret-free IaC resolution, and cross-ecosystem ownership (#225).
- Publish the GitHub Pages documentation at `https://docs.graphforge.sh`, with
root-relative site URLs and current `CurateLabs/graphforge` source links (#223).

Expand Down
4 changes: 4 additions & 0 deletions docs-site/astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,10 @@ export default defineConfig({
label: '0015 — Embedded Write Modes',
slug: 'adr/0015-embedded-write-modes',
},
{
label: '0016 — Repository Integration',
slug: 'adr/0016-repository-integration-and-deployment-configuration',
},
],
},
],
Expand Down
1 change: 1 addition & 0 deletions docs-site/scripts/sync-content.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,7 @@ const PAGES = [
'adr/0013-project-generation-protocol.md',
'adr/0014-workspace-checkpoints.md',
'adr/0015-embedded-write-modes.md',
'adr/0016-repository-integration-and-deployment-configuration.md',
'releases/roadmap.md',
'legal/licensing.md',
'community/security.md',
Expand Down
143 changes: 143 additions & 0 deletions docs/adr/0016-repository-integration-and-deployment-configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
# ADR 0016: Repository integration and deployment configuration boundary

**Status:** Accepted

**Date:** 2026-07-30

**Build target:** post-v0.5.0

**Contracts:** [`graphforge-project-config/1`](../contracts/graphforge-project-config-v1.schema.json), [`graphforge-resolved-config/1`](../contracts/graphforge-resolved-config-v1.schema.json)

**Related:** ADR 0013 (project generations), ADR 0014 (workspace checkpoints), ADR 0015 (embedded write modes), issues #215, #219, and #225

## Context

GraphForge has a durable embedded project format, native bindings, a Rust CLI,
and agent workflows, but a code repository has no canonical way to declare its
GraphForge inputs, keep generated graph data out of Git, or present one validated
deployment intent to local tools and infrastructure-as-code systems.

The contract must keep configuration, ontology, schemas, migrations, and
reproducibility recipes reviewable while excluding graph contents, imported
datasets, materialized seeds, snapshots, exports, locks, journals, and caches.
Rust must remain the behavioral authority; Python and Node must stay thin and
equivalent. Pulumi and Terraform need deterministic preview/plan input without
credentials, network access, or provisioning. Remote authorities remain peer
extensions, not a server added to core. Python-only skill installation must not
require Node.

ADR 0013 makes a validated generation selected by `CURRENT` the sole project
authority. ADR 0014 keeps root configuration outside checkpoints unless adopted
into canonical workspace participants. ADR 0015 keeps concurrency embedded and
transport-neutral. Repository integration must not create another persistence
or coordination authority.

## Options considered

1. **One `.graphforge/` namespace with selectively ignored data directories.**
Discoverable and cohesive, but cleanup and ignore management must protect
tracked siblings.
2. **Separate `.graphforge/` runtime and `graphforge/` definitions.** The split
is physical, but the near-identical names are confusing to people and tools.
3. **Entirely ignored `.graphforge/` plus scattered root configuration.** Simple
ignore behavior, but no coherent reviewed home for related definitions.
4. **Commit or deploy the live project directory.** Git cannot safely merge
immutable generations, atomic pointers, locks, and binary participants; IaC
would confuse deployment intent with mutable user data.
5. **Ecosystem-specific contracts with Python delegating skills to NPX.** Less
initial packaging work, but it creates semantic drift, requires Node for
Python users, and weakens offline operation.

## Decision

### One repository namespace

```text
.graphforge/
├── graphforge.yaml
├── ontology/
├── schemas/
├── seeds/
├── migrations/
├── imports/
├── exports/
└── state/
```

`graphforge.yaml`, ontology sources, schemas, migration definitions, and seed
recipes/manifests are tracked. Seed manifests may contain stable identities,
external locations, digests, mappings, and generator parameters, never rows.

`state/`, `imports/`, and `exports/` are ignored. Actual graph contents, source
datasets, materialized seeds, Arrow/Parquet/database files, generations,
snapshots, archives, locks, journals, caches, and trash never belong in code
Git. `init` owns an idempotent ignore block for exactly those three directories;
it never stages, untracks, commits, or ignores all of `.graphforge/`.

The live embedded project is `.graphforge/state/` and retains ADR 0013
semantics: `CURRENT`, not Git or YAML, selects authoritative data. Every
worktree gets its own default state. Explicit external roots must pass the same
containment, symlink, and format validation as repository-local roots.

### Closed portable configuration

`.graphforge/graphforge.yaml` implements the closed
`graphforge-project-config/1` schema. Unknown fields, unsupported versions,
unsafe paths, inline secrets, and unbounded collections fail before mutation.
It describes definition paths, digest-addressed external sources, named target
intent, pinned artifacts, write mode, storage, finite resources, networking,
health, observability, backup, and secret references—never secret values.

Provider-specific accounts, regions, registries, clusters, networks, identities,
and secret managers are Pulumi/Terraform inputs, not an open configuration map.

Resolution emits canonical UTF-8 JSON plus LF conforming to
`graphforge-resolved-config/1`: keys sorted lexicographically, explicit defaults,
repository-relative `/` paths, sources and targets ordered by ID, and secret
references without resolving values. Static validity, infrastructure plan,
connectivity, health, and capability compatibility remain distinct states.

### Ownership boundaries

Rust owns repository discovery, validation/resolution, path safety, ignore-file
editing, project lifecycle, portable interchange, checkpoint revert, and
destructive-operation guards. The Rust `gf` CLI is the reference behavior;
Python and Node expose thin `uvx graphforge` and `npx @graphforge/cli` surfaces.

Skills have one checked-in source. Python and npm ship parity-checked copies and
install directly under `.agents/skills/`; Python never shells out to NPX.

GraphForge validates intent and readiness. Pulumi and Terraform own provider
configuration, preview/plan, provisioning, drift, and teardown. IaC state is
not project state. Destroy removes only IaC-owned resources and never invokes
local project removal. Remote apply consumes the checksum-pinned authority
artifact owned by #215; core gains no server, transport, authentication system,
or distributed authority. Local state is never uploaded implicitly, and data
initialization is a separately authorized external digest-addressed import.

## Consequences

### Positive

- One discoverable repository convention keeps actual data out of Git.
- Rust, bindings, agents, Pulumi, and Terraform share one versioned contract.
- Preview/plan can fail closed without credentials, network access, or mutation.
- Python-only users can install skills without Node; core stays embedded.

### Negative

- Selective ignores require more care than ignoring `.graphforge/` wholesale.
- Python and npm must ship parity-checked generated schema and skill assets.
- Provider settings remain separate, so deployment needs IaC stack config.
- Remote provisioning cannot finish until a compatible #215 artifact exists.

### Compatibility and follow-up

Existing v0.5 project roots remain valid and are not migrated automatically.
Using `.graphforge/state/` is additive; raw historical directories are not
portable imports. Schema changes are explicit and fail closed while pre-v1.

Issue #219 remains the close gate. Its native sub-issues implement lifecycle,
interchange, binding CLIs, skills, static IaC validation, and #215-dependent
remote provisioning. Direct contract and clean-environment evidence is required
before the canonical tracker closes.
1 change: 1 addition & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ are not retained in this tree.
| 0013 | [Durable v0.5 project-generation protocol](0013-project-generation-protocol.md) | `0013-project-generation-protocol.md` |
| 0014 | [Complete-workspace checkpoints and generation-preserving revert](0014-workspace-checkpoints.md) | `0014-workspace-checkpoints.md` |
| 0015 | [Three embedded project-write modes](0015-embedded-write-modes.md) | `0015-embedded-write-modes.md` |
| 0016 | [Repository integration and deployment configuration boundary](0016-repository-integration-and-deployment-configuration.md) | `0016-repository-integration-and-deployment-configuration.md` |

## Numbering

Expand Down
3 changes: 2 additions & 1 deletion docs/book/architecture/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,7 @@ Shipped v0.5.0 expects these surfaces to stay green on `main`:
- [Algorithm Verbs](algorithms.md) — full algorithm catalog across rank/cluster/paths/analyze/similar
- [Execution Model](execution-model.md) — DataFusion integration, custom graph operators, Arrow result streams
- [Storage](storage.md) — StorageProvider trait, Parquet provider
- [ADR Index](../../adr/README.md) — contiguous v0.5.0 decision log (`0001`–`0015`)
- [ADR Index](../../adr/README.md) — contiguous decision log (`0001`–`0016`)
- [ADR 0001: Rust Core](../../adr/0001-rust-core.md) — Rust core and binding strategy
- [ADR 0002: RD+Pratt Parser](../../adr/0002-lr1-grammar.md) — Parser algorithm decision
- [ADR 0003: Progressive Ontology](../../adr/0003-progressive-ontology.md) — exploration-first ontology modes
Expand All @@ -266,4 +266,5 @@ Shipped v0.5.0 expects these surfaces to stay green on `main`:
- [ADR 0013: Project Generations](../../adr/0013-project-generation-protocol.md) — durable project-generation protocol
- [ADR 0014: Workspace Checkpoints](../../adr/0014-workspace-checkpoints.md) — complete-workspace checkpoints and revert
- [ADR 0015: Embedded Write Modes](../../adr/0015-embedded-write-modes.md) — single, queued, and optimistic project writes
- [ADR 0016: Repository integration and deployment configuration](../../adr/0016-repository-integration-and-deployment-configuration.md) — tracked definitions, local data, CLI, skills, and IaC ownership boundaries
- [Roadmap](../../releases/roadmap.md) — Milestones and timeline
1 change: 1 addition & 0 deletions docs/contracts/examples/graphforge-resolved-v1.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"contract":"graphforge-resolved-config/1","project":{"exports":".graphforge/exports","imports":".graphforge/imports","integration_root":".graphforge","migrations":".graphforge/migrations","ontology":".graphforge/ontology","schemas":".graphforge/schemas","seeds":".graphforge/seeds","state":".graphforge/state"},"secrets":[{"id":"service-token","source":"secret_manager"}],"sources":[{"id":"example-data","media_type":"application/vnd.apache.parquet","sha256":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","uri":"https://example.invalid/graphforge/example.parquet"}],"targets":[{"artifact":{"kind":"python_wheel","sha256":"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb","version":"0.5.0"},"backup":{"checkpoints":false},"health":{"timeout_seconds":30},"id":"local","kind":"embedded","network":{"exposure":"none","tls_required":false},"observability":{"logs":true,"metrics":false,"traces":false},"resources":{},"secret_ids":[],"source_ids":["example-data"],"storage":{"kind":"local","persistent":true},"write":{"mode":"single_writer"}},{"artifact":{"kind":"oci_image","sha256":"cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc","version":"0.5.1"},"backup":{"checkpoints":true,"retention_count":14},"health":{"timeout_seconds":30},"id":"production","kind":"service","network":{"exposure":"private","port":8443,"tls_required":true},"observability":{"logs":true,"metrics":true,"traces":false},"resources":{"cpu_millis":1000,"memory_bytes":2147483648},"secret_ids":["service-token"],"source_ids":["example-data"],"storage":{"capacity_bytes":10737418240,"kind":"volume","persistent":true},"write":{"mode":"queued_writer","queue_capacity":64}}]}
39 changes: 39 additions & 0 deletions docs/contracts/examples/graphforge-v1.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
schema_version: 1
project:
ontology: .graphforge/ontology
schemas: .graphforge/schemas
seeds: .graphforge/seeds
migrations: .graphforge/migrations
sources:
- id: example-data
uri: https://example.invalid/graphforge/example.parquet
sha256: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
media_type: application/vnd.apache.parquet
secrets:
- id: service-token
source: secret_manager
targets:
local:
kind: embedded
artifact:
kind: python_wheel
version: 0.5.0
sha256: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
write: { mode: single_writer }
storage: { kind: local, persistent: true }
source_ids: [example-data]
production:
kind: service
artifact:
kind: oci_image
version: 0.5.1
sha256: cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc
write: { mode: queued_writer, queue_capacity: 64 }
storage: { kind: volume, persistent: true, capacity_bytes: 10737418240 }
resources: { cpu_millis: 1000, memory_bytes: 2147483648 }
network: { exposure: private, port: 8443, tls_required: true }
health: { timeout_seconds: 30 }
observability: { logs: true, metrics: true, traces: false }
backup: { checkpoints: true, retention_count: 14 }
source_ids: [example-data]
secret_ids: [service-token]
Loading
Loading