Skip to content

feat(hub): publish Projects, Branches and Forks to a Hub with gf publish - #1756

Merged
DecisionNerd merged 26 commits into
mainfrom
feat/1749-gf-publish-hub
Oct 2, 2026
Merged

DecisionNerd merged 26 commits into
mainfrom
feat/1749-gf-publish-hub

Conversation

@DecisionNerd

@DecisionNerd DecisionNerd commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Provider-neutral Hub publishing and gf publish (#1749).

  • Wire contract (graphforge-hub-publish, ADR 0053, docs/reference/hub-publish/v1/ schemas + conformance corpus): capabilities document, publish sessions, resumable data-plane uploads (PUT with Content-Range, HEAD offset), atomic commit (replay check → every object verified → manifest validated → expected-revision precondition → ref advance → receipt), operation lookup for retries, RFC 8628 device flow, stable error codes (auth_denied, entitlement_denied, GF_IDEMPOTENCY_CONFLICT, ref_conflict, unsupported_future, integrity_failure, invalid_input).
  • ReferenceHub: in-memory reference implementation of the full publish and read mapping, used by the conformance corpus and the CLI tests.
  • gf publish <owner/repo> (--ref <branch> | --version-uuid <uuid>) [--fork-of <owner/repo>] [--operation-uuid <uuid>]: derives the Project package, research Version package, summary, ontology modules and registry-derived lineage client-side with the existing exporter and the same derivation as the Hub fixture generator; the Hub never re-derives semantics. Uploads go directly to Hub-issued data-plane URLs, which are treated as capabilities (the bearer token is sent only to the control plane, and the URLs are never logged). Retries are decided from a semantic intent digest (Version identity plus the committed Project generation the package is exported from) before any export; a new attempt of the same uncommitted intent supersedes its open session, so a rerun after ref_conflict lands on the newer revision. Ref advances carry an expected-revision precondition; a Fork is a new repository identity (create-if-absent) whose lineage carries the origin citation, verified against the origin repository's published lineage.
  • Credential: short-lived scoped token from the device flow (or GRAPHFORGE_HUB_PUBLISH_TOKEN for CI), held only in memory; never written to project files, config participants or logs.
  • HTTP plumbing shared by gf clone, gf ontology module fetch and gf publish moves to hub_http.rs with clone's network-safety rules unchanged.
  • Python and Node reach gf publish through the existing CLI shim; parity cases in tests/contracts/repository-cli-parity.json.

Closes #1749

Acceptance evidence (crates/graphforge-cli/src/hub_publish/tests.rs; real projects, real CLI path over HTTP to ReferenceHub, real gf clone back)

Criterion Test
Publishing the same Version twice yields one Hub version and the original receipt publishing_the_same_version_twice_returns_the_original_receipt (byte-equal receipts, one revision, zero data-plane PUTs on rerun; reused operation id with other content → GF_IDEMPOTENCY_CONFLICT)
An interrupted upload resumes, and a corrupt object is refused before the ref moves interrupted_upload_resumes_from_hub_offset (rerun resumes from the Hub's offset and clones back); corrupt_object_is_refused_before_ref_moves (in-transit and at-rest corruption → integrity_failure, refs unchanged, no receipt)
A published Fork clones back with its origin citation intact published_fork_clones_back_with_origin_citation

Supporting: publish_token_never_leaves_the_control_plane, stale_expected_revision_is_a_ref_conflict_and_a_rerun_lands_on_top, a_changed_project_publishes_a_new_snapshot_and_conflicts_under_a_reused_operation, an_object_corrupted_at_rest_must_be_uploaded_again, device_flow_waits_are_bounded_whatever_the_hub_advertises, a_slow_but_progressing_upload_succeeds_in_bounded_chunks, device_flow_publishes_after_pending_slow_down_and_approval, denied_device_authorization_is_auth_denied, quota_denial_is_entitlement_denied, upload_urls_are_validated_as_capabilities, repository_cli_parity_cases_match_the_rust_cli; the protocol crate replays every conformance case.

Test plan

  • cargo test -p graphforge-cli --lib (112 passed), cargo test -p graphforge-hub-publish
  • Mutations of replay, resume offset, commit digest check, Fork origin check, token routing, expected revision and device-flow handling, session supersession, Project identity binding and write bounds each fail their test
  • make check, make test-python, make test-node
  • CI Gate on PR head

🤖 Generated with Claude Code

…ry (#1748)

Wire the lineage document into discovery v1.1, verify per-Version packages,
derive lineage from the research registry for publishers, and extend gf clone
with --ref and --version-uuid for research heads and immutable Versions.

Co-authored-by: Cursor <cursoragent@cursor.com>
@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: CurateLabs/graphforge/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 90b0fb21-3bc2-4e67-bc33-5d8a0bfb7510

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

Warning

Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added core Core source code changes documentation Improvements or additions to documentation labels Oct 2, 2026
DecisionNerd and others added 6 commits October 2, 2026 18:58
Add lineage field to discovery test manifests, register new public methods
with test evidence, and exercise GraphForge lineage builder in unit tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
Update generator source digest after lineage manifest changes and align
the non-cypher surface gate inventory test with three new public methods.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
)

Introduce graphforge-hub-publish with transport trait, MemoryHub object
store, and idempotency error codes as the first slice toward gf publish.

Co-authored-by: Cursor <cursoragent@cursor.com>
Add ResumableUploadHub, MemoryPublishSession with ref preconditions,
and publish receipt idempotency tests toward Hub publish acceptance.

Co-authored-by: Cursor <cursoragent@cursor.com>
@DecisionNerd
DecisionNerd force-pushed the feat/1749-gf-publish-hub branch from 2809a7b to f8a6d81 Compare October 2, 2026 20:13
DecisionNerd and others added 7 commits October 2, 2026 20:22
# Conflicts:
#	crates/graphforge-cli/src/hub_fixture_artifacts.rs
#	tests/fixtures/hub/generated/v1/fixture.json
…ineage

Branch refs must target the lineage's repository snapshot; Proposals must
agree with their payload projection's source, package and listed Branch; a
projection cannot cite itself. Research packages follow the Project package
byte rules. Also restores CloneArgs fields dropped by the main merge.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#1749)

Replace the in-memory stubs with versioned graphforge-hub-publish/1 wire
documents (capabilities, session open/response, commit, receipt, upload
status, device flow), a request commitment bound at commit, and
ReferenceHub: one-lock handling of the full publish and clone read mapping
with RFC 8628 device-flow stub, scoped expiring bearer tokens, quota knob,
resumable Content-Range uploads with HEAD offsets, and an atomic commit
(replay, re-verify objects, manifest admission, expected-revision
precondition, ref advance, receipt).

Fixes from review of the draft: ref advance and receipt now commit
together; resumable offset query; declared lengths bounded at open; session
ids serialize; digests require canonical lowercase hex; serialized error
code for unsupported_future matches its wire token.

JSON Schemas and a conformance corpus under docs/reference/hub-publish/v1
are generated from and replayed by the crate's contract_artifacts test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…1749)

ADR 0053 gains the full HTTP mapping, commit order, request commitment,
credential handling (device flow, in-memory token, CI env var), and error
status table; it stays Proposed until gf publish ships. Add the
hub-publish v1 reference README, allow the directory in the docs tree
policy, and register graphforge-hub-publish in the license inventories,
crate publish plan test, RELEASING new-crate list, and its NOTICE copy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Research clones derive their import operation from the selected Version
  UUID and identity digest, so a Branch-head clone and an immutable-Version
  clone of one snapshot import as distinct generations; the Project-package
  clone keeps its historical derivation.
- verify_discovered_research_version requires the verified package's research
  registry to carry the selected Version with the lineage identity digest and
  kind before any destination exists; the clone also checks the imported
  package digest.
- The lineage builder derives Version kinds, Proposals and the Fork origin from
  the registry. Frozen Proposal payloads and other derivatives outside a
  Branch are projections even without a repacked graph projection.
- End-to-end tests publish a real Fork with two Branches and a Proposal,
  serve it, list it, and clone a Branch head, an immutable Version and the
  Proposal projection through gf clone; unknown required research
  capabilities fail before package reads.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ing rules

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the tooling Developer tooling and automation label Oct 2, 2026
DecisionNerd and others added 7 commits October 2, 2026 20:55
# Conflicts:
#	docs-site/astro.config.mjs
#	docs-site/scripts/sync-content.mjs
#	docs/adr/README.md
#	docs/engineering/adrs/README.md
…okup

A retry must be classified before the client derives any package, and the
request commitment binds package bytes and the expected revision, which a
retry after commit cannot reproduce. Session requests now carry a
client-defined intent_digest that the commitment binds, and an authorized
GET {repo}/.gf/publish/operations/{operation_uuid} returns the operation's
intent digest and, once committed, its original receipt.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The research lineage builder now emits exactly the Branches the publisher
names instead of refusing a Project with any unnamed Branch, so one Branch
can be published without publishing every local Branch. Naming a Branch the
registry does not hold still fails closed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
gf publish <owner/repo> (--ref <branch> | --version-uuid <uuid>)
[--fork-of <owner/repo>] [--operation-uuid <uuid>] derives one repository
snapshot client-side and publishes it over graphforge-hub-publish/1:

- Exports and verifies the Project package (data components), the selected
  Version's research package, the summary and ontology module packages, and
  the registry-derived lineage document; earlier published Versions,
  Branches, and Proposals carry forward unchanged.
- Classifies a retry by operation identity and intent digest before any
  export: the original receipt replays with zero uploads, and a different
  publication under the same identity fails GF_IDEMPOTENCY_CONFLICT.
- Uploads only what the Hub has not retained, resuming at the HEAD-reported
  offset; upload URLs are capabilities validated separately and never
  receive the token or appear in errors.
- Commits on the expected revision (create-if-absent for a new repository
  or Fork); a Fork verifies its cited origin Version is published with the
  same identity first.
- Obtains a scoped token from GRAPHFORGE_HUB_PUBLISH_TOKEN or the RFC 8628
  device flow (interactive terminals only); stable hub.publish.* codes.

Clone's HTTPS plumbing moves to hub_http.rs unchanged, and the manifest,
summary, and module derivation moves from the fixture generator to
hub_publication.rs, which both now use; only the fixture generator source
digest changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Nest the shared discovery derivation under hub_publish, gate the device
flow on a flag the streaming gf process sets instead of threading an
argument through run, and rebind the fixture generator source digest.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ADR 0053 is Accepted with the client snapshot rules (Data components Project
package, carried lineage, ref advance), the intent digest and operation
status, credential and error projection. The publish reference documents
gf publish and its stable codes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…o hub publish

Main's squash of the lineage discovery work matches the merged lineage tip
apart from the ontology module fetch (#1758) and the two binding re-pins.
Clone's HTTPS plumbing, including the new parse_input_at, now lives in
hub_http.rs and serves module_fetch.rs; clone and publish dispatch share one
lib.rs entry to stay within the source-size bound.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the testing Test coverage and testing infrastructure label Oct 2, 2026
@DecisionNerd DecisionNerd changed the title feat(hub): hub publish wire contract and MemoryHub (slice 1) feat(hub): publish Projects, Branches and Forks to a Hub with gf publish Oct 2, 2026
@DecisionNerd
DecisionNerd marked this pull request as ready for review October 2, 2026 22:03

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

DecisionNerd and others added 5 commits October 2, 2026 22:17
…stored objects

An operation is one content identity (its intent digest); a session is one
attempt at committing it. A new attempt of the same uncommitted intent, such as
a rerun after ref_conflict that read a newer revision, now supersedes the open
session instead of failing idempotency_conflict; retained bytes carry over by
digest. A committed operation replays its receipt and a different intent still
conflicts. Objects stored in the repository start complete only while they
still verify, so a copy corrupted at rest is uploaded again and replaced.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… paths

- The intent and default operation UUID bind the exported generation's
  manifest digest, so publishing after new data is committed is a new
  snapshot instead of a stale replay, and a reused --operation-uuid with a
  changed Project is GF_IDEMPOTENCY_CONFLICT.
- Device-flow polling caps the interval at 60 s and the wait at 15 minutes.
- Writes use per-phase timeouts with no whole-request deadline and 1 MiB
  chunks, so a slow but progressing upload completes; reads are unchanged.
- Failures are classified only from a parsed Hub error body.
- Tests: rerun after ref_conflict lands on the newer revision without
  re-sending bytes, a changed Project, bounded device-flow waits, a slow
  upload, a proxy status, and a Fork origin with a different identity.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…bounds

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…unds

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@DecisionNerd
DecisionNerd added this pull request to the merge queue Oct 2, 2026
Merged via the queue into main with commit 798beca Oct 2, 2026
19 checks passed
@DecisionNerd
DecisionNerd deleted the feat/1749-gf-publish-hub branch October 2, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core source code changes documentation Improvements or additions to documentation testing Test coverage and testing infrastructure tooling Developer tooling and automation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(cli): publish Projects, Branches and Forks to a Hub

1 participant