Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 12 additions & 6 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,11 @@ target, and produces one fail-closed aggregate report. Missing targets,
mixed SHAs or per-language versions, fallback execution, failed or unclassified
cases, and parity differences reject the candidate. This workflow does not
create a tag or release and does not publish to PyPI or npm.
Its final assembly derives one aligned root version, packs the complete PyPI,
npm, and crates.io surfaces, records four non-overlapping artifact groups, and
reopens every archive with `graphforge-release-candidate-v2` completeness
validation. A checksum-valid archive with missing entrypoints, types, native
modules, dependency metadata, or legal files is rejected.
After the maturin wheel build, the workflow verifies that any inherited Rust
compiler wrapper is still executable before Python contracts may launch Cargo;
an unavailable wrapper is cleared without printing the job environment or PATH.
Expand Down Expand Up @@ -135,11 +140,10 @@ construction issues; those close on outcomes (see `AGENTS.md` § Issue close).

### `binding-release-candidate.yml`, `release-credential-preflight.yml`, and `publish.yaml`

The exact-SHA Binding RC retains tested release bytes and their checksum record
for 30 days. Credential preflight verifies the npm/crates.io secret projections
without publishing. The release-event workflow consumes the retained candidate,
attaches its record, and publishes PyPI, npm, then crates.io in fail-closed order;
ordinary PRs do not repeat that certification.
The exact-SHA Binding RC retains tested release bytes and their partitioned v2
candidate manifest for 30 days. Credential preflight verifies the npm/crates.io
secret projections without publishing. The release-event workflow consumes the
retained candidate; ordinary PRs do not repeat that certification.

### `clean-env-verify.yml`

Expand All @@ -148,7 +152,9 @@ PyPI/npm only and runs the #167 lanes (pip quickstart, npm
smoke, NPX CLI and skills compatibility, create/close/reopen Arrow
rows, docs/package URL resolve, optional checksum match against a
`graphforge-release-record-v1` file). Preflight fails closed when the requested
version is unpublished. Ordinary PRs run only the harness unit tests via
version is unpublished. Candidate v2 manifests and historical
`graphforge-release-record-v1` files are both accepted for checksum lookup.
Ordinary PRs run only the harness unit tests via
Repository Policy — they never claim clean-env success against missing packages.
See [`docs/development/clean-environment-verification.md`](../../docs/development/clean-environment-verification.md).

Expand Down
43 changes: 33 additions & 10 deletions .github/workflows/binding-release-candidate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,25 @@ jobs:
[[ "$EVIDENCE_SHA" =~ ^[0-9a-f]{40}$ ]]
test "$(git rev-parse HEAD)" = "$EVIDENCE_SHA"

- name: Derive the one root release version
shell: bash
run: |
python3 scripts/set_release_version.py --check
release_version="$(python3 - <<'PY'
import re
from pathlib import Path
text = Path("Cargo.toml").read_text(encoding="utf-8")
match = re.search(r'(?m)^version = "([^"]+)"$', text)
if match is None:
raise SystemExit("workspace release version is missing")
print(match.group(1))
PY
)"
case "$release_version" in
*dev*) echo "release candidate requires a non-development version" >&2; exit 1 ;;
esac
printf 'RELEASE_VERSION=%s\n' "$release_version" >> "$GITHUB_ENV"

- uses: actions/setup-python@v6
with:
python-version: "3.13"
Expand Down Expand Up @@ -469,17 +488,19 @@ jobs:
python3 "$GITHUB_WORKSPACE/scripts/ci/validate-napi-artifacts.py" \
--npm-dir npm --manifest package.json
pnpm exec napi pre-publish -t npm --skip-optional-publish --no-gh-release
python3 "$GITHUB_WORKSPACE/scripts/ci/prepare-napi-packages.py" \
--npm-dir npm --legal-dir .
for package_dir in npm/*; do
npm pack "./$package_dir" --ignore-scripts \
--pack-destination "$GITHUB_WORKSPACE/candidate/release-artifacts/npm"
done
npm pack . --ignore-scripts \
--pack-destination "$GITHUB_WORKSPACE/candidate/release-artifacts/npm"
popd
npm pack ./packages/cli --ignore-scripts \
--pack-destination candidate/release-artifacts/npm
npm pack ./packages/agent-skills --ignore-scripts \
--pack-destination candidate/release-artifacts/npm
pnpm --dir packages/cli pack \
--pack-destination "$GITHUB_WORKSPACE/candidate/release-artifacts/npm"
pnpm --dir packages/agent-skills pack \
--pack-destination "$GITHUB_WORKSPACE/candidate/release-artifacts/npm"

- name: Package the complete Rust surface
shell: bash
Expand All @@ -494,7 +515,7 @@ jobs:
done < <(python3 scripts/ci/crate-publish-plan.py list)
cargo package "${package_args[@]}" --allow-dirty --no-verify
for crate in "${crates[@]}"; do
cp "target/release-candidate/package/${crate}-0.5.0.crate" \
cp "target/release-candidate/package/${crate}-${RELEASE_VERSION}.crate" \
candidate/release-artifacts/crates/
done

Expand All @@ -507,18 +528,20 @@ jobs:

- name: Create and validate the immutable checksum record
run: |
recorded_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
python3 scripts/record_release_artifacts.py \
--version 0.5.0 \
--version "$RELEASE_VERSION" \
--dist-dir candidate/release-artifacts \
--out candidate/v0.5.0-artifacts.json \
--out "candidate/v${RELEASE_VERSION}-artifacts.json" \
--recorded-at "$recorded_at" \
--notes "M1 Binding Release Candidate run $GITHUB_RUN_ID; exact SHA $EVIDENCE_SHA"
python3 scripts/ci/clean-env-verify.py validate-release-record \
candidate/v0.5.0-artifacts.json
"candidate/v${RELEASE_VERSION}-artifacts.json"
python3 scripts/ci/release-candidate.py validate \
--record candidate/v0.5.0-artifacts.json \
--record "candidate/v${RELEASE_VERSION}-artifacts.json" \
--artifacts-dir candidate/release-artifacts \
--expected-sha "$EVIDENCE_SHA" \
--version 0.5.0
--version "$RELEASE_VERSION"

- name: Retain the release candidate for publication
uses: actions/upload-artifact@v7
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ jobs:
run: |
python3 scripts/ci/test-release-publish-preflight.py
python3 scripts/ci/test-release-candidate.py
python3 scripts/ci/test-prepare-napi-packages.py
python3 scripts/ci/test-release-notes.py
python3 scripts/ci/test-publish-npm-artifacts.py
python3 scripts/ci/test-amend-npm-main-artifact.py
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

- Replace checksum-only release records with a deterministic, partitioned
candidate manifest that enforces the complete 24-node public package set,
one root version, exact dependency edges, archive entrypoints/legal files,
retention, and an explicit registry-independent publication state model
before any write (#293).
- Adopt ADR 0017's single-version release invariant: all public Rust crates,
Python and Node/native adapters, CLI, and agent skills must publish one exact
GraphForge version, and partial-publication recovery may not introduce
Expand Down
117 changes: 67 additions & 50 deletions docs/development/release-artifact-record.md
Original file line number Diff line number Diff line change
@@ -1,67 +1,84 @@
# Release artifact record
# Release candidate manifest

This page is the §5 checklist home for **checksums, SBOM/provenance, licenses, and
contents** of v0.5.0 release-candidate artifacts
([M1 #192](https://github.com/CurateLabs/graphforge/issues/192)).
GraphForge publication consumes one immutable, partitioned candidate. The
candidate manifest is the authority for release identity, package inventory,
dependency order, exact bytes, and retained-artifact availability. A matching
checksum proves byte identity; it does **not** prove that a package contains its
required runtime, metadata, and legal files.

It does **not** replace the authoritative publication order / stop conditions in
[`publication-order.md`](publication-order.md).
This page does not authorize publication or replace the operator stop conditions
in [`publication-order.md`](publication-order.md).

## Same-tagged-commit rule
## Canonical contract

Every first-party publishable artifact for version `0.5.0` must be built from one
verified commit (the eventual `v0.5.0` tag target) or have an explicit reproducible
link to that commit recorded in the artifact JSON. Do not mix bytes from different
commits under the same version.
`graphforge-release-candidate-v2` has one root `version` and no per-node version
field. The public node set is fixed:

## How to record
- 15 `graphforge-*` crates on crates.io;
- `graphforge` on PyPI (three tested wheels and one source distribution);
- five native npm packages and `@curatelabs/graphforge`;
- `@curatelabs/graphforge-cli` and
`@curatelabs/graphforge-agent-skills`.

1. Freeze the RC SHA and surface versions (`scripts/set_release_version.py` / #192).
2. Dispatch `Binding Release Candidate` for that exact current `main` SHA. Its
final job builds `M1-Release-Candidate-<sha>` from the tested wheels/addons,
then adds the sdist, npm tarballs, all 15 `.crate` archives, dry-run evidence,
and license reports.
3. The workflow runs the equivalent of:
Every archive records its byte length, SHA-256, SHA-256/SHA-512 SRI integrities,
package identity, required files, member count, and an inventory digest. Validation reopens the
exact archive and compares those facts. It rejects missing Python import/native
surfaces, Node entrypoints or types, native addons, CLI/skills entrypoints, crate
sources, legal files, or exact-version first-party dependency metadata—even when
the recorded checksum matches the incomplete archive.

```bash
python3 scripts/record_release_artifacts.py \
--version 0.5.0 \
--dist-dir path/to/artifacts \
--out docs/releases/records/v0.5.0-artifacts.json \
--notes "RC sha=<40-char> built via <workflow/run>"
```
The dependency graph includes crate-to-crate publication prerequisites, all five
native npm packages before the npm main package, main before CLI, and CLI before
agent skills. It must be complete, refer only to declared nodes, and be acyclic.

4. `publish.yaml` validates the complete bundle and attaches the JSON to the
GitHub Release before the first registry write (#194).
5. Post-release clean-env verification (#167) matches `sha256` values from this record.
## Artifact groups and retention

The generated document uses the same `graphforge-release-record-v1` schema
consumed by `clean-env-verify.py`. Validate it before attaching:
Candidate bytes are routed into four non-overlapping groups:

```bash
python3 scripts/ci/clean-env-verify.py validate-release-record \
docs/releases/records/v0.5.0-artifacts.json
```
| Group | Contents |
| --- | --- |
| `python` | Three tested wheels and one sdist |
| `npm` | Five native packages, main package, CLI, and agent skills |
| `crates` | All 15 `.crate` archives |
| `evidence` | Five tested Node addons plus dry-run and legal reports |

The small manifest lives beside those partitions. Each group declares its
retention period and expiry. Missing, expired, overlapping, unrecorded, or
wrongly routed files fail closed. Later recovery may download only a needed
partition, but it may never rebuild or substitute candidate bytes.

Template-only (no files yet):
## Publication states

The manifest names the release state vocabulary without deriving state from a
workflow job result: `not_attempted`, `absent`, `accepted_pending_visibility`,
`verified`, `conflict`, `indeterminate`, and `failed`. Registry observation and
recovery planning define how those states are reached; the candidate only fixes
their meanings and the bytes being observed.

## Build and validate offline

After the binding workflow has assembled the four directories, it creates the
manifest and immediately validates the complete candidate before any registry
write:

```bash
python3 scripts/record_release_artifacts.py \
--version 0.5.0 \
--dist-dir target/release-artifacts \
--allow-empty \
--out docs/releases/records/v0.5.0-artifacts.template.json
--version "$RELEASE_VERSION" \
--dist-dir candidate/release-artifacts \
--out "candidate/v${RELEASE_VERSION}-artifacts.json" \
--recorded-at "$RECORDED_AT"

python3 scripts/ci/release-candidate.py validate \
--record "candidate/v${RELEASE_VERSION}-artifacts.json" \
--artifacts-dir candidate/release-artifacts \
--expected-sha "$RELEASE_SHA" \
--version "$RELEASE_VERSION"
```

## License / third-party pointers

- First-party: `Apache-2.0`, shipped `LICENSE` + `NOTICE` (`make package-license-verify` / #218).
- Third-party inventory: [`legal/THIRD_PARTY_NOTICES.md`](../../legal/THIRD_PARTY_NOTICES.md) (#218).

## SBOM / provenance
The recorder produces stable JSON for the same version, SHA, timestamp, notes,
and exact partitions. The validator uses only local bytes; it performs no
registry access, tag creation, release creation, or publication.

When the release process emits SBOM or provenance files, place them in the same
`--dist-dir` so `record_release_artifacts.py` classifies them (`sbom` /
`provenance`). If none are configured for a surface, the record’s
`sbom_provenance.configured` stays false — that is an explicit disposition, not a
silent skip.
`clean-env-verify.py` continues to accept historical
`graphforge-release-record-v1` documents while also reading the v2 artifact list.
Historical v0.5.0 records remain immutable.
5 changes: 5 additions & 0 deletions docs/reference/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

- Replace checksum-only release records with a deterministic, partitioned
candidate manifest that enforces the complete 24-node public package set,
one root version, exact dependency edges, archive entrypoints/legal files,
retention, and an explicit registry-independent publication state model
before any write (#293).
- Adopt ADR 0017's single-version release invariant: all public Rust crates,
Python and Node/native adapters, CLI, and agent skills must publish one exact
GraphForge version, and partial-publication recovery may not introduce
Expand Down
13 changes: 9 additions & 4 deletions scripts/ci/clean-env-verify.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@
ROOT = Path(__file__).resolve().parents[2]
EVIDENCE_SCHEMA = "graphforge-clean-env-evidence-v1"
RELEASE_RECORD_SCHEMA = "graphforge-release-record-v1"
RELEASE_CANDIDATE_SCHEMA = "graphforge-release-candidate-v2"
RELEASE_RECORD_SCHEMAS = (RELEASE_RECORD_SCHEMA, RELEASE_CANDIDATE_SCHEMA)
DEFAULT_VERSION = "0.5.0"
DEFAULT_DOCS_BASE = "https://docs.graphforge.sh"
DEFAULT_CRATES = (
Expand Down Expand Up @@ -109,9 +111,10 @@ def parse_json(data: bytes, *, context: str) -> Any:


def validate_release_record(record: dict[str, Any]) -> dict[str, Any]:
if record.get("schema") != RELEASE_RECORD_SCHEMA:
if record.get("schema") not in RELEASE_RECORD_SCHEMAS:
raise VerifyError(
f"release record schema must be {RELEASE_RECORD_SCHEMA!r}, got {record.get('schema')!r}"
"release record schema must be one of "
f"{RELEASE_RECORD_SCHEMAS!r}, got {record.get('schema')!r}"
)
version = record.get("version")
if not isinstance(version, str) or not version:
Expand Down Expand Up @@ -897,7 +900,7 @@ def build_parser() -> argparse.ArgumentParser:
run.add_argument("--crate", action="append", default=[])
run.add_argument("--lane", action="append", choices=list(ALL_LANES))
run.add_argument("--all", action="store_true")
run.add_argument("--release-record", help=f"Path to {RELEASE_RECORD_SCHEMA} JSON")
run.add_argument("--release-record", help="Path to release record or candidate manifest JSON")
run.add_argument("--work", help="Work directory (default: temp dir)")
run.add_argument("--output", help="Write evidence JSON to this path")
run.add_argument(
Expand All @@ -917,7 +920,9 @@ def build_parser() -> argparse.ArgumentParser:
ve.add_argument("--require-ok", action="store_true")
ve.set_defaults(func=cmd_validate_evidence)

vr = sub.add_parser("validate-release-record", help=f"Validate {RELEASE_RECORD_SCHEMA}")
vr = sub.add_parser(
"validate-release-record", help="Validate a release record or candidate manifest"
)
vr.add_argument("path")
vr.set_defaults(func=cmd_validate_release_record)

Expand Down
47 changes: 47 additions & 0 deletions scripts/ci/prepare-napi-packages.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
#!/usr/bin/env python3
"""Add the required legal inventory to generated native npm packages."""

from __future__ import annotations

import argparse
import json
from pathlib import Path
import shutil

LEGAL_FILES = ("LICENSE", "NOTICE", "THIRD_PARTY_NOTICES.md")


def prepare(npm_dir: Path, legal_dir: Path) -> None:
package_dirs = sorted(path.parent for path in npm_dir.glob("*/package.json"))
if not package_dirs:
raise ValueError(f"no generated npm packages under {npm_dir}")
for source_name in LEGAL_FILES:
if not (legal_dir / source_name).is_file():
raise ValueError(f"legal source is missing: {legal_dir / source_name}")
for package_dir in package_dirs:
manifest_path = package_dir / "package.json"
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
files = manifest.get("files")
if not isinstance(files, list):
raise ValueError(f"{manifest_path} files must be an array")
for source_name in LEGAL_FILES:
shutil.copyfile(legal_dir / source_name, package_dir / source_name)
if source_name not in files:
files.append(source_name)
manifest["files"] = files
manifest_path.write_text(json.dumps(manifest, indent=2) + "\n", encoding="utf-8")


def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--npm-dir", type=Path, required=True)
parser.add_argument("--legal-dir", type=Path, required=True)
args = parser.parse_args()
try:
prepare(args.npm_dir, args.legal_dir)
except (OSError, ValueError, json.JSONDecodeError) as error:
raise SystemExit(f"prepare-napi-packages: {error}") from error


if __name__ == "__main__":
main()
Loading
Loading