Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,29 @@ journal. Provider, SKU, runner registration, cost approval, and teardown remain
explicit maintainer decisions; see the
[certification runbook](../../docs/development/g500-certification.md).

### `fly-tiny-qualification.yml` — Disposable Fly environment smoke

This protected manual workflow checks out the exact current `main` commit and
runs the non-ladder #958 qualification on a Blacksmith Linux runner. It builds
the commit-pinned image with hosted Docker, pushes it to the disposable app's
Fly registry path, resolves the immutable digest, runs only the 10 GiB
`performance-1x` filesystem smoke, and independently verifies teardown. A
separate recovery job has its own timeout budget after execution, and
`fly-tiny-recovery.yml` provides receipt-bound manual orphan cleanup. The
workflow uploads a one-day pre-creation ownership receipt plus sanitized
plan/result documents and qualified evidence. The receipt binds the commit to
a fresh 128-bit app-name nonce and never contains provider resource identifiers
or credentials.

### `fly-tiny-recovery.yml` — Manual Fly orphan recovery

This protected manual janitor downloads the one-day ownership receipt from a
specified qualification run and refuses deletion unless its exact nonce-bound
app and commit match. It runs the trusted controller from current `main` and
shares the qualification concurrency lock, so it cannot race the source run.
It exists for runner loss, workflow cancellation, or a failed independent
recovery job; it cannot infer ownership from an app prefix.

### CodeRabbit

CodeRabbit automatic review is disabled to preserve its limited quota. After
Expand Down
283 changes: 283 additions & 0 deletions .github/workflows/fly-tiny-qualification.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,283 @@
name: Fly tiny qualification

on:
workflow_dispatch:
inputs:
commit_sha:
description: Exact current main commit to qualify
required: true
type: string
organization:
description: Fly organization slug
required: true
default: personal
type: string
region:
description: Fixed admitted Fly region
required: true
default: dfw
type: string
confirm_disposable:
description: Authorize one tiny app, 10 GiB volume, and performance-1x Machine
required: true
default: false
type: boolean

permissions:
contents: read

concurrency:
group: fly-tiny-qualification
cancel-in-progress: false

env:
COMMIT_SHA: ${{ inputs.commit_sha }}
FLY_ORGANIZATION: ${{ inputs.organization }}
FLY_REGION: ${{ inputs.region }}
PREFLIGHT_ARTIFACT: fly-tiny-preflight-${{ github.run_id }}-${{ github.run_attempt }}

jobs:
prepare:
name: Admit and persist cleanup ownership
outputs:
app_name: ${{ steps.identity.outputs.app_name }}
machine_name: ${{ steps.identity.outputs.machine_name }}
volume_name: ${{ steps.identity.outputs.volume_name }}
runs-on: blacksmith-4vcpu-ubuntu-2404
environment: scale-certification
timeout-minutes: 20
steps:
- name: Require explicit authorization and credential
env:
CONFIRMED: ${{ inputs.confirm_disposable }}
FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
run: |
test "$CONFIRMED" = true
test -n "$FLY_API_TOKEN"
case "$COMMIT_SHA" in
(????????????????????????????????????????) ;;
(*) exit 2 ;;
esac
case "$COMMIT_SHA" in (*[!0-9a-f]*) exit 2 ;; esac

- name: Checkout exact commit
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ inputs.commit_sha }}
fetch-depth: 1

- name: Require exact current main
run: |
test "$(git rev-parse HEAD)" = "$COMMIT_SHA"
git fetch --no-tags --depth=1 origin main
test "$(git rev-parse FETCH_HEAD)" = "$COMMIT_SHA"

- name: Install uv
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
version: "0.10.9"
enable-cache: false

- name: Install flyctl
uses: superfly/flyctl-actions/setup-flyctl@ed8efb33836e8b2096c7fd3ba1c8afe303ebbff1 # v1.6

- name: Install locked benchmark environment
working-directory: benchmarks
run: uv sync --locked

- name: Create collision-resistant disposable identity
id: identity
run: |
python - "$GITHUB_OUTPUT" <<'PY'
from pathlib import Path
import secrets
import sys

nonce = secrets.token_hex(16)
with Path(sys.argv[1]).open("a", encoding="utf-8") as output:
output.write(f"app_name=gf-q958-{nonce}\n")
output.write(f"volume_name=gfq958_{nonce[:20]}\n")
output.write(f"machine_name=gf-q958-machine-{nonce[:16]}\n")
PY

- name: Run no-spend admission
env:
APP_NAME: ${{ steps.identity.outputs.app_name }}
FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
MACHINE_NAME: ${{ steps.identity.outputs.machine_name }}
VOLUME_NAME: ${{ steps.identity.outputs.volume_name }}
run: >-
PYTHONPATH=benchmarks/harness uv run --project benchmarks python -m
graphforge_bench.fly_tiny_qualification
--expected-sha "$COMMIT_SHA"
--org "$FLY_ORGANIZATION"
--app "$APP_NAME"
--region "$FLY_REGION"
--volume-name "$VOLUME_NAME"
--machine-name "$MACHINE_NAME"
--prerequisite-955 merged
--prerequisite-956 merged
--prerequisite-957 merged
--build-authority hosted-docker
--ledger "$RUNNER_TEMP/fly-q958-ledger.json"
--evidence-out "$RUNNER_TEMP/fly-q958-evidence.json"
--result-out "$RUNNER_TEMP/fly-q958-plan.json"

- name: Persist pre-creation ownership receipt
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ env.PREFLIGHT_ARTIFACT }}
path: |
${{ runner.temp }}/fly-q958-ledger.json
${{ runner.temp }}/fly-q958-plan.json
if-no-files-found: error
retention-days: 1

execute:
name: Hosted image and tiny Fly smoke
needs: prepare
runs-on: blacksmith-4vcpu-ubuntu-2404
environment: scale-certification
timeout-minutes: 55
env:
APP_NAME: ${{ needs.prepare.outputs.app_name }}
MACHINE_NAME: ${{ needs.prepare.outputs.machine_name }}
VOLUME_NAME: ${{ needs.prepare.outputs.volume_name }}
steps:
- name: Checkout exact commit
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ inputs.commit_sha }}
fetch-depth: 1

- name: Install uv
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
version: "0.10.9"
enable-cache: false

- name: Install flyctl
uses: superfly/flyctl-actions/setup-flyctl@ed8efb33836e8b2096c7fd3ba1c8afe303ebbff1 # v1.6

- name: Install locked benchmark environment
working-directory: benchmarks
run: uv sync --locked

- name: Restore pre-creation ownership receipt
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: ${{ env.PREFLIGHT_ARTIFACT }}
path: ${{ runner.temp }}

- name: Execute one tiny qualification
timeout-minutes: 50
env:
FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
DOCKER_CONFIG: ${{ runner.temp }}/fly-q958-docker
run: |
git fetch --no-tags --depth=1 origin main
test "$(git rev-parse HEAD)" = "$COMMIT_SHA"
test "$(git rev-parse FETCH_HEAD)" = "$COMMIT_SHA"
mkdir -p "$DOCKER_CONFIG"
PYTHONPATH=benchmarks/harness uv run --project benchmarks python -m \
graphforge_bench.fly_tiny_qualification \
--expected-sha "$COMMIT_SHA" \
--org "$FLY_ORGANIZATION" \
--app "$APP_NAME" \
--region "$FLY_REGION" \
--volume-name "$VOLUME_NAME" \
--machine-name "$MACHINE_NAME" \
--prerequisite-955 merged \
--prerequisite-956 merged \
--prerequisite-957 merged \
--build-authority hosted-docker \
--ledger "$RUNNER_TEMP/fly-q958-ledger.json" \
--evidence-out "$RUNNER_TEMP/fly-q958-evidence.json" \
--result-out "$RUNNER_TEMP/fly-q958-result.json" \
--execute \
--confirm-disposable

- name: Remove temporary Docker credentials
if: always()
env:
DOCKER_CONFIG: ${{ runner.temp }}/fly-q958-docker
run: rm -rf -- "$DOCKER_CONFIG"

- name: Upload sanitized execution result
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: fly-tiny-execution-${{ inputs.commit_sha }}
path: |
${{ runner.temp }}/fly-q958-result.json
${{ runner.temp }}/fly-q958-evidence.json
if-no-files-found: error
retention-days: 1

recover:
name: Independent orphan recovery
if: ${{ always() && needs.prepare.result == 'success' }}
needs: [prepare, execute]
runs-on: blacksmith-4vcpu-ubuntu-2404
environment: scale-certification
timeout-minutes: 30
env:
APP_NAME: ${{ needs.prepare.outputs.app_name }}
MACHINE_NAME: ${{ needs.prepare.outputs.machine_name }}
VOLUME_NAME: ${{ needs.prepare.outputs.volume_name }}
steps:
- name: Checkout exact commit
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ inputs.commit_sha }}
fetch-depth: 1

- name: Install uv
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
version: "0.10.9"
enable-cache: false

- name: Install flyctl
uses: superfly/flyctl-actions/setup-flyctl@ed8efb33836e8b2096c7fd3ba1c8afe303ebbff1 # v1.6

- name: Install locked benchmark environment
working-directory: benchmarks
run: uv sync --locked

- name: Restore pre-creation ownership receipt
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: ${{ env.PREFLIGHT_ARTIFACT }}
path: ${{ runner.temp }}

- name: Recover and verify teardown
env:
FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
run: >-
PYTHONPATH=benchmarks/harness uv run --project benchmarks python -m
graphforge_bench.fly_tiny_qualification
--expected-sha "$COMMIT_SHA"
--org "$FLY_ORGANIZATION"
--app "$APP_NAME"
--region "$FLY_REGION"
--volume-name "$VOLUME_NAME"
--machine-name "$MACHINE_NAME"
--prerequisite-955 merged
--prerequisite-956 merged
--prerequisite-957 merged
--build-authority hosted-docker
--ledger "$RUNNER_TEMP/fly-q958-ledger.json"
--evidence-out "$RUNNER_TEMP/fly-q958-evidence.json"
--result-out "$RUNNER_TEMP/fly-q958-cleanup-result.json"
--cleanup-only
--confirm-disposable

- name: Upload sanitized cleanup result
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: fly-tiny-cleanup-${{ inputs.commit_sha }}
path: ${{ runner.temp }}/fly-q958-cleanup-result.json
if-no-files-found: error
retention-days: 1
Loading