Skip to content

chore(selfhost): script git-backed self-host updates, ignore stray backups - #4151

Merged
JSONbored merged 1 commit into
mainfrom
chore/selfhost-git-deploy-hygiene-1660
Jul 8, 2026
Merged

chore(selfhost): script git-backed self-host updates, ignore stray backups#4151
JSONbored merged 1 commit into
mainfrom
chore/selfhost-git-deploy-hygiene-1660

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Follow-up to #1660's Git-backed conversion: the maintainer host deploy is already a clean checkout
tracking origin/main (done directly on the live host, outside this PR). Two gaps remained:

  • No general backup-file hygiene. .gitignore already ignores gittensory-config.backup-*/
    and .deploy-backups/ (specific, known backup locations), but nothing catches an arbitrary
    ad-hoc operator snapshot anywhere else in the tree — e.g. cp docker-compose.yml docker-compose.yml.bak-notes-20260707 before hand-editing something. That file sits untracked and
    dirties git status indefinitely, with nothing to stop it.
  • No scripted update flow. scripts/deploy-selfhost-prebuilt.sh (rebuild) and
    scripts/selfhost-post-update-check.sh (post-update health probe, docs(selfhost): define safe update and rollback flow #1823) already existed, but
    "pull upstream changes" itself was undocumented manual process, not a script: the docs showed a
    plain two-line git pull + ./scripts/deploy-selfhost-prebuilt.sh sequence with no guard — a
    diverged local history would let git pull silently create a merge commit (or fail confusingly),
    and a dirty tree would get rebuilt with whatever local edits happened to be present.

This PR closes both gaps:

  1. Two trailing .gitignore catch-alls, *.bak-* and *.backup-*, placed after the existing
    narrower patterns they generalize. Verified via git ls-files | grep -E '\.bak-|\.backup-'
    (empty) that nothing currently tracked matches either pattern before adding them, and pinned that
    check as a permanent regression test (see Validation).
  2. scripts/selfhost-update.sh — a thin wrapper: git fetchgit merge --ff-only
    deploy-selfhost-prebuilt.shselfhost-post-update-check.sh. It refuses to proceed, with a
    clear error and no side effects and no wrapped script invoked, on a dirty working tree, a
    checkout not on the expected branch (main by default, override with
    SELFHOST_UPDATE_BRANCH), or a non-fast-forward divergence — it never rebases, force-merges, or
    picks a side for the operator. SELFHOST_UPDATE_REMOTE and SELFHOST_SKIP_POST_UPDATE_CHECK are
    the two other overrides.
  3. Documented in the existing "Updating and rolling back" section of
    docs.self-hosting-operations.tsx (apps/gittensory-ui) alongside the manual two-step flow it
    wraps, plus an explicit note that .env, gittensory-config/, .deploy-backups/, and any
    *.local override files all already survive an update untouched (all already gitignored, none
    touched by this script).

Closes #1660.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (Closes #1660).

Validation

  • git diff --check
  • npm run actionlint (no workflow files touched; ran anyway as part of the full gate)
  • npm run typecheck — clean, twice (before and after the pre-push rebase)
  • npm run test:coverage locally — 584 test files passed / 2 skipped, 11915 tests passed / 7 skipped, unsharded. No src/**/packages/** lines changed (only .gitignore, one docs route, scripts/**, test/**), so there is no new Codecov codecov/patch obligation — ran it anyway to confirm nothing broke.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate — 0 vulnerabilities
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries — two new test files:
    • test/unit/selfhost-update-script.test.tsreal end-to-end execution of
      scripts/selfhost-update.sh (via spawnSync) against a throwaway local git remote, with
      deploy-selfhost-prebuilt.sh/selfhost-post-update-check.sh swapped for logging stubs: happy
      path (fetch → ff → rebuild → verify, HEAD lands on the new upstream tip), already-up-to-date
      no-op, SELFHOST_SKIP_POST_UPDATE_CHECK=1, rebuild-step failure stops before the health check,
      dirty tree / wrong branch / non-fast-forward divergence all refuse with no script invoked and no
      HEAD movement, a non-default branch via SELFHOST_UPDATE_BRANCH, a renamed remote via
      SELFHOST_UPDATE_REMOTE, and running outside a git checkout entirely. 10/10 passing.
    • test/unit/docs-selfhost-git-deploy-hygiene.test.ts — drift guard (matches the existing
      docs-selfhost-*.test.ts convention) asserting the .gitignore patterns exist and are ordered
      after the narrower ones they generalize, a real git ls-files check that nothing tracked
      matches either pattern (not just a static approximation), the script's safety invariants
      (never git push/reset --hard/--force/clean -f/merge --no-ff), and that the docs
      reference the script and its guarantees. 8/8 passing.
    • Also ran shellcheck scripts/selfhost-update.sh (clean) and manually exercised the script in an
      ad hoc sandbox before writing the permanent tests.

If any required check was skipped, explain why:

  • npm run test:engine-parity and npm run db:migrations:check / npm run db:schema-drift:check /
    npm run selfhost:env-reference:check / npm run selfhost:validate-observability /
    npm run cf-typegen:check aren't in this template's checklist but are part of npm run test:ci
    and all passed (test:ci ran green end-to-end twice: once before the pre-push rebase, once after,
    since the rebase picked up one same-file, different-section upstream commit — see Notes).
  • Did not run the gittensory-mcp pre-submit predictors (check_before_start,
    validate_linked_issue, check_slop_risk, lint_pr_text, predict_gate): those need an
    interactive GitHub device-flow login this session can't complete, and as the repo owner this PR is
    held for manual merge rather than auto-closed on an adverse gate signal.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. (N/A — no such changes; this is a deploy-hygiene script and its docs.)
  • API/OpenAPI/MCP behavior is updated and tested where needed. (N/A — no such changes; ui:openapi:check confirms no drift.)
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks. (N/A — static docs prose only, no data-fetching component touched.)
  • Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository. (No visual/layout change — new paragraphs and code blocks inside the existing "Updating and rolling back" section of an already-shipped docs page; see UI Evidence below.)
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

UI Evidence

Not applicable: this PR only adds prose paragraphs and <CodeBlock> snippets inside the existing
"Updating and rolling back" section of docs.self-hosting-operations.tsx — no new component,
layout, route, or styling. The rendered page structure (headings, Callout/FeatureRow/CodeBlock
primitives already in use throughout the page) is identical to what is already live; only the words
and code samples describing the update flow changed.

Notes

  • Rebased onto the current origin/main tip before pushing (git fetch origin && git rebase origin/main); one upstream commit (perf(selfhost): document retention/concurrency sizing, fix stale runner docs #4142, merged during this session) touched the same docs file
    but a different section (the runner CPU/memory table around line ~330-420, versus my edits around
    line ~870-995) — rebase applied with zero conflicts, and both npm run typecheck and the two new
    test files were re-run clean afterward.
  • Verified against source, not assumed: grepped scripts/*.sh and docker-compose.yml for any
    existing .bak/.backup usage before adding the ignore patterns (only scripts/backup.sh's
    unrelated sqlite3 ".backup" SQL command, not a filename pattern); confirmed the "real rebuild
    step" by grepping every scripts/*.sh for docker compose ... build rather than guessing
    (scripts/deploy-selfhost-prebuilt.sh is the only source-based rebuild path, already documented
    as "Path 2: build from the current git checkout").
  • scripts/** carries no Codecov codecov/patch obligation (only src/**/packages/** are
    measured), so the execution-based test suite above is extra rigor, not a coverage requirement.

…ckups (#1660)

The maintainer host deploy is now a clean checkout tracking origin/main, but
two gaps remained: nothing stopped an ad-hoc operator snapshot (e.g. `cp
file file.bak-notes-20260707`) from silently dirtying `git status`, and the
"pull upstream changes" flow was undocumented process rather than a script --
an operator had to remember to run git pull, then deploy-selfhost-prebuilt.sh,
in that order, with no guard against a diverged local history quietly
producing a merge commit.

Add trailing `*.bak-*`/`*.backup-*` catch-alls to .gitignore (verified via
`git ls-files` that nothing tracked matches either pattern), and add
scripts/selfhost-update.sh: a thin wrapper around `git fetch` + `git merge
--ff-only` + the existing deploy-selfhost-prebuilt.sh rebuild step +
selfhost-post-update-check.sh. It refuses to proceed -- with no side effects
and no script invoked -- on a dirty working tree, a checkout that isn't on
the expected branch, or a non-fast-forward divergence, so it never rebases,
force-merges, or picks a side on the operator's behalf.

Document the new script in the self-hosting operations docs (the existing
"Updating and rolling back" section) alongside the manual two-step flow it
wraps, and note that .env, gittensory-config/, .deploy-backups/, and any
*.local override files all already survive an update untouched.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
gittensory-ui d6e5c3f Commit Preview URL

Branch Preview URL
Jul 08 2026, 08:03 AM

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.72%. Comparing base (97f087b) to head (d6e5c3f).
⚠️ Report is 7 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4151   +/-   ##
=======================================
  Coverage   93.72%   93.72%           
=======================================
  Files         385      385           
  Lines       36243    36243           
  Branches    13281    13281           
=======================================
  Hits        33968    33968           
  Misses       1618     1618           
  Partials      657      657           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 8, 2026
@loopover-orb

loopover-orb Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-08 08:16:30 UTC

5 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • Touches a guarded path — held for manual review: This PR changes guardrail-protected path(s): apps/gittensory-ui/src/routes/docs.self-hosting-operations.tsx (matched apps/gittensory-ui/src/routes/**), scripts/selfhost-update.sh (matched scripts/**).

Review summary
This adds scripts/selfhost-update.sh, a fetch -> ff-only-merge -> rebuild -> health-check wrapper for Git-backed self-host operators, plus two trailing .gitignore catch-alls (*.bak-*, *.backup-*) for stray operator backup files, docs updates, and two new test files. The script correctly orders its guards (branch check, then dirty-tree check) before any fetch, so a rejected run truly has no side effects as claimed, and the ff-only merge genuinely refuses to silently create a merge commit on divergence. The end-to-end test suite spins up real git remotes/checkouts and stubs the wrapped scripts to verify call order and refusal paths, rather than asserting on script source text alone, which is the right way to cover this kind of shell wiring. I don't see a reachable-path correctness defect in the diff.

Nits — 7 non-blocking
  • scripts/selfhost-update.sh's ff-only merge failure message always says 'unpushed commits or a manual edit' even when the real cause is something else (e.g. REMOTE/BRANCH doesn't exist after a bad SELFHOST_UPDATE_BRANCH override) — consider checking `git rev-parse --verify $REMOTE/$BRANCH` first to give a more precise error.
  • test/unit/docs-selfhost-git-deploy-hygiene.test.ts's comment on the 'does not shadow any tracked file' test says a future match 'would silently untrack it the moment someone re-clones' — that's not how .gitignore works (it doesn't untrack already-tracked files on clone/checkout); the real risk is just that `git add -A`/`git add .` would silently skip the file going forward, so the comment should be corrected.
  • current_branch="$(git rev-parse --abbrev-ref HEAD)" returns the literal string 'HEAD' in a detached-HEAD checkout, so the error message reads 'currently on 'HEAD', expected 'main'' — correct but a one-line special case would read clearer to operators.
  • Per the external review brief, linkage to build(selfhost): make host deploys Git-backed and image-ready #1660 is 'partial' — worth an explicit line in the PR description confirming which specific open issue this PR (as opposed to the original build(selfhost): make host deploys Git-backed and image-ready #1660 work) is closing, per this repo's issue-linkage convention.
  • scripts/selfhost-update.sh: add a `git rev-parse --verify "$REMOTE/$BRANCH"` check right after fetch, and give a distinct error when the ref doesn't exist vs. when it exists but isn't a fast-forward.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #1660
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 52 registered-repo PR(s), 43 merged, 496 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 52 PR(s), 496 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Linked issue satisfaction

Partially addressed
The PR adds a scripted git fetch/ff-only/rebuild/verify flow, .gitignore backup catch-alls, and docs updates that address several requirements, but it explicitly defers the core deliverable of converting the host deploy to a clean Git-backed checkout to a prior out-of-band action ('done directly on the live host, outside this PR'), so this diff alone doesn't demonstrate the rollback snapshot, comp

Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 52 PR(s), 496 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 8, 2026
@JSONbored
JSONbored merged commit 274d6f3 into main Jul 8, 2026
13 checks passed
@JSONbored
JSONbored deleted the chore/selfhost-git-deploy-hygiene-1660 branch July 8, 2026 08:17
JSONbored added a commit that referenced this pull request Jul 8, 2026
…date script (#4158)

selfhost-update.sh's ff-only merge failure previously printed the same
"diverged history" message for two unrelated causes: a genuine
non-fast-forward divergence, and a bad SELFHOST_UPDATE_BRANCH/
SELFHOST_UPDATE_REMOTE override pointing at a ref that doesn't exist at all
(git merge --ff-only fails the same way for both). Add an explicit
`git rev-parse --verify` check right after the fetch so a missing ref gets
its own distinct, more actionable error instead of being misreported as a
divergence. Also give a checkout in a detached HEAD state (where
`git rev-parse --abbrev-ref HEAD` literally returns "HEAD") its own message
instead of the generic branch-mismatch one, which read oddly ("currently on
'HEAD', expected 'main'") and suggested a nonsensical
SELFHOST_UPDATE_BRANCH=HEAD override.

Also fix an inaccurate comment in
test/unit/docs-selfhost-git-deploy-hygiene.test.ts's tracked-file-shadowing
test: .gitignore has no effect on files git already tracks (verified: `git
add -A` still stages a modification to an already-tracked-but-now-ignored
file). The real risk the test guards against is the opposite direction -- a
future PR adding a genuinely new tracked file whose name happens to match
the pattern would have it silently excluded from `git status`'s untracked
list and from `git add -A`/`git add .`, not "untracked on reclone" as the
comment previously and incorrectly claimed.

Add test coverage for both new error paths (detached HEAD, and
SELFHOST_UPDATE_BRANCH naming a branch the remote doesn't have) to
selfhost-update-script.test.ts.

Both nits were flagged by the Gittensory Orb review on #4151 (merged).

Closes #4156
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

build(selfhost): make host deploys Git-backed and image-ready

1 participant