Skip to content

feat: cache and reuse wizard CI build artifacts #1747

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gewenyu99 merged 3 commits into from
Jun 12, 2026
Merged

feat: cache and reuse wizard CI build artifacts #1747

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
37c0a2a
feat: cache and reuse wizard CI build artifacts
gewenyu99 Jun 3, 2026
b9f4433
Fix some review items
gewenyu99 Jun 3, 2026
978b590
fix: resolve refs against fully qualified branch/tag names
gewenyu99 Jun 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
223 changes: 223 additions & 0 deletions .github/actions/setup-wizard-deps/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
name: 'Setup Wizard Dependencies'
description: 'Clone, build, and cache the wizard CLI, context-mill, and PostHog MCP. Restores each from a cache keyed on the dependency''s resolved commit SHA, building only on a miss. Prerequisite: the caller must have Node.js and pnpm already set up (this action shells out to pnpm/npm but does not install the toolchain).'

inputs:
wizard_ref:
description: 'Wizard repo branch/tag/sha'
required: true
context_mill_ref:
description: 'Context Mill repo branch/tag/sha'
required: true
posthog_ref:
description: 'PostHog repo branch/tag/sha (for MCP)'
required: true
app_token:
description: 'GitHub App token with access to the private warlock dependency'
required: true
save_cache:
description: 'Save freshly built dependencies back to the cache. Set on the warm-up job only; matrix jobs restore read-only.'
required: false
default: 'false'

runs:
using: 'composite'
steps:
- name: Resolve dependency SHAs
id: resolve
shell: bash
env:
WIZARD_REF: ${{ inputs.wizard_ref }}
CONTEXT_MILL_REF: ${{ inputs.context_mill_ref }}
POSTHOG_REF: ${{ inputs.posthog_ref }}
APP_TOKEN: ${{ inputs.app_token }}
run: |
# Resolve each ref to a concrete commit SHA so the cache key busts when a
# branch (e.g. main/master) moves. A ref that is already a full SHA is
# used verbatim.
resolve_sha() {
local repo="$1" ref="$2" sha
if printf '%s' "$ref" | grep -Eq '^[0-9a-f]{40}$'; then
printf '%s' "$ref"
return
fi
# Query the fully qualified refs only. A bare pattern like "master"
# also matches any branch whose tail is /master (e.g.
# refs/heads/codebeaver/master), which sorts first and wins.
sha=$(git ls-remote "https://x-access-token:${APP_TOKEN}@github.com/PostHog/${repo}.git" \
"refs/heads/$ref" "refs/tags/$ref" \
| awk 'NR==1{print $1}')
if [ -z "$sha" ]; then
echo "::error::Could not resolve ref '$ref' for PostHog/$repo to a commit SHA" >&2
return 1
fi
printf '%s' "$sha"
}
WIZARD_SHA=$(resolve_sha wizard "$WIZARD_REF")
CONTEXT_MILL_SHA=$(resolve_sha context-mill "$CONTEXT_MILL_REF")
POSTHOG_SHA=$(resolve_sha posthog "$POSTHOG_REF")
echo "wizard_sha=$WIZARD_SHA" >> "$GITHUB_OUTPUT"
echo "context_mill_sha=$CONTEXT_MILL_SHA" >> "$GITHUB_OUTPUT"
echo "posthog_sha=$POSTHOG_SHA" >> "$GITHUB_OUTPUT"
echo "Resolved SHAs:"
echo " wizard ($WIZARD_REF) -> $WIZARD_SHA"
echo " context-mill ($CONTEXT_MILL_REF) -> $CONTEXT_MILL_SHA"
echo " posthog ($POSTHOG_REF) -> $POSTHOG_SHA"

# ------------------------------------------------------------------ wizard
- name: Restore wizard cache
id: cache-wizard
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
with:
path: ~/wizard
key: wizard-${{ runner.os }}-${{ steps.resolve.outputs.wizard_sha }}

- name: Build wizard
if: steps.cache-wizard.outputs.cache-hit != 'true'
shell: bash
env:
WIZARD_SHA: ${{ steps.resolve.outputs.wizard_sha }}
run: |
echo "::group::Fetching wizard repo at $WIZARD_SHA"
# Fetch the exact resolved SHA (works for branch/tag/commit inputs alike,
# unlike `clone --branch` which rejects a commit SHA) and pins to the
# same commit the cache key is built from.
rm -rf "$HOME/wizard"
git init -q "$HOME/wizard"
cd "$HOME/wizard"
git remote add origin https://github.com/PostHog/wizard.git
git fetch --depth 1 origin "$WIZARD_SHA"
git checkout -q FETCH_HEAD
echo "::endgroup::"

echo "::group::Installing wizard dependencies"
pnpm install --frozen-lockfile
echo "::endgroup::"

echo "::group::Building wizard"
# build:ci inlines NODE_ENV=ci so the binary keeps --ci enabled
# (published builds disable it). Otherwise the --ci runs below fail
# with "Unknown argument: ci". Requires a wizard ref that has the
# build:ci script (PostHog/wizard#499 or later).
pnpm build:ci
echo "::endgroup::"

# Drop the .git dir before caching — not needed to run, just bloat.
rm -rf "$HOME/wizard/.git"

- name: Save wizard cache
if: inputs.save_cache == 'true' && steps.cache-wizard.outputs.cache-hit != 'true'
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
with:
path: ~/wizard
key: wizard-${{ runner.os }}-${{ steps.resolve.outputs.wizard_sha }}

- name: Export WIZARD_PATH
shell: bash
run: echo "WIZARD_PATH=$HOME/wizard" >> "$GITHUB_ENV"

# ------------------------------------------------------------ context-mill
- name: Restore context-mill cache
id: cache-context-mill
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
with:
path: ~/context-mill
key: context-mill-${{ runner.os }}-${{ steps.resolve.outputs.context_mill_sha }}

- name: Build context-mill
if: steps.cache-context-mill.outputs.cache-hit != 'true'
shell: bash
env:
CONTEXT_MILL_SHA: ${{ steps.resolve.outputs.context_mill_sha }}
APP_TOKEN: ${{ inputs.app_token }}
run: |
echo "::group::Configure git auth for private deps"
# context-mill depends on the private @posthog/warlock repo. Rewrite its
# clone URL to use the App token over HTTPS so the install can fetch it.
git config --global \
url."https://x-access-token:${APP_TOKEN}@github.com/PostHog/warlock".insteadOf \
"https://github.com/PostHog/warlock"
git config --global \
url."https://x-access-token:${APP_TOKEN}@github.com/PostHog/warlock".insteadOf \
"ssh://git@github.com/PostHog/warlock"
echo "::endgroup::"

echo "::group::Fetching context-mill repo at $CONTEXT_MILL_SHA"
rm -rf "$HOME/context-mill"
git init -q "$HOME/context-mill"
cd "$HOME/context-mill"
git remote add origin https://github.com/PostHog/context-mill.git
git fetch --depth 1 origin "$CONTEXT_MILL_SHA"
git checkout -q FETCH_HEAD
echo "::endgroup::"

echo "::group::Installing context-mill dependencies"
npm install
echo "::endgroup::"

echo "::group::Building context-mill MCP resources"
npm run build
echo "::endgroup::"

# Drop the .git dir before caching — not needed to run, just bloat.
rm -rf "$HOME/context-mill/.git"

- name: Save context-mill cache
if: inputs.save_cache == 'true' && steps.cache-context-mill.outputs.cache-hit != 'true'
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
with:
path: ~/context-mill
key: context-mill-${{ runner.os }}-${{ steps.resolve.outputs.context_mill_sha }}

- name: Export CONTEXT_MILL_PATH
shell: bash
run: echo "CONTEXT_MILL_PATH=$HOME/context-mill" >> "$GITHUB_ENV"

# ----------------------------------------------------------- posthog (MCP)
- name: Restore posthog cache
id: cache-posthog
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
with:
path: ~/posthog
key: posthog-${{ runner.os }}-${{ steps.resolve.outputs.posthog_sha }}

- name: Build PostHog MCP
if: steps.cache-posthog.outputs.cache-hit != 'true'
shell: bash
env:
POSTHOG_SHA: ${{ steps.resolve.outputs.posthog_sha }}
run: |
echo "::group::Fetching PostHog monorepo at $POSTHOG_SHA"
rm -rf "$HOME/posthog"
git init -q "$HOME/posthog"
cd "$HOME/posthog"
git remote add origin https://github.com/PostHog/posthog.git
git fetch --depth 1 origin "$POSTHOG_SHA"
git checkout -q FETCH_HEAD
echo "::endgroup::"

echo "::group::Installing PostCSS dependencies (required for UI apps build)"
pnpm add -Dw autoprefixer postcss-preset-env cssnano
echo "::endgroup::"

echo "::group::Installing MCP dependencies"
pnpm install --frozen-lockfile
echo "::endgroup::"

echo "::group::Building MCP UI apps"
cd "$HOME/posthog/services/mcp"
pnpm build
echo "::endgroup::"

# Drop the .git dir before caching — not needed to run, just bloat.
rm -rf "$HOME/posthog/.git"

- name: Save posthog cache
if: inputs.save_cache == 'true' && steps.cache-posthog.outputs.cache-hit != 'true'
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
with:
path: ~/posthog
key: posthog-${{ runner.os }}-${{ steps.resolve.outputs.posthog_sha }}

- name: Export MCP_PATH
shell: bash
run: echo "MCP_PATH=$HOME/posthog/services/mcp" >> "$GITHUB_ENV"
134 changes: 56 additions & 78 deletions .github/workflows/wizard-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -407,11 +407,54 @@ jobs:

console.log('Created initial PR comment with trigger ID: ' + triggerId);

# ============================================================================
# SETUP-DEPS: Build wizard, context-mill, and MCP once and warm the cache
# before the matrix fans out, so the parallel app jobs restore instead of
# each rebuilding the same artifacts.
# ============================================================================
setup-deps:
needs: discover
if: needs.discover.result == 'success'
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Generate GitHub App token
id: app-token
uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1.12.0
with:
app-id: ${{ secrets.GH_APP_POSTHOG_WIZARD_CI_BOT_APP_ID }}
private-key: ${{ secrets.GH_APP_POSTHOG_WIZARD_CI_BOT_PRIVATE_KEY }}
owner: PostHog
repositories: |
wizard-workbench
warlock

- name: Checkout repository
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1

- name: Setup Node.js
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: '24'

- name: Setup pnpm
uses: pnpm/action-setup@41ff72655975bd51cab0327fa583b6e92b6d3061

- name: Build and cache dependencies
uses: ./.github/actions/setup-wizard-deps
with:
wizard_ref: ${{ needs.discover.outputs.input_wizard_ref }}
context_mill_ref: ${{ needs.discover.outputs.input_context_mill_ref }}
posthog_ref: ${{ needs.discover.outputs.input_posthog_ref }}
app_token: ${{ steps.app-token.outputs.token }}
save_cache: 'true'

# ============================================================================
# WIZARD-CI: Run wizard on each app (matrix strategy)
# ============================================================================
wizard-ci:
needs: [discover, notify-start, notify-pr-start]
needs: [discover, notify-start, notify-pr-start, setup-deps]
if: always() && needs.discover.result == 'success'
runs-on: ${{ contains(matrix.app, 'swift/') && 'macos-latest' || 'ubuntu-latest' }}
timeout-minutes: 60
Expand Down Expand Up @@ -476,83 +519,18 @@ jobs:
exit 1
fi

- name: Clone and build Wizard CLI
run: |
echo "::group::Cloning wizard repo"
git clone --depth 1 --branch ${{ needs.discover.outputs.input_wizard_ref }} \
https://github.com/PostHog/wizard.git $HOME/wizard
echo "::endgroup::"

echo "::group::Installing wizard dependencies"
cd $HOME/wizard
pnpm install --frozen-lockfile
echo "::endgroup::"

echo "::group::Building wizard"
# build:ci inlines NODE_ENV=ci so the binary keeps --ci enabled
# (published builds disable it). Otherwise the --ci runs below fail
# with "Unknown argument: ci". Requires a wizard ref that has the
# build:ci script (PostHog/wizard#499 or later).
pnpm build:ci
echo "::endgroup::"

echo "WIZARD_PATH=$HOME/wizard" >> $GITHUB_ENV

- name: Clone and build Context Mill
run: |
echo "::group::Configure git auth for private deps"
# context-mill depends on the private @posthog/warlock repo. Rewrite its
# clone URL to use the App token over HTTPS so the install can fetch it.
git config --global \
url."https://x-access-token:${APP_TOKEN}@github.com/PostHog/warlock".insteadOf \
"https://github.com/PostHog/warlock"
git config --global \
url."https://x-access-token:${APP_TOKEN}@github.com/PostHog/warlock".insteadOf \
"ssh://git@github.com/PostHog/warlock"
echo "::endgroup::"

echo "::group::Cloning context-mill repo"
git clone --depth 1 --branch ${{ needs.discover.outputs.input_context_mill_ref }} \
https://github.com/PostHog/context-mill.git $HOME/context-mill
echo "::endgroup::"

echo "::group::Installing context-mill dependencies"
cd $HOME/context-mill
npm install
echo "::endgroup::"

echo "::group::Building context-mill MCP resources"
npm run build
echo "::endgroup::"

echo "CONTEXT_MILL_PATH=$HOME/context-mill" >> $GITHUB_ENV
env:
APP_TOKEN: ${{ steps.app-token.outputs.token }}

- name: Clone PostHog monorepo and install MCP
run: |
echo "::group::Cloning PostHog monorepo"
git clone --depth 1 --branch ${{ needs.discover.outputs.input_posthog_ref }} \
https://github.com/PostHog/posthog.git $HOME/posthog
echo "::endgroup::"

echo "::group::Installing PostCSS dependencies (required for UI apps build)"
cd $HOME/posthog
pnpm add -Dw autoprefixer postcss-preset-env cssnano
echo "::endgroup::"

echo "::group::Installing MCP dependencies"
cd $HOME/posthog
# Install packages.
pnpm install --frozen-lockfile
echo "::endgroup::"

echo "::group::Building MCP UI apps"
cd $HOME/posthog/services/mcp
pnpm build
echo "::endgroup::"

echo "MCP_PATH=$HOME/posthog/services/mcp" >> $GITHUB_ENV
- name: Setup wizard dependencies
# Restores wizard / context-mill / MCP from the cache warmed by the
# setup-deps job (keyed on each repo's resolved commit SHA), falling
# back to a clean build on a miss (e.g. macOS runners, cache eviction).
# Exports WIZARD_PATH / CONTEXT_MILL_PATH / MCP_PATH.
uses: ./.github/actions/setup-wizard-deps
with:
wizard_ref: ${{ needs.discover.outputs.input_wizard_ref }}
context_mill_ref: ${{ needs.discover.outputs.input_context_mill_ref }}
posthog_ref: ${{ needs.discover.outputs.input_posthog_ref }}
app_token: ${{ steps.app-token.outputs.token }}
save_cache: 'false'

- name: Configure Git
run: |
Expand Down
Loading