ci(vcpkg): Switch binary cache from local files to GitHub Packages NuGet feed

[bobtista]

Follow up for #2371 and #2515. Reverts #2544.

Replaces the local file-based vcpkg binary cache (actions/cache + files backend) that was fragile (janky and hand-rolled) with a NuGet feed hosted on GitHub Packages, and re-enables the win32-vcpkg CI presets that were disabled in #2544 because it kept on breaking every month or two.

Details

With NuGet binary caching, vcpkg uses its own ABI hashes directly as package identifiers in the feed. If a compiler update changes an ABI hash, vcpkg asks the feed for a package with the new hash, doesn't find it, builds once, uploads it, and every subsequent run restores it. There is no GHA cache key to get wrong anymore - This is the approach Microsoft's vcpkg docs recommend for GitHub Actions, and it works when I've tested it.

vcpkg's experimental x-gha backend was considered and rejected: it stores binaries in the Actions cache, which brings back the 10 GB per-repo limit and eviction churn for large packages like ffmpeg.

Changes

• Replace VCPKG_BINARY_SOURCES from files backend to nuget backend pointing at GitHub Packages
• Add VCPKG_NUGET_REPOSITORY for GitHub Packages package-repository association
• Add NuGet source authentication step using GITHUB_TOKEN
• Add packages: write permission to ci.yml and build-toolchain.yml
• Scope vcpkg setup steps to vcpkg presets (contains(inputs.preset, 'vcpkg') instead of startsWith(inputs.preset, 'win32'))
• Remove cache key computation, actions/cache/restore, actions/cache/save, local cache directory setup
• Re-enable the win32-vcpkg presets in the CI matrix

Notes

• GITHUB_TOKEN is read-only on PRs from forks, so fork PRs restore from the feed but cannot upload. A fork PR that changes an ABI hash (e.g. baseline bump) builds cold on each run until a same-repo build populates the new hash. Existing packages cannot be overwritten — the feed rejects re-pushing an existing version.
• Old package versions accumulate as ABI hashes change; a periodic cleanup workflow can be added later if feed size becomes a concern.

Test plan

• Tested on fork (bobtista/GeneralsGameCode) — cold build uploaded packages successfully
• Second run restored 7 packages from NuGet in 3.7s, ffmpeg installed in 28.8ms
• Total job time dropped from ~24 min to ~7 min on cache hit
• First CI run after merge will be a cold build (~24 min), subsequent runs should restore from feed

[greptile-apps]

Greptile Summary

This PR replaces the fragile file-based vcpkg binary cache (actions/cache + files backend) with a NuGet feed hosted on GitHub Packages, eliminating the GHA cache-key/ABI-hash synchronization problem that caused full rebuilds whenever the two systems drifted. It also re-enables the three win32-vcpkg* build matrix entries that had been commented out.

• build-toolchain.yml: Removes cache-key computation, actions/cache/restore, actions/cache/save, and the local cache-directory setup; replaces them with a single NuGet source authentication step (nuget sources add + setapikey using GITHUB_TOKEN) that runs only for vcpkg presets. VCPKG_BINARY_SOURCES is now set to the GitHub Packages NuGet feed at job level.
• ci.yml: Adds packages: write to the top-level permissions block and uncomments the win32-vcpkg, win32-vcpkg-profile, and win32-vcpkg-debug matrix entries in both the build-generals and build-generalsmd-win32 jobs.

Confidence Score: 5/5

Safe to merge — a well-scoped CI infrastructure change with no application code affected.

The change is a clean swap of one caching backend for another, following the canonical vcpkg + GitHub Packages NuGet pattern. The NuGet authentication step correctly fetches the nuget CLI bundled with vcpkg, gates both the setup and credential steps on contains(inputs.preset, 'vcpkg'), and uses the ephemeral GITHUB_TOKEN (masked in logs). The packages: write permission is correctly added to both the caller (ci.yml) and callee (build-toolchain.yml) workflows, and the previously-commented-out vcpkg matrix entries are re-enabled in lockstep with the new cache infrastructure.

No files require special attention; both workflow files are straightforward and internally consistent.

Important Files Changed

Filename	Overview
.github/workflows/build-toolchain.yml	Replaces file-based vcpkg binary cache with NuGet/GitHub Packages; removes cache-key computation, restore, and save steps; adds conditional NuGet source authentication; no logical issues found.
.github/workflows/ci.yml	Adds packages:write permission and re-enables the three previously-commented-out win32-vcpkg matrix entries in both build-generals and build-generalsmd-win32 jobs.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant CI as ci.yml (matrix job)
    participant BT as build-toolchain.yml
    participant vcpkg as vcpkg.exe
    participant NuGet as GitHub Packages NuGet
    participant CMake as CMake Build

    CI->>BT: workflow_call (preset, game, tools, extras)
    BT->>BT: "Set env VCPKG_BINARY_SOURCES=nuget feed"

    alt contains(preset, 'vcpkg')
        BT->>vcpkg: lukka/run-vcpkg (runVcpkgInstall: false)
        vcpkg-->>BT: sets VCPKG_ROOT
        BT->>vcpkg: fetch nuget → returns nuget.exe path
        BT->>NuGet: nuget sources add (GitHubPackages, GITHUB_TOKEN)
        BT->>NuGet: nuget setapikey (GITHUB_TOKEN)
        BT->>BT: Set VCPKG_OVERLAY_TRIPLETS, VCPKG_INSTALL_OPTIONS
    end

    BT->>CMake: cmake --preset (configure)
    CMake->>vcpkg: install dependencies
    vcpkg->>NuGet: check binary cache (ABI hash as package id)
    alt Cache hit
        NuGet-->>vcpkg: restore package (fast)
    else Cache miss
        vcpkg->>vcpkg: build from source
        vcpkg->>NuGet: upload package (push)
    end
    CMake->>CMake: cmake --build
    BT-->>CI: artifacts

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant CI as ci.yml (matrix job)
    participant BT as build-toolchain.yml
    participant vcpkg as vcpkg.exe
    participant NuGet as GitHub Packages NuGet
    participant CMake as CMake Build

    CI->>BT: workflow_call (preset, game, tools, extras)
    BT->>BT: "Set env VCPKG_BINARY_SOURCES=nuget feed"

    alt contains(preset, 'vcpkg')
        BT->>vcpkg: lukka/run-vcpkg (runVcpkgInstall: false)
        vcpkg-->>BT: sets VCPKG_ROOT
        BT->>vcpkg: fetch nuget → returns nuget.exe path
        BT->>NuGet: nuget sources add (GitHubPackages, GITHUB_TOKEN)
        BT->>NuGet: nuget setapikey (GITHUB_TOKEN)
        BT->>BT: Set VCPKG_OVERLAY_TRIPLETS, VCPKG_INSTALL_OPTIONS
    end

    BT->>CMake: cmake --preset (configure)
    CMake->>vcpkg: install dependencies
    vcpkg->>NuGet: check binary cache (ABI hash as package id)
    alt Cache hit
        NuGet-->>vcpkg: restore package (fast)
    else Cache miss
        vcpkg->>vcpkg: build from source
        vcpkg->>NuGet: upload package (push)
    end
    CMake->>CMake: cmake --build
    BT-->>CI: artifacts

Loading

_{Reviews (4): Last reviewed commit: "ci: Re-enable win32 vcpkg presets" | Re-trigger Greptile}

[bobtista]

Greptile Summary

This PR replaces the fragile actions/cache + files vcpkg binary cache with a GitHub Packages NuGet feed, eliminating the GHA cache-key / vcpkg ABI-hash synchronization doom loop described in the PR description. The structural change is sound and fork-tested.

• build-toolchain.yml: VCPKG_BINARY_SOURCES now points at the NuGet feed; a new step fetches vcpkg's bundled NuGet binary and registers the GitHub Packages source with GITHUB_TOKEN credentials; the old cache-key computation, actions/cache/restore, directory setup, and actions/cache/save steps are all removed.
• ci.yml: packages: write added so the GITHUB_TOKEN issued to calling jobs has sufficient scope to push packages to GitHub Packages (required because GitHub Actions intersects caller and callee permissions for reusable workflows).
• Minor: nuget sources add is not idempotent — a step retry or self-hosted runner reuse would cause a "Source already exists" failure. Replacing with an existence check + update fallback would make the step more resilient.

Confidence Score: 4/5

Safe to merge; the NuGet-based caching approach is sound and fork-tested, with only a minor idempotency edge case on step retries.

The structural approach is correct and well-understood (vcpkg + GitHub Packages NuGet is a documented pattern). packages:write is correctly added to both caller and callee workflows. The only concern is nuget sources add failing on retry or self-hosted runners, which is low-probability on ephemeral GitHub-hosted runners. The first post-merge run will be a cold build but subsequent runs should be fast.

.github/workflows/build-toolchain.yml — specifically the NuGet auth step around lines 119-128 for the idempotency concern.

Important Files Changed

Filename Overview
.github/workflows/build-toolchain.yml Replaces files-based vcpkg binary cache with GitHub Packages NuGet feed; adds packages:write permission, NuGet auth step via vcpkg-bundled nuget binary, and removes old cache key/restore/save steps — minor idempotency concern on nuget sources add
.github/workflows/ci.yml Adds packages:write permission to the caller workflow so the GITHUB_TOKEN can write to GitHub Packages when delegated through the reusable build-toolchain.yml

Sequence Diagram

sequenceDiagram
    participant GHA as GitHub Actions
    participant vRun as lukka/run-vcpkg
    participant vcpkg as vcpkg.exe
    participant NuGet as NuGet CLI
    participant GPkg as GitHub Packages

    GHA->>vRun: Setup vcpkg (doNotCache:true)
    vRun-->>GHA: VCPKG_ROOT set
    GHA->>vcpkg: fetch nuget
    vcpkg-->>GHA: path/to/nuget.exe
    GHA->>NuGet: sources add GitHubPackages (GITHUB_TOKEN)
    NuGet-->>GHA: source registered
    GHA->>NuGet: setapikey (GITHUB_TOKEN)
    NuGet-->>GHA: API key stored
    GHA->>vcpkg: cmake --preset (triggers vcpkg install)
    vcpkg->>GPkg: GET package@{abi-hash}
    alt cache hit
        GPkg-->>vcpkg: package restored (e.g. 28.8ms for ffmpeg)
    else cache miss
        GPkg-->>vcpkg: 404 Not Found
        vcpkg->>vcpkg: build from source
        vcpkg->>GPkg: PUT package@{abi-hash}
    end

Loading

Loading
Prompt To Fix All With AI

This is a comment left during a code review.
Path: .github/workflows/build-toolchain.yml
Line: 120-127

Comment:
**`nuget sources add` is not idempotent**

If this step is retried (e.g., via GitHub Actions "Re-run failed jobs") or runs on a self-hosted runner where the `GitHubPackages` source was previously registered, `nuget sources add` will exit with "Source already exists" and fail the step. For ephemeral GitHub-hosted runners this is fine, but wrapping it in an existence check would make the step resilient to retries:

$src = "https://nuget.pkg.github.com/${{ github.repository_owner }}/index.json"
$exists = (& $nuget sources list) -match "GitHubPackages"
if ($exists) {
& $nuget sources update -Name GitHubPackages -Source $src -StorePasswordInClearText
-UserName "${{ github.repository_owner }}" -Password "${{ secrets.GITHUB_TOKEN }}" } else { & $nuget sources add -Name GitHubPackages -Source $src
-StorePasswordInClearText -UserName "${{ github.repository_owner }}"
-Password "${{ secrets.GITHUB_TOKEN }}"
}
& $nuget setapikey "${{ secrets.GITHUB_TOKEN }}" -Source $src

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "ci(vcpkg): Switch binary cache from loca..." | Re-trigger Greptile

We use ephemeral GitHub-hosted runners, do we need to worry about this? Every job starts with a clean NuGet config

[xezon]

What is now the strategy with vcpkg? We still have it disabled because it caused long CI build times every now and then.

[bobtista]

What is now the strategy with vcpkg? We still have it disabled because it caused long CI build times every now and then.

The strategy is to move the dependency build output into the GitHub Packages NuGet binary cache instead of continuing with handmade caching tweaks trying to use actions/cache as a big shared folder cache. We should have used nuget from the beginning - Microsoft’s vcpkg docs specifically recommend the NuGet provider for this use case.

I tested this on my fork. The first GenCI run still took about 24 minutes for the vcpkg jobs, which I think is expected because it had to build/populate the cache. Then I ran the same workflow again on the same commit, and the vcpkg jobs dropped to about 5.5-8 minutes. The CMake configure step was only around 30-45 seconds on the second run, so ffmpeg was restored from the cache instead of rebuilt.

The workflow is still red on my fork because the replay checks need R2 secrets I don’t have set up, but all six win32-vcpkg build variants passed on both runs.

[bobtista]

Rebased onto main and updated the PR, updated the description here too - this is ready for review and merge IMHO