Skip to content

CI: restore the pinned Db2 image from actions/cache, not icr.io per shard (Fixes #5262) - #5265

Merged
squid-protocol merged 3 commits into
mainfrom
churn/5262-20261016T1408Z
Oct 16, 2026
Merged

squid-protocol merged 3 commits into
mainfrom
churn/5262-20261016T1408Z

Conversation

@squid-protocol

@squid-protocol squid-protocol commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

Fixes #5262.

What

Every db2 N/4 shard (det-sweep) and db2-part (evidence-refresh) pulled the multi-GB pinned image from icr.io; outages outlast #5048's 4-attempt retry. Now the image comes from actions/cache, keyed db2-image-<pinned digest>.

  • tests/tools/db2_image_cache.py (stdlib only): key, load DIR, save DIR. load does zstd -d | docker load, then checks the loaded image id against the id recorded at save time (taken from the image pulled by the pinned digest), and tags a local alias. A miss, a failed load or an id mismatch exits 1 and the workflow runs the existing ci_retry.sh 4 15 -- docker pull of the digest ref.
  • equivalence_db2.py: IMAGE_ALIAS + run_ref(). docker load drops the digest name (checked with a real docker save/load round trip: only "Loaded image ID" comes back, the digest ref no longer resolves; a digest cannot be a local tag), so the harness runs from the alias only when the digest ref is absent. pull_image treats the alias as present; _check_container_image compares against the same ref. The pin (IMAGE) is unchanged.
  • Workflow changes (owner-approved for this item):
    • det-sweep.yml db2 job: name the key, actions/cache/restore, background step is load || ci_retry pull, then docker save + actions/cache/save guarded by miss, not pull_request, refs/heads/main, shard 1/N (single writer; det-sweep's db2 job runs on PRs and dispatch, so this writer only fires on a dispatch on main).
    • evidence-refresh.yml db2-part: same restore / load-or-pull; save from part 1/4. This job runs on main's nightly schedule, so it is what warms the cache for PRs. PRs restore main's entry (cache scope) and never write their own copy.

Workflow edits: owner approval

.github/workflows/det-sweep.yml and evidence-refresh.yml are edited on purpose: the workflow change is the fix. Issue #5262 states "Workflow edits approved by the owner (2026-10-16, this request; #5048's 2026-10-14 approval covers this class)", and the worker brief carried allow_workflows for this item. If the night-shift queue item lacks that flag, the owner must confirm before merge; the code half (db2_image_cache.py, equivalence_db2.py alias) is inert without the workflow half.

Class

CI network fetches that fail as infra flakes (#5048 family). Covered: both workflows that start the Db2 container (det-sweep db2, evidence-refresh db2-part incl. records). Not covered: Muninn's Docker Hub pulls (see sibling gaps in the report).

Measurements

Before (runs 38830516356 / 38823577266, 2026-10-15/16, successful shards): "Wait for the background Db2 start" took 2m50s-3m45s (pull overlapped with setup, plus Db2 start); on the six failing runs the step failed after the 4 pull attempts. After: not yet measurable, the cache is empty until the first main/nightly run saves it; I will add the warm-cache numbers (restore + load seconds, compressed size, "cache: hit" line, no icr.io request) after the first dispatch on this branch. Cache budget caveat: the repo's caches total ~11.2 GB already (over the 10 GB limit, LRU eviction), so a multi-GB entry can evict others or itself be evicted; size is reported by the save step ("saved ..., N MB compressed").

Tests

tests/cobol_mainframe/test_db2_image_cache.py (10): key = digest, restore everywhere / save guarded to main, miss falls back to a pull, id mismatch refused, already-present image needs no cache, alias used by the harness. Also green: test_db2_odbc_pin, test_db2_image_pin, test_ci_retry. pr_gates --fast 6/6. No golden change (no parsing code).

Cross-repo: none.

🤖 Generated with Claude Code

…hard (Fixes #5262)

Cache key = the pinned digest; restore everywhere, save from main only (one shard, on a miss).
A miss or an image-id mismatch falls back to the existing retried, digest-pinned pull.
The harness runs a cache-loaded image by a local alias (a digest cannot be a tag).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@squid-protocol squid-protocol added the shepherd:merge Maintainer: merge when green (auto-merge, squash). Guide: docs/ci.md label Oct 16, 2026
@squid-protocol

Copy link
Copy Markdown
Owner Author

No description provided.

@squid-protocol
squid-protocol marked this pull request as ready for review October 16, 2026 14:17
@squid-protocol

Copy link
Copy Markdown
Owner Author

No description provided.

@squid-protocol
squid-protocol enabled auto-merge (squash) October 16, 2026 14:28
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

@squid-protocol squid-protocol added the shepherd:retrying Bot: an infra/flake rerun is pending on this head. Guide: docs/ci.md label Oct 16, 2026
@squid-protocol

Copy link
Copy Markdown
Owner Author

PR #5265 @ d39056a: 2 failed check(s)

== full-suite part 3/3 (failure) -- real
https://github.com/squid-protocol/gitgalaxy/actions/runs/38914601969/job/117023735856
repro: python -m pytest -q tests/test_source_reads.py::test_no_source_read_loses_a_byte
failing tests: tests/test_source_reads.py::test_no_source_read_loses_a_byte
errors:
fail-on-cache-miss: false
E AssertionError: #3813: decode through gitgalaxy.core.source_text (read_source / open_source / decode_bytes) instead of a lossy errors= handler or a text-mode read of an estate file:
tests/test_source_reads.py:146: AssertionError
FAILED tests/test_source_reads.py::test_no_source_read_loses_a_byte - AssertionError: #3813: decode through gitgalaxy.core.source_text (read_source / open_source / decode_bytes) instead of a lossy errors= handler or a text-mode read of an e
##[error]Process completed with exit code 1.
log tail:
PYTHONUTF8: 1
GITGALAXY_LICENSE_KEY: COMMUNITY_FREE_TIER
OMP_NUM_THREADS: 1
pythonLocation: /opt/hostedtoolcache/Python/3.12.15/x64
PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.12.15/x64/lib/pkgconfig
Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.15/x64
Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.15/x64
Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.15/x64
LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.15/x64/lib
PYTEST_ADDOPTS: --ignore=tests/cobol_mainframe
##[endgroup]
With the provided path, there will be 1 file uploaded
Artifact name is valid!
Root directory input is valid!
Uploading artifact: test-times-part-3.zip
Beginning upload of artifact content to blob storage
Uploaded bytes 170117
Finished uploading artifact content to blob storage!
SHA256 digest of uploaded artifact is 75d2a374926bba8a306bf24059388760655a9dfdbb1c078a102a1c499355ce5f
Finalizing artifact upload
Artifact test-times-part-3 successfully finalized. Artifact ID 12054115856
Artifact test-times-part-3 has been successfully uploaded! Final size is 170117 bytes. Artifact ID is 12054115856
Artifact download URL: https://github.com/squid-protocol/gitgalaxy/actions/runs/38914601969/artifacts/12054115856
Post job cleanup.
[command]/usr/bin/git version
git version 2.55.0
Temporarily overriding HOME='/home/runner/work/_temp/f6a293d4-b577-41d2-86d3-b132d8086612' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/usr/bin/git config --global --add safe.directory /home/runner/work/gitgalaxy/gitgalaxy
Removing SSH command configuration
[command]/usr/bin/git config --local --name-only --get-regexp core.sshCommand
[command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
Removing HTTP extra header
[command]/usr/bin/git config --local --name-only --get-regexp http.https://github.com/.extraheader
[command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http.https://github.com/.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
Removing includeIf entries pointing to credentials config files
[command]/usr/bin/git config --local --name-only --get-regexp ^includeIf.gitdir:
[command]/usr/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
Cleaning up orphan processes
Terminate orphan process: pid (10234) (sleep)

== muninn (failure) -- infra
https://github.com/squid-protocol/gitgalaxy/actions/runs/38914602227/job/117018727152
repro: no local equivalent; read the check page
errors:
fail-on: info
muninn: scanning . (formats=sarif,comment, fail-on=info)
log tail:
muninn: wrote muninn.sarif
muninn: found 1 finding(s) at or above "info" severity
##[group]Run github/codeql-action@24c5418
with:
sarif_file: muninn.sarif
checkout_path: /home/runner/work/gitgalaxy/gitgalaxy
token: ***
matrix: null
wait-for-processing: true
##[endgroup]
Job run UUID is 3c572fbe-259f-4ce7-a37e-383963e49f71.
Post-processing sarif files: ["muninn.sarif"]
Validating muninn.sarif
Adding fingerprints to SARIF file. See https://docs.github.com/en/code-security/reference/code-scanning/sarif-support-for-code-scanning#data-for-preventing-duplicated-alerts for more information.
##[group]Uploading code scanning results
Uploading results
Successfully uploaded results
##[endgroup]
##[group]Waiting for processing to finish
Analysis upload status is complete.
##[endgroup]
Post job cleanup.
##[group]Uploading combined SARIF debug artifact
##[endgroup]
Post job cleanup.
[command]/usr/bin/git version
git version 2.55.0
Temporarily overriding HOME='/home/runner/work/_temp/571bb22e-a884-40fb-8a1e-f35e9accdabb' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/usr/bin/git config --global --add safe.directory /home/runner/work/gitgalaxy/gitgalaxy
Removing SSH command configuration
[command]/usr/bin/git config --local --name-only --get-regexp core.sshCommand
[command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
Removing HTTP extra header
[command]/usr/bin/git config --local --name-only --get-regexp http.https://github.com/.extraheader
[command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http.https://github.com/.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
Removing includeIf entries pointing to credentials config files
[command]/usr/bin/git config --local --name-only --get-regexp ^includeIf.gitdir:
[command]/usr/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
Cleaning up orphan processes

@squid-protocol squid-protocol added the shepherd:needs-fix Bot: a real failure on this head; see the digest comment. Guide: docs/ci.md label Oct 16, 2026
squid-protocol and others added 2 commits October 16, 2026 10:51
…s the .test_durations PEM-header-in-a-test-id gitleaks hit (Part of #5262)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@squid-protocol squid-protocol removed the shepherd:merge Maintainer: merge when green (auto-merge, squash). Guide: docs/ci.md label Oct 16, 2026
@squid-protocol

Copy link
Copy Markdown
Owner Author

CI shepherd: the head of this PR (7800c2b) is not the one a maintainer labelled shepherd:merge on (new commits, or never approved), so the label was removed and nothing merges unreviewed code. Re-add the label when the new head is ready (shepherd:reapprove marks this PR until then).

@squid-protocol squid-protocol added shepherd:reapprove Bot: new commits dropped shepherd:merge; re-add it to merge this head. Guide: docs/ci.md shepherd:merge Maintainer: merge when green (auto-merge, squash). Guide: docs/ci.md and removed shepherd:needs-fix Bot: a real failure on this head; see the digest comment. Guide: docs/ci.md shepherd:retrying Bot: an infra/flake rerun is pending on this head. Guide: docs/ci.md shepherd:reapprove Bot: new commits dropped shepherd:merge; re-add it to merge this head. Guide: docs/ci.md labels Oct 16, 2026
@squid-protocol

Copy link
Copy Markdown
Owner Author

No description provided.

@squid-protocol
squid-protocol enabled auto-merge (squash) October 16, 2026 14:54
@squid-protocol

Copy link
Copy Markdown
Owner Author

No description provided.

@squid-protocol
squid-protocol merged commit 6857f34 into main Oct 16, 2026
45 checks passed
@squid-protocol
squid-protocol deleted the churn/5262-20261016T1408Z branch October 16, 2026 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

shepherd:merge Maintainer: merge when green (auto-merge, squash). Guide: docs/ci.md

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Db2 image: restore the pinned image from the Actions cache instead of pulling icr.io in every det shard (6 det failures 2026-10-16)

1 participant