Repository navigation
evidence: refresh stale records and the evidence report - #4820
Merged
Merged
Conversation
Contributor
squid-protocol
added a commit
that referenced
this pull request
Oct 9, 2026
…--all re-run Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
squid-protocol
added a commit
that referenced
this pull request
Oct 9, 2026
…e bot-generated (#4811) * feat(#4789): per-case det-sweep coverage ledger; CICS spec status page bot-generated The det-sweep coverage ledger (one tests/equivalence/det_sweep_coverage.json) and docs/language_status/cics_spec_status.md were rewritten by every case PR and every evidence-refresh bot run, so any two open PRs conflicted on them. - det_coverage_ledger.py: one file per case, tests/equivalence/det_sweep_coverage/<case>.json (format det-sweep-coverage/2). load()/write() are the only file-aware parts; build, check, last_coverage, fresh_coverage, migrate keep their semantics (#4730 stale-readable, #4758 entry only while proven, #4731 per-component fingerprints). write() touches only changed files and deletes a dropped case's file. - `split --all` (the one-shot migration, run here) and `split --base REV` (a pre-#4789 branch merging main: applies only the entries it changed since REV). - Compat: check / update / migrate and test_det_coverage_ledger fail while the single file exists, printing the split recipe. - evidence_history reads the per-case files (single file for older commits). - det_sweep_plan: a case's ledger file keeps the PR sweep narrow (that case). - evidence-refresh bot commits the per-case files and renders cics_spec_status.md; the page's currency test is advisory in PR CI (EVIDENCE_REPORT_ADVISORY, as #4703), smoke-test prints whether it would change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(#4789): no side effects inside asserts (CodeQL py/side-effect-in-assert) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(#4789): an unchanged re-sweep rewrites no per-case file (#4813 idempotency kept) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Merge origin/main; re-render carddemo-tranadd's evidence page (main red since #4820, same fix as #4823) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
squid-protocol
added a commit
that referenced
this pull request
Oct 9, 2026
…r estate with a schema (Fixes #4723) (#4821) * readiness scanner: feature inventory extractor -- a deterministic JSON inventory per estate, with a schema (Fixes #4723) tests/tools/feature_inventory.py ESTATE | --all-burned --write | --check: per estate, the statement forms, data types and PICTURE features, CICS commands and options, SQL forms, intrinsic functions, compile options (estate_options resolver) and copybook gaps, each with a count and the programs that use it. Unknown constructs are counted in each category's unknown bucket; a program the det parser can't read is listed in unparsed_programs with its reason and still counted by a regex fallback. Copybooks resolve through det_survey.copy_dirs. CICS command totals equal cics_census's on all 6 burned estates (240/751/500/118/34/67); 4 programs stay unparsed, each for a copybook the repo does not ship (MQ CMQODV/CMQGMOV, LE CEEIGZCT). Fixtures for the 6 burned estates, schema, doc. Written by agy (Gemini 3.1 Pro, three rounds) under orchestration; reviewed, with test guards added: the tree_sitter_language_pack importorskip, a corpus skip for the fixture --check, and a built-in schema check (jsonschema is not installed, so the schema test had silently passed). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Merge origin/main; re-render carddemo-tranadd's evidence page (main red since #4820, same fix as #4823) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
squid-protocol
added a commit
that referenced
this pull request
Oct 9, 2026
…nces) (#4823) * fix main: re-render carddemo-tranadd's evidence page; ci_digest fetches job logs with ANSI escapes Main's ground-truth / full-suite went red at #4820: the evidence-refresh bot's refresh was computed on a main before #4759 (which re-proved carddemo-tranadd), then squash-merged on top of it, leaving the record (CI oracle image d7e387df360c) and its rendered page (local image 42d04bb08a2a) out of step. `evidence.py render` from the record; no record changed. Every open PR inherited the failure (test_the_rendered_pages_are_current). ci_digest: `gh api` refuses to print a job log holding terminal escapes unless passed --allow-escape-sequences, so every log came back "unavailable"; clean() already strips them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci_digest: registry / Docker Hub rate limits and network resets are infra (rerun once), not real #4821's muninn failed on Docker Hub's 429 Too Many Requests while building a third-party action's image; the digest called it real and the shepherd only posted a note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci_shepherd: an infra failure is rerun up to 3 times, 15 min apart (a flake once) Docker Hub's 429 / auth 504 on 2026-10-09 failed muninn on every PR for over an hour; one rerun was not enough. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
squid-protocol
added a commit
that referenced
this pull request
Oct 9, 2026
… the ports a translator change moves, shards up to 20 (#4825 1a, #4814 v1) (#4826) * ci: the core checks run on every PR, so they can be required (#4825 step 1a) main has no required status checks: every gate is a convention, and bot PRs auto-merged before their tests meant anything (#4820 turned main red). A required check must report on EVERY PR, or a PR it skips waits forever. - det-sweep: no trigger path filter; det_sweep_plan.py holds the det inputs (DET_INPUTS, moved from the trigger) and plans mode "none" (0 shards) when no changed file is one -- `det` then passes in about a minute. A file det never reads (docs, README, other engine code) also no longer widens a case-only plan to a full sweep. - ruff-audit, mypy-audit, smoke-test (full-suite / smoke-test), mainframe-ground-truth: the pull_request trigger drops its docs-only paths-ignore (push filters unchanged). Next (1b, repo setting once this lands): a ruleset on main requiring det, ruff-audit, mypy-audit, full-suite, smoke-test and ground-truth. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TBJdDBRtJis5hzE244NQKV * det sweep: a translator change proves only the ports it changes; work-sized shards up to 20 (#4814 v1) A change confined to gitgalaxy/tools/cobol_to_java/det/ (translator + cobolrt runtime: everything a port's bytes come from) used to re-prove every case on 6 runners (~30 min). Now the plan job runs `det_port.py check` (translate every case at origin/main and at HEAD, no Docker / JDK; ~4 min locally) and plans only the ports that differ, the cases the PR touched, and their dependents; the rest are carried forward from main. Anything that can move a verdict without moving a port byte (harness, proof_reach.py, oracle, generator) stays a full sweep; a missing/garbled port diff is a full sweep; the nightly schedule run is always full (the honesty check). Db2 cases a diff touches are listed (db2_not_swept) -- CI skips Db2 as before. MAX_SHARDS 6 -> 20 (a narrow plan still starts one runner per ~3 cases). Tests: 32 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TBJdDBRtJis5hzE244NQKV --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
squid-protocol
added a commit
that referenced
this pull request
Oct 10, 2026
…nce Refresh nightly only (#4825 step 2) Owner decisions 2026-10-09 (#4825): rendered output off main, records stay, the bot refreshes nightly only. - docs/language_status/evidence/*.md (47 rendered pages) are no longer committed: git-ignored on main. PRs re-rendered and committed them with every re-proof and the bot rewrote them under every open PR -- the files PRs collided on (#4820 squash-merged a stale page and turned main red). - New .github/workflows/evidence-pages.yml, their only writer: after a push to main that changes a record or the renderer, `evidence.py render` from main's records (no proofs) and push to the `generated` branch, stamped with the main SHA (GENERATED_FROM). Never pushes to main; a GITHUB_TOKEN push triggers nothing else. - evidence-refresh.yml: the push trigger is gone (nightly + dispatch); its PR no longer carries pages. The evidence report and spec-status page stay on main (only this bot writes them; the report keeps the measurements its next refresh builds on), now written once a night. - test_evidence: "committed pages are current" becomes "every record renders, deterministically" plus "main tracks no pages". Docs, the PR-worker rules and links point at the generated branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TBJdDBRtJis5hzE244NQKV
squid-protocol
added a commit
that referenced
this pull request
Oct 10, 2026
…nce Refresh nightly only (#4825 step 2) (#4827) Owner decisions 2026-10-09 (#4825): rendered output off main, records stay, the bot refreshes nightly only. - docs/language_status/evidence/*.md (47 rendered pages) are no longer committed: git-ignored on main. PRs re-rendered and committed them with every re-proof and the bot rewrote them under every open PR -- the files PRs collided on (#4820 squash-merged a stale page and turned main red). - New .github/workflows/evidence-pages.yml, their only writer: after a push to main that changes a record or the renderer, `evidence.py render` from main's records (no proofs) and push to the `generated` branch, stamped with the main SHA (GENERATED_FROM). Never pushes to main; a GITHUB_TOKEN push triggers nothing else. - evidence-refresh.yml: the push trigger is gone (nightly + dispatch); its PR no longer carries pages. The evidence report and spec-status page stay on main (only this bot writes them; the report keeps the measurements its next refresh builds on), now written once a night. - test_evidence: "committed pages are current" becomes "every record renders, deterministically" plus "main tracks no pages". Docs, the PR-worker rules and links point at the generated branch. Claude-Session: https://claude.ai/code/session_01TBJdDBRtJis5hzE244NQKV Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refreshed as of commit 8130b9a (main at the time of the run).
Automated re-proof of the evidence records (#4048) that a harness, oracle or generator change made stale.
Each record's proof was run again on the committed port and case; the records and
docs/language_status/evidence/ are rewritten from those runs. A proof that now fails is recorded as
not proven. The evidence reports (docs/language_status/evidence_report/) are re-rendered from them (#4703).
Approvals are never written by this job. Auto-merge (squash) is enabled when every proof succeeded.