diff --git a/.claude/agents/gitgalaxy-pr-worker.md b/.claude/agents/gitgalaxy-pr-worker.md index e853699bfc..bad863fdc9 100644 --- a/.claude/agents/gitgalaxy-pr-worker.md +++ b/.claude/agents/gitgalaxy-pr-worker.md @@ -12,7 +12,7 @@ You land one PR. The orchestrator gave you issue(s), acceptance criteria and coo ## Setup (once) 1. Primary checkout: `PRIMARY=${GG_PRIMARY:-$(dirname "$(cd "$(git rev-parse --git-common-dir)" && pwd)")}`. Worktrees live in `$PRIMARY/../gitgalaxy-worktrees/`. Slug = short kebab name. `git -C $PRIMARY fetch origin && git -C $PRIMARY worktree add $PRIMARY/../gitgalaxy-worktrees/ -b /- origin/main`. (`tests/tools/worktree_env.sh ` does the same and refuses the primary checkout.) Never edit the primary checkout. 2. Before starting, look for existing work on the issue and do not duplicate landed work: `gh api 'repos/squid-protocol/gitgalaxy/pulls?state=all&per_page=100' -q '.[]|select((.title+.head.ref+(.body//""))|contains("#"))|[.number,.state,.head.ref]|@tsv'` plus `git ls-remote origin '**'`. -3. Python: `PY=${GG_PY:-}` (the box's shared venv, e.g. `$PRIMARY/../venvs/*/bin/python`). It MUST have the `full` AND `translator` extras (`pip install -e .[full,translator]`). Without `translator` the det tests silently skip; without `full` 11 `test_security_auditor` tests fail spuriously. Check with `pytest -rs` on a det test and SAY SO in your report if tests skip. +3. Python: `PY=${GG_PY:-}` (the box's shared venv, e.g. `$PRIMARY/../venvs/*/bin/python`). It MUST have the `full` AND `translator` extras (`pip install -e .[full,translator]`). Without `translator` the det tests silently skip; without `full` 11 `test_security_auditor` tests fail spuriously. Check with `pytest -rs` on a det test and SAY SO in your report if tests skip. `python` is not on PATH in a worker worktree: call the venv's interpreter by path (#5100). A failing `python -` heredoc can silently skip the edits it carried: check `git diff` after each scripted edit. 4. Env for every command: `PYTHONPATH=`, `GITGALAXY_LICENSE_KEY=COMMUNITY_FREE_TIER`, `PATH=$(dirname $PY):$PATH`, and the corpora (defaults beside `$PRIMARY`, overridable): `KEYWORD_ROSETTA_PATH`, `LANGUAGE_CRUCIBLE_PATH`, `ESTATE_CRUCIBLE_PATH`, `CICS_CRUCIBLE_PATH`, `GITGALAXY_MAINFRAME_CORPORA` (`$PRIMARY/.mainframe_corpora`). `pr_gates.py` and the crucible tools fill in these defaults themselves. The defaults resolve beside `$PRIMARY`, so a worktree of any other clone gets false FAIL/SKIP from `--ratchets`: fetch the branch into a worktree of the box's primary clone and gate there. For equivalence / CICS crucible / det work: `eval "$($PY tests/tools/equivalence_env.py --shell)"` sets all of these plus a verified JDK 17 (JAVA_HOME / JDK_17: under the default JDK 21 every forge-compile / java cell FALSELY fails "release version 17 not supported"), and `$PY tests/tools/equivalence_env.py --check` prints PASS / FAIL per item (JDK 17, mvn, Docker + GnuCOBOL image, corpora, LANGUAGE_CRUCIBLE_PATH at its pin -- a wrong one fails golden falsely -- cics-crucible at its pin, tree_sitter_language_pack, gitgalaxy imported from the worktree). Any new test touching the det translator / tree-sitter starts with `pytest.importorskip("tree_sitter_language_pack")`: the ground-truth CI job has no `translator` extra (#4560, #4563). @@ -63,7 +63,7 @@ Do NOT commit the rendered evidence report (`docs/language_status/evidence_repor - cics-crucible: case PRs only; never tag or bump the pin without an orchestrator instruction; prove off-pin cases with `--unpinned`. ## Running -- Use `gh api` REST, not `gh issue view` / `gh pr edit` (gh 2.45 fails on Projects-classic GraphQL). Keep API use low. No `jq` in loops; use `gh -q`. `commits//check-runs` pages at 30: add `?per_page=100` (or `--paginate`), or you miss legs. +- Use `gh api` REST, not `gh issue view` / `gh pr edit` (gh 2.45 fails on Projects-classic GraphQL). Keep API use low. No `jq` in loops; use `gh -q`. When computing stats from tabular `gh api` output, print a header row first: an empty column shifts fields silently (#5139). Never run `cat > file` without a heredoc in a compound command: it waits on stdin and hangs the worker (#5157). `commits//check-runs` pages at 30: add `?per_page=100` (or `--paginate`), or you miss legs. - Tests first: the narrowest failing test, then fix, then the language/extraction tests. - Golden: bless only via `golden-lock.sh python tests/tools/crucible_check.py --update --yes`, on BOTH legs (both modes it runs), only after a real-scan proof. Attribute every diff to your change (`scope_check.py --expect `); never bless a sibling's drift. REFUSE to bless if the run excluded files via timeouts (load-induced fake diffs, #4247); rerun when quieter. - Bless with `crucible_check.py --update --yes` ONLY (CI-matched venvs per leg), never `update_golden_master` from `~/venvs/galaxy_venv` (py3.12 vs CI pin gave ~2.9k phantom diffs, #4258). After a sibling merge that moved the golden masters, merge origin/main and re-bless (a sibling merge that did not is no reason to merge). @@ -72,7 +72,7 @@ Do NOT commit the rendered evidence report (`docs/language_status/evidence_repor - Gates before push: `pr_gates.py --fast` and `--ratchets` ONLY, via `heavy-run.sh`, with JDK 17 from `equivalence_env --shell`; never two gate runs at once. The full suite is CI's job (owner, 2026-10-07). It warns if local ruff/mypy/python differ from CI; follow its private-venv hint. Re-read new subclasses for a missing `super().__init__()`. - Taking over a stale or foreign branch: check divergence first (`git rev-list --left-right --count origin/main...HEAD`). A red caused by staleness (pins, golden masters) is fixed by the merge rule in the next bullet, not by merging just to be current. - Merge main ONLY for a reason (#5134, owner 2026-10-14): the PR is CONFLICTING/dirty, a required check fails because of main, or golden masters / generated files must be re-blessed after a sibling merge. Never merge main just to be current or "behind": every push to a ready PR costs a full CI run (27% of CI minutes were superseded-SHA runs), and the shepherd no longer updates a behind branch. How: `git merge origin/main`, never rebase, never force-push. Generated files: take main's, then regenerate with their tools; check baselines git may silently revert (ruff baseline: regenerate, don't hand-merge). -- Commit only your own files (explicit `git add `; never `-A`); check `git diff origin/main --stat` has nothing unrelated. Commit messages and the PR body end with the attribution lines the launching session gave you, verbatim. +- Commit only your own files (explicit `git add `; never `-A`; `docs/self_scan/gitgalaxy_master.db-shm` / `-wal` show up untracked in plan-first worktrees: never add them, #5103); check `git diff origin/main --stat` has nothing unrelated. Commit messages and the PR body end with the attribution lines the launching session gave you, verbatim. - `gh auth setup-git`, push, open ONE draft PR to main (a `WIP:` draft early is fine as a checkpoint). Edit its body later with `gh api -X PATCH repos/squid-protocol/gitgalaxy/pulls/ -f body=...`. ## Hygiene diff --git a/.claude/rules/python-test-traps.md b/.claude/rules/python-test-traps.md new file mode 100644 index 0000000000..d2f8dfd820 --- /dev/null +++ b/.claude/rules/python-test-traps.md @@ -0,0 +1,16 @@ +--- +description: "Python and test-writing traps that cost fleet workers a round trip: optional dict keys, default-arg binding, float tolerances, ReDoS timing transport, whole-repo scans in tests" +globs: "gitgalaxy/**/*.py,tests/**/*.py" +--- +# Python and Test Traps + +* **A new optional key in a coverage/reading dict is added only when non-empty.** An always-present empty key breaks + exact-equality and evidence-report tests (#5100). +* **A default arg binds at definition time.** `def f(path=BASELINE)` ignores a test's monkeypatch of the module + constant; default to `None` and resolve inside the function (#5169). +* **Tolerance checks compare against the scaled bound.** `total / budget - 1 > 0.10` misjudges the exact boundary in + floats; write `total > budget * (1 + tol)` (#5169). +* **A child stuck in a regex never flushes `multiprocessing.Queue.put`** (the feeder thread needs the GIL). Send + batched ReDoS timings through a `Pipe` (#5171). +* **A test that builds `Orchestrator(".")` and calls `execute_pipeline` scans the whole repo** through the ecosystem + audits. Give it a temp root (#5171). diff --git a/.claude/rules/unicode-identifiers.md b/.claude/rules/unicode-identifiers.md index 30b023c9df..0fd531d39a 100644 --- a/.claude/rules/unicode-identifiers.md +++ b/.claude/rules/unicode-identifiers.md @@ -7,6 +7,8 @@ globs: "gitgalaxy/**/*.py" * **Never write literal runs of non-ASCII characters** in regexes or constants: X-Ray flags them as high entropy. Splice the named classes from `gitgalaxy/standards/language_standards/identifiers.py` (`ID_START`, `ID_CONTINUE`, `CAPITAL`, `SMALL`, `VIRAMA`, `NAME`, `NATIONAL`, `WIDE_DIGITS`, `WIDE_HYPHENS`), or build characters with `chr()`. +* **A dense regex literal is flagged the same way** (`pr_gates --fast` X-Ray entropy anomaly, #5157): build character + classes from parts, e.g. `"A-Z0-9" + "@#$_"`, instead of one long literal. * **Keep each class's ASCII meaning** when swapping `[A-Z]` / `[a-z_]` / `\w` for a Unicode class: `[A-Z]` -> `CAPITAL`, `[a-z_]` -> `SMALL`; `\w` has no combining marks (Mn / Mc), so use `ID_CONTINUE`, not `\w`. * Diff the changed rules' comments against main, and prove the swap with byte-identical golden masters. diff --git a/.claude/skills/cics-command-slice/SKILL.md b/.claude/skills/cics-command-slice/SKILL.md index 115d3e7781..6ca595e3f1 100644 --- a/.claude/skills/cics-command-slice/SKILL.md +++ b/.claude/skills/cics-command-slice/SKILL.md @@ -95,6 +95,11 @@ Traps `--check` exists for: - **importorskip.** Any test that touches the translator (det, tree-sitter) starts with `pytest.importorskip("tree_sitter_language_pack")`: the ground-truth CI job installs no `translator` extra (#4560 / #4563 broke on it). +- **`python -m gitgalaxy.standards.cics regen --check`** (#5103) says quickly whether a spec edit needs the generated + runtime tables regenerated (`regen` without `--check` rewrites them). +- **IBM macro reference** (#5153): FCT/DCT/TST macros are in the CICS TS for OS/390 R3 Resource Definition Guide + (publibfp.dhe.ibm.com/epubs/pdf/dfha400.pdf, ch.13 macros, ch.16 macro->RDO table). PCT/PPT macros are in no + reachable IBM doc: say so rather than guess. - **test_tool_regex_redos** (#4477) is flaky under load: rerun once before blaming your regex. - **The full-suite OS grid does not run on PRs** (#4518). For a path- or OS-sensitive harness change dispatch it: `gh workflow run full-suite-gate.yml --ref `. diff --git a/.claude/skills/det-port/SKILL.md b/.claude/skills/det-port/SKILL.md index bb36e73f03..d04adbfe09 100644 --- a/.claude/skills/det-port/SKILL.md +++ b/.claude/skills/det-port/SKILL.md @@ -42,8 +42,9 @@ export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 GITGALAXY_LICENSE_KEY=COMMUN |---|---| | translate + prove cases | `$PY tests/tools/det_port.py run CASE... --work DIR [--style structured] [--typed [--groups]] [--jobs 2]` | | translate every case only | `$PY tests/tools/det_port.py run --all-cases --translate-only --work DIR` | -| **did my change move a port?** | `$PY tests/tools/det_port.py check --work DIR` (base: `origin/main`; `--base-ref REF`, `--base DIR`). A runtime class counts only for the ports that name it | +| **did my change move a port?** | `$PY tests/tools/det_port.py check --work DIR` (base: `origin/main`; `--base-ref REF`, `--base DIR`). A runtime class counts only for the ports that name it, but a change to the shared `CicsTask.java` marks every port changed in `check`: re-prove the ones that can reach the changed code and name them in the report (#5147) | | structural parity of one port | printed by `run` and `check` as `parity warning:` lines (`parity_warnings` in summary.json / check.json); `tests/tools/det_parity.py`, fit and bands in `det_port_design.md` | +| **run a det case** | `det_port.py run`: only it applies the deterministic port. `equivalence.py run` uses the generated skeleton, not the deterministic port (#5147) | | prove one port by hand | `$PY tests/tools/equivalence.py run CASE --port DIR/CASE/port --keep DIR/proof --faults all` | | all CI gates | `$PY tests/tools/pr_gates.py` (`--fast` skips the golden masters and the suite) | | **re-prove everything** (a runtime / harness / oracle change) | `$PY tests/tools/proof_sweep.py --work DIR` -- every det and model port, checked against the cases not proven on purpose; det ~9 min at `--jobs 4`, model ~12 min (2026-10-02) |