Skip to content

pipeline: compile shared C stubs and LD/UL loaders once per sweep, not per case (measure first) #4905

Closed as not planned
Closed as not planned
pipeline: compile shared C stubs and LD/UL loaders once per sweep, not per case (measure first)#4905
Labels
ci-cdContinuous integration, delivery, and automated runnersenhancementNew feature, sensor, or structural signaturepriority: highCore feature broken, but workarounds existscope:infraCI, tooling, churn/factory, docs infrastructure

Description

@squid-protocol
squid-protocol
opened on Oct 11, 2026

Repeated work per equivalence case (main 7e9856e):

  1. tests/tools/equivalence.py ~262: cobc -x of LD
    .cbl / UL
    .cbl for every indexed dataset of every case; their source depends only on (dd, reclen, keys) -> memoize binaries by source hash.
  2. Shared C stubs recompiled inside every case's cobc: equivalence.py ~268-277 (ggabend, ggfault, ggdisplay, LE models), equivalence_cics.py ~2028-2117 (ggcics.c, ggsql.c, ggsqlrun.c), equivalence_call.py ~184-189 -> precompile each stub set to .o once per sweep, keyed by stub hash.
  3. equivalence_java.py ~558: mvn -q -B test without -o (cics_crucible.py ~416 supports it) -> offline once ~/.m2 is primed.

Measure first: time one CICS case and one batch case (det_port run) split by step before changing anything; do the change only where it pays. Acceptance: identical verdicts (det sweep on a sample of cases incl. one Db2 and one CICS), before/after per-case time, cache keyed so a stub change invalidates it.

Evidence: pipeline audit 2026-10-11 (four read-only scouts; full reports in the orchestrator handoffs, pipeline-audit-2026-10-11/A-D). Precedent: #4899 (unicode gauntlet nested pools: 182 s -> 39 s, peak load 82.5 -> 10.5, byte-identical output), #4882 (heavy-run.sh GG_JOB_CORES), #4862/#4865 (macOS hang; pytest --timeout).

🤖 Generated with Claude Code

Activity

  1. squid-protocol
    added
    ci-cdContinuous integration, delivery, and automated runners
    enhancementNew feature, sensor, or structural signature
    priority: highCore feature broken, but workarounds exist
    on Oct 11, 2026

  2. squid-protocol commented on Oct 11, 2026

    @squid-protocol
    squid-protocol
    on Oct 11, 2026
    OwnerAuthor

    Measured first, as the issue asked (quiet box, det_port run via heavy-run.sh). None of the three caches pays, so no code was changed:

    • Batch with indexed files, carddemo-readacct (65 s): each docker step takes 0.72 s, of which the LD+UL loader compiles are 0.14 s and the ggdisplay gcc 0.04 s. Maven takes 7.4 s on the first run plus 3.8 s on each of 12 fault runs, about 53 s: over 80% of the case.
    • CICS, carddemo-acctview (24 s): docker 0.76 s, Maven 12.6 s.
    • Db2 CICS, cbsa-bnk1cac: the whole-task cobc takes 0.72 s with stubs inline vs 0.53 s with precompiled stub objects, so the stubs cost about 0.2 s.
    • mvn -o with ~/.m2 primed: 0.83 s either way.

    So stub, loader and offline caching would save under 0.3 s on cases that take 15–65 s. That's not worth the cache-invalidation machinery. All 5 cases run proved.

    The real lever is Maven + Spring start-up per fault run. _run_direct in equivalence_java.py already skips Maven but is only used with --reuse; filed separately. Closing this one with the measurement as its answer.

  3. squid-protocol
    closed this as not plannedon Oct 11, 2026
  4. squid-protocol
    added
    scope:infraCI, tooling, churn/factory, docs infrastructure
    on Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ci-cdContinuous integration, delivery, and automated runnersenhancementNew feature, sensor, or structural signaturepriority: highCore feature broken, but workarounds existscope:infraCI, tooling, churn/factory, docs infrastructure

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions