Skip to content

[agentic-token-optimizer] Optimize Daily Rendering Scripts Verifier: cut ~40 % of turns spent on environment probing #48677

Description

@github-actions

Target workflow

Daily Rendering Scripts Verifier (.github/workflows/daily-rendering-scripts-verifier.md) — selected as the highest-AIC workflow not optimized in the last 14 days and not self-targeting (excludes "Token"-named workflows). No source: provenance frontmatter, so it is safe to edit directly in this repo.

Analysis period + runs analyzed

Cost profile

Metric Value
Total AIC (8 snapshot days) 1,970.99
Runs counted in snapshots 5
Avg AIC/run ~394
Avg turns/run (3 sampled runs) 74 (63–85)
Avg cost/run (sampled) $4.48
Cache-read ratio ~97–98 % (healthy — cache is not the waste driver)
Recent reliability 3 of last 3 daily runs (2026-07-26 → 2026-07-28) failed outright before the agent could do any work

Ranked recommendations

1. Fix the blocking model-pricing failure (highest priority — currently 100% failure rate)

  • Estimated AIC savings: full run cost recovery (~$4–5/run currently wasted on 4 failed API retries with 0 tokens billed, but burns ~40s wall-clock and blocks the daily check entirely for 3 straight days)
  • Evidence: all 3 most recent scheduled runs (§30342547715, §30251308561, §30194656716) fail immediately with API Error: 400 Model "claude-opus-5" has no AI credits pricing and no default pricing is configured. This is an environment/model-pool config issue upstream of the prompt, but it means the workflow has produced zero useful audits since 2026-07-25. Flag to whoever owns apiProxy.defaultAiCreditsPricing / the claude model-pool alias so claude-opus-5 gets a fallback price or is excluded from the pool used by engine: claude.

2. Remove blind node-availability probing loop (~8–10 turns/run wasted)

  • Estimated AIC savings: ~10-12% per run
  • Concrete action: Every successful sampled run repeats the same discovery sequence — node -e "console.log(...)" (denied, "requires approval"), retried with dangerouslyDisableSandbox (denied again), then a full-path node invocation, then falling back to npm run test:js via vitest. Add a short note to Phase 4 stating plainly: "Bare node invocations are permission-gated in this sandbox and will always be denied — do not attempt them. Use npm run test:js -- --run <spec> (vitest) as the only working harness, per cached prior-run notes." This removes 3–4 failed-then-retried tool calls per run.
  • Evidence: run §30150974376 logs 10 separate node/path-probe commands, all failing until the agent falls back to npm run test:js; the same dead-end pattern recurs in the other 2 sampled runs (3–4 occurrences each).

3. Consolidate repeated git status --porcelain checks (3–4 duplicate calls/run)

  • Estimated AIC savings: ~5%
  • Concrete action: Phase 7 asks the agent to verify a clean working tree before finishing; in practice it calls git status --porcelain 3–4 times with slightly different flags (-C <path>, cd && git status, piped | head). Instruct: "Run git -C ${{ github.workspace }} status --porcelain exactly once after cleanup; do not repeat this check."
  • Evidence: §30150974376 has 4 near-identical git-status invocations; the other 2 sampled runs have 3 each.

4. Add explicit non-interactive polling guidance for the logs/audit MCP tools (5–6 sleep+tail calls/run)

  • Estimated AIC savings: ~8%
  • Concrete action: Phase 1's logs tool call is followed in every run by manual sleep N; tail ... polling loops against background task output files (the tool call itself appears to run async/backgrounded and 60s-time out per cache notes surfaced in the model's own summary). Add to Phase 1: "If the logs tool call appears to hang or time out, it is still completing the download server-side — wait once with a single sleep 60, then read the file_path result directly; do not poll repeatedly."
  • Evidence: 4–6 sleep; tail polling commands appear in every sampled run; one run's own final summary self-reports: "The agenticworkflows logs MCP tool 60s-times-out on every call, but still downloads artifacts server-side... readable afterward."

Optional structural optimization: shared setup note

Phases 1, 4, and 7 each restate ad hoc environment caveats (bash-cannot-access-/tmp, node-is-gated, MCP tool timeouts) inline and inconsistently. Recommend consolidating these into one ## Known Environment Constraints section placed right after ## Phase 0: Setup, listing:

  • bash cannot access /tmp/ — use Read/Write tools for /tmp/gh-aw/... paths
  • bare node execution is permission-gated (always denied) — use npm run test:js instead
  • the logs MCP tool may appear to hang/time out but completes server-side — wait once, don't poll repeatedly

This avoids restating these three caveats in three different phases and gives the agent one authoritative reference, estimated 5–8% additional turn reduction from reduced re-discovery.

Caveats

  • Sample size is 3 full run logs plus 8 snapshot-days; wider sampling could refine the exact turn-savings percentages.
  • Recommendation rejig docs #1 (model pricing) is an infra/config issue, not a prompt change — flagging here because it currently blocks all audits and dominates recent reliability data, but it may need to be actioned outside this workflow's .md file.
  • No inline sub-agent candidates identified: this workflow's phases are tightly sequential (each phase's output feeds the next — engine/output-file from Phase 2 is required by Phase 4 and 5), so it does not meet the independence bar for extraction.

References:

Generated by Agentic Workflow AIC Usage Optimizer · sonnet50 · 85 AIC · ⊞ 10K ·

  • expires on Aug 4, 2026, 7:53 AM UTC-08:00

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions