[cli-consistency] CLI Consistency Issues - 2026-08-11

[github-actions]

Summary

A comprehensive inspection of the gh-aw CLI comparing real --help output (/tmp/gh-aw/agent/all-help.txt, generated from run 31497017446) against docs/src/content/docs/setup/cli.md found 1 confirmed documentation gap. All other flag sets, descriptions, and examples checked (add, add-wizard, compile, validate, run, update, upgrade, deploy, trial, secrets, env, mcp, project, doctor, health, checks, forecast, outcomes, enable/disable/remove, list/status, hash-frontmatter) were consistent between CLI help and docs.

Severity Breakdown

• High: 0
• Medium: 1
• Low: 0

Findings

1. --runtime flag undocumented for logs and audit (Medium)

Affected commands: gh aw logs, gh aw audit

Both commands expose a real, working --runtime flag:

# gh aw logs --help
      --runtime string        Filter to runs using a specific sandbox agent runtime (e.g., gvisor, docker-sbx)

# gh aw audit --help
      --runtime string      Filter to runs using a specific sandbox agent runtime (e.g., gvisor, docker-sbx)

It is also demonstrated in the audit help examples:

gh aw audit 1234567890 --runtime gvisor   # Skip run unless sandbox agent runtime matches

However, docs/src/content/docs/setup/cli.md never mentions --runtime anywhere. The only match for the string "runtime" in the whole file is an unrelated sentence about Dependabot ("analyzing runtime tools across all workflows", line 349).

• logs Options line (cli.md:518) lists: --after-run-id, --artifacts, --before-run-id, --cache-before, --count/-c, --end-date, --engine/-e, --evals, --exclude-staged, --filtered-integrity, --firewall, --format, --json/-j, --last, --no-firewall, --output/-o, --parse, --ref, --report-file, --repo/-r, --safe-output, --start-date, --stdin, --summary-file, --timeout, --tool-graph, --train — missing --runtime.
• audit Options line (cli.md:547) lists: --artifacts, --evals, --experiment, --format, --json/-j, --output/-o, --parse, --repo/-r, --stdin, --variant — missing --runtime.

Expected: Both Options lines should include --runtime, and ideally a short usage example (mirroring the existing gh aw audit 1234567890 --runtime gvisor example already present in the CLI help) should be added to the logs and audit sections of cli.md.

Suggested fix: Add `--runtime` to both Options lines and add one example line each, e.g.:

gh aw logs --runtime gvisor          # Filter logs by sandbox agent runtime
gh aw audit 1234567890 --runtime gvisor  # Skip run unless sandbox agent runtime matches

Priority: Medium — this is a real, functioning flag useful for debugging sandboxed/gVisor runs, and its complete absence from the reference docs means users won't discover it without reading --help directly.

Inspection Metadata

• Commands inspected: All 30+ top-level commands and their subcommands (add, add-wizard, audit, checks, compile, completion, deploy, disable, doctor, domains, enable, env [get/update], experiments [list/analyze], fix, forecast, hash-frontmatter, health, help, init, lint, list, logs, mcp [list/list-tools/inspect/add], mcp-server, new, outcomes [history], pr [transfer], project [new], remove, run, secrets [set/bootstrap], status, trial, update, upgrade, validate, version), 429 individual per-flag help snapshots.
• Method: Extracted the Flags: section (excluding Global Flags:) from each command's pre-collected --help output and diffed the flag names against the corresponding **Options:** line in docs/src/content/docs/setup/cli.md, cross-checked flag descriptions, defaults, and examples for wording/behavior mismatches, and scanned for typos/grammar issues across the full help corpus (none found beyond the gap above).
• Date: 2026-08-11
• **Source (redacted) /tmp/gh-aw/agent/all-help.txt (22,530 lines) and /tmp/gh-aw/agent/help-output/*.txt (429 files), pre-collected by pre-agent-steps for run 31497017446.

Generated by ✅ CLI Consistency Checker · auto · 98.3 AIC · ⌖ 3.09 AIC · ⊞ 6.7K · ◷

• expires on Aug 13, 2026, 5:47 AM UTC-08:00

[github-actions]

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · gpt54 · 2.89 AIC · ⌖ 8.05 AIC · ⊞ 10K · ◷