Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/aw/intent.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ Evals and operational value answer different questions. PromptPex evals test whe

Select graders that test a concrete risk to achieving the intent; do not enable metrics merely because they are available. Start with the known builtin graders: use `tool-success-rate` and `tool-failure-count` when reliable collection is required; `retries`, `loops`, `execution-step-count`, and `execution-duration` when boundedness or timely escalation matters; `working-set-rebuild-factor` and `context-growth` when repeated context is an attention or cost risk; `trajectory-efficiency` when unnecessary tool churn is a concern; and `artifact-production` only when producing the intended artifacts is itself useful diagnostic evidence.

Then inspect the implemented fragments in [`shared/graders/`](../workflows/shared/graders/README.md). Use `policy-near-miss` for explicit guard or no-op requirements. For intents that require efficient investigation rather than repeated exploration, consider `state-revisit-probability-rep`, `recurrence-rate`, `recurrence-determinism`, `recurrence-laminarity`, or `recurrence-trapping-time`. For intents where varied, non-repetitive investigation is relevant, consider `event-entropy-rate` or `lempel-ziv-trajectory-complexity`. Import only the applicable implemented fragments and ensure their documented trace prerequisites are available. These trace graders diagnose execution behavior; they do not replace scenario evals or the operational-value attainment metric.
Then inspect the implemented fragments in [`shared/graders/`](../workflows/shared/graders/README.md). Use `policy-near-miss` for explicit guard or no-op requirements, and `skill-constraint-coverage` when a harness/skill declares behavioral requirements that should be exercised. Use `exploration-error` when failure may stem from insufficient search, and `exploitation-error` when the agent had enough information but failed to use it. For intents that require efficient investigation rather than repeated exploration, consider `state-revisit-probability-rep`, `recurrence-rate`, `recurrence-determinism`, `recurrence-laminarity`, or `recurrence-trapping-time`. For intents where varied, non-repetitive investigation is relevant, consider `event-entropy-rate` or `lempel-ziv-trajectory-complexity`. Use `tool-output-consumption-rate` when tool outputs going unused by later actions is a risk. Import only the applicable implemented fragments and ensure their documented trace prerequisites are available. These trace graders diagnose execution behavior; they do not replace scenario evals or the operational-value attainment metric.

## Preserve Intent on Updates

Expand Down
5 changes: 2 additions & 3 deletions .github/aw/mcp-clis.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,8 @@ Each server is a standalone executable on `PATH`. Invoke from bash with `--name

**Examples:**
```bash
playwright --help
playwright browser_navigate --url https://example.com
playwright browser_snapshot
github --help
github issue_read --method get --owner org --repo repo --issue_number 42

safeoutputs add_comment --item_number 42 --body "Analysis complete"

Expand Down
3 changes: 2 additions & 1 deletion .github/aw/syntax-core.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ The YAML frontmatter supports these fields:
- Object: Complex trigger configuration
- Special: `slash_command:` for /mention triggers
- **`forks:`** - Fork allowlist for `pull_request` triggers (array or string). By default, workflows block all forks and only allow same-repo PRs. Use `["*"]` to allow all forks, or specify patterns like `["org/*", "user/repo"]`
- **`stop-after:`** - Can be included in the `on:` object to set a deadline for workflow execution. Supports absolute timestamps ("YYYY-MM-DD HH:MM:SS") or relative time deltas (+25h, +3d, +1d12h). The minimum unit for relative deltas is hours (h). Uses precise date calculations that account for varying month lengths.
- **`stop-after:`** - Can be included in the `on:` object to set a deadline for workflow execution. Supports absolute timestamps ("YYYY-MM-DD HH:MM:SS"), relative time deltas (+25h, +3d, +1d12h), or a GitHub Actions expression (e.g. `${{ inputs.stop-after }}`) resolved at runtime instead of compile time. The minimum unit for relative/literal deltas is hours (h). Literal values use precise date calculations that account for varying month lengths; recompile with `gh aw compile --refresh-stop-time` to reset a literal deadline.
- **`cooldown:`** - Can be included in the `on:` object to block the `agent` job from starting again shortly after the most recent completed run. Value must be a literal Go duration string of at least 5 minutes (e.g. `"30m"`, `"2h"`); GitHub Actions expressions are rejected. Fails open (allows activation) if run history cannot be queried.
- **`reaction:`** - Add emoji reactions to triggering items
- **`status-comment:`** - Post status comments when workflow starts/completes on the triggering issue, pull request, or discussion (boolean, or object with `issues`/`pull-requests`/`discussions` booleans to control trigger groups independently). Defaults to `true` for `slash_command` and `label_command` triggers; defaults to `false` for all other triggers. Must be explicitly enabled for non-command triggers with `status-comment: true`.
- **`manual-approval:`** - Require manual approval using environment protection rules
Expand Down
9 changes: 5 additions & 4 deletions .github/aw/visual-regression.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,12 @@ permissions:
contents: read
pull-requests: read
engine: copilot
network:
allowed:
- local
- playwright
tools:
playwright:
allowed_domains:
- localhost
- 127.0.0.1
cache-memory:
key: visual-regression-baselines-${{ github.event.pull_request.base.ref }}
retention-days: 30
Expand Down Expand Up @@ -53,7 +54,7 @@ Otherwise compare each screenshot to its baseline. Post a comment summarizing: p

- **`cache-memory` key per base branch** — scopes baselines to `main`, `develop`, etc.
- **Explicit baseline source** — state whether baselines come from `cache-memory`, a generated artifact, or a branch directory; do not leave baseline origin implicit.
- **`allowed_domains: [localhost, 127.0.0.1]`** — prevents SSRF; serve app locally
- **`network.allowed: [local, playwright]`** — prevents SSRF; serve app locally, allow browser binary downloads
- **`retention-days: 30`** — beyond the default 7-day cache expiry
- **Filesystem-safe timestamps** — `YYYY-MM-DD-HH-MM-SS`; colons break artifact filenames
- **Minimal permissions** — all PR writes go through `safe-outputs`
Expand Down
Loading