gh-aw-estc-pr-buildkite-detective.lock.yml is not configured correctly

[v1v]

elastic/ai-github-actions/.github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml is not configured correctly

Review https://github.com/elastic/observability-robots-playground-public/actions/runs/22860770568/job/66313801435 and https://github.com/elastic/observability-robots-playground-public/actions/runs/22860770568/workflow

• Grant the buildkite bot in the approval list
• Update events in the example

---

In addition it reported

● Create session plan file
  $ cat > /home/runner/.copilot/session-state/68586be5-d62d-42a1-9f57-65c561045724/plan.md <<'EOF'
  1. Gather repo and PR context
  2. Locate failed Buildkite status/logs via GitHub
  3. Trace root cause in code and verify locally
  4. Post actionable PR comment...
  └ 2 lines...
● github-actions_get
  └ {"id":22860770568,"name":"PR Buildkite Detective","node_id":"WFR_kwLOPGxjBc8A...
● github-list_pull_requests
  └ [{"id":3373980233,"number":18,"state":"open","locked":false,"title":"test","c...
● github-search_pull_requests: repo:elastic/observability-robots-playground-public is:pr
ac3f8e26db47e30050f8690f2df01dc4e0391546
  └ {"total_count":0,"incomplete_results":false}
● github-search_pull_requests: repo:elastic/observability-robots-playground-public is:pr is:open
  └ {"total_count":5,"incomplete_results":false,"items":[{"id":4046206261,"number...
● github-pull_request_read
  └ {"number":9,"title":"test: simpler","state":"open","draft":true,"merged":fals...
● github-pull_request_read
  └ {"number":7,"title":"[main][Automation] Bump VM Image version to 1757725254",...
● github-pull_request_read
  └ {"number":12,"title":"Update README.md","state":"open","draft":false,"merged"...
● github-pull_request_read
  └ {"number":18,"title":"test","state":"open","draft":true,"merged":false,"merge...
● github-pull_request_read
  └ {"number":11,"title":"test: pre-command hook","state":"open","draft":false,"m...
● Todo completed: gather-context
● safeoutputs-noop
  └ {"result":"success"}
No matching open PR was found for failed commit `ac3f8e26db47e30050f8690f2df01dc4e0391546` (from workflow run `22860770568`), so there is no PR to investigate/comment on. I completed the workflow with a `noop` safe-output: “No pull request associated with failed commit status; nothing to do.”
Total usage est:        1 Premium request
API time spent:         1m 8s
Total session time:     1m 16s
Total code changes:     +0 -0

But the PR is there elastic/observability-robots-playground-public#18

That's the one that triggered:

• https://github.com/elastic/observability-robots-playground-public/actions/runs/22860770568

In fact it shows elastic/observability-robots-playground-public@ac3f8e2 but that's for the current commit in main, however it was triggered because of the PR elastic/observability-robots-playground-public#18

See https://github.com/elastic/observability-robots-playground-public/actions/runs/22860770568/job/66313801219

that points to https://buildkite.com/elastic/observability-robots-playground-public/builds/22

[github-actions]

3 related issues

• pr-buildkite-detective does not work with workflow_run, Buildkite uses GH commnit status #421 — it documents the same Buildkite PR-detection failure mode where commit-status context diverges from PR context, which matches the observed behavior in this issue.
• [workflow-patrol] Workflow drift detected: Buildkite MCP server missing method allowlist #567 — it tracks configuration drift in the same gh-aw-estc-pr-buildkite-detective workflow file, so it may help when correcting that workflow’s config.
• pr-buildkite-detective should support public buildkite pipelines #423 — it covers prior Buildkite detective workflow behavior and fixes in the same area, which may provide implementation context for this issue.

---

What is this? | From workflow: Trigger Duplicate Issue Detector

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[github-actions]

tl;dr: The workflow is currently misconfigured for the reported scenario because it only allowlists github-actions[bot] and relies on add_comment target fallback ("triggering") for status/check_run runs where PR association can be ambiguous.

Recommendation

1. Allow Buildkite bot actors by default (or in the trigger) for this workflow’s activation gate.
2. Stop relying on implicit comment target fallback for this path; pass an explicit PR target (target-pr-number) whenever available, and emit noop when no PR can be resolved.
3. No event change is needed in this repo’s current example: estc-pr-buildkite-detective already uses status + check_run.

Findings

• The workflow only allowlists one bot by default:
  • .github/workflows/gh-aw-estc-pr-buildkite-detective.md:39-43
  • .github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml:50-53
  • Default is github-actions[bot] only.
• Activation enforces this allowlist during membership checks:
  • .github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml:1424-1429 (GH_AW_ALLOWED_BOTS: ${{ inputs.allowed-bot-users }}).
• Caller does not override allowed-bot-users or pass a PR target:
  • .github/workflows/trigger-estc-pr-buildkite-detective.yml:22-24
  • gh-agent-workflows/estc-pr-buildkite-detective/example.yml:20-22
• Safe outputs currently target comments with fallback to triggering:
  • .github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml:676
  • .github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml:1488
• The add-comment schema explicitly warns that implicit auto-targeting is event-sensitive:
  • .github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml:693-694
• Related history confirms this workflow family already migrated from workflow_run to status/check_run:
  • Issue #421, PR #422, PR #431.

Verification

I validated configuration and generated outputs locally:

$ make compile
# Result: success (includes estc workflow + trigger sync/compile)

$ make lint-workflows
# Result: success; includes
# "Checking .github/workflows/trigger-estc-pr-buildkite-detective.yml..."

And I verified the key config points above directly from the source and compiled files (line references in Findings).

Detailed Action Plan

1. Expand bot allowlist for this workflow

   • File: .github/workflows/gh-aw-estc-pr-buildkite-detective.md
   • Update allowed-bot-users default (:39-43) to include the Buildkite bot actor(s) used in your repos.
   • Keep bots: wiring (:65-66) unchanged.

2. Expose this in the consumer example

   • File: gh-agent-workflows/estc-pr-buildkite-detective/example.yml
   • In with: (:20-22), add a documented allowed-bot-users override example so consumers can align to their CI bot identity.

3. Make comment targeting explicit for non-PR trigger contexts

   • File: .github/workflows/gh-aw-estc-pr-buildkite-detective.md
   • Ensure the runtime path sets/passes target-pr-number when PR is found from the failed signal, and avoid relying on implicit triggering target fallback for this workflow path.
   • Keep/strengthen the existing noop behavior when no PR can be reliably associated.

4. Recompile and validate

   • Run make compile and make lint-workflows.

Related Items

Type	Link	Relevance
Issue	#632	Current misconfiguration report
Issue	#421	Prior fix request to move Buildkite detective off workflow_run
PR	#422	Implemented status/check_run trigger model
PR	#431	Renamed to estc-pr-buildkite-detective
Issue	#567	Related estc Buildkite detective config drift report
File	.github/workflows/gh-aw-estc-pr-buildkite-detective.md	Source config for allowlist and workflow inputs
File	.github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml	Effective compiled behavior (activation + safe outputs target)
File	gh-agent-workflows/estc-pr-buildkite-detective/example.yml	Consumer-facing trigger/example wiring

---

What is this? | From workflow: Trigger Issue Triage

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[v1v]

/ai As far as I see the github payload contains the correct commit if you search for b4d880adb627cf171dbf469c0eadad88471b050b in https://github.com/elastic/observability-robots-playground-public/actions/runs/22860770568/job/66313801219#step:2:291 you can see the value .event.commit.sha

What changes are required to be done to support that value instead of .sha

[github-actions]

You’re right about the payload shape in that run: github.sha is the workflow commit (ac3f8e…), while the status payload commit is under github.event.commit.sha (b4d880…) in the debug job log (job 66313801219, around lines 35–73 and 318).

To support that value, update the status-branch extraction in .github/workflows/gh-aw-estc-pr-buildkite-detective.md (current line ~101) from:

echo "BK_COMMIT_SHA=$(jq -r '.sha' "$GITHUB_EVENT_PATH")" >> "$GITHUB_ENV"

to:

echo "BK_COMMIT_SHA=$(jq -r '.event.commit.sha // .commit.sha // .sha // empty' "$GITHUB_EVENT_PATH")" >> "$GITHUB_ENV"

Then run make compile so the generated .github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml gets the same change (current line ~614).

If you want to make this fully explicit from the caller side too, add a dedicated input (e.g. failed-commit-sha) and pass ${{ github.event.commit.sha || github.event.sha }} from the trigger workflow, but the minimal change above is sufficient for the payload shape you pointed out.

---

What is this? | From workflow: Trigger Mention in Issue

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[strawgate]

the pull request trigger is fine please look at the actions detective prompt for how we should be doing this. Also add buildkite-limited-access[bot] to the allowed bot list