PR Buildkite Detective creates one github comment per build failure on a PR

[v1v]

Reduce spamming when a PR contains multiple commits and each commit failed in the CI.

By reusing the existing comment.

For instance, given elastic/elastic-agent#14012

It contains:

• Import Fleet Server API types from fleet-server pkg/api elastic-agent#14012 (comment)
• Import Fleet Server API types from fleet-server pkg/api elastic-agent#14012 (comment)
• Import Fleet Server API types from fleet-server pkg/api elastic-agent#14012 (comment)

Instead one unique comment per failure, let's reuse the comments if a similar comment exists

[v1v]

cc @fr4nc1sc0-r4m0n

[github-actions]

This issue looks like a duplicate of #647 and may be safe to close.

Possible duplicates:

• https://github.com/elastic/integrations/pull/17730 contains multiple comments for the PR Buildkite Detective #647 — it requests the same behavior in the same workflow: reuse an existing PR Buildkite Detective comment instead of posting a new comment on each failure to reduce PR spam.

2 related issues

• Combine multiple automated comments into one with collapsable sections #649 — it captures the same noise-reduction direction at broader scope, with discussion about consolidating automated output into a single evolving PR comment.
• PR actions detective no-op if no failures #620 — it documents adjacent comment-volume control in detective workflows, which may help with implementation constraints around when to emit or suppress comments.

Note

🔒 Integrity filter blocked 13 items

The following items were blocked because they don't meet the GitHub integrity level.

• #428 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #420 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #225 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #17 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #237 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #53 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #239 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #230 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #153 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #20 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #594 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #684 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #605 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

---

What is this? | From workflow: Trigger Duplicate Issue Detector

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[v1v]

/ai create pull request

[github-actions]

tl;dr: This is reproducible and currently expected from workflow design: ESTC Buildkite Detective posts a new comment per failing event and only hides older ones; it does not reuse/update an existing comment. I recommend treating this as a duplicate of #647 and implementing true comment upsert.

Recommendation

1. Mark #1098 as duplicate of #647 (same root request, #647 already has prior triage and implementation direction).
2. Implement single-comment upsert for ESTC Buildkite Detective (update existing detector comment on the PR instead of creating a new one).
3. Keep current noop comparison as a second guard, and optionally tighten trigger dedupe to reduce repeated runs.

Findings

• The workflow is triggered on both failing status and failing check_run events, so multiple failure events on a PR can produce multiple runs:
  • gh-agent-workflows/estc-pr-buildkite-detective/example.yml:2-5
  • gh-agent-workflows/estc-pr-buildkite-detective/example.yml:17-19
• Concurrency is run/event-scoped, not PR-scoped:
  • .github/workflows/gh-aw-estc-pr-buildkite-detective.md:53-55
• Safe output behavior is add-comment + hide older, not update existing:
  • .github/workflows/gh-aw-fragments/safe-output-add-comment-pr-hide-older.md:2-9
  • .github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml:604
• Prompt-level dedupe only noops when diagnosis/remediation are materially the same; otherwise it still posts add_comment:
  • .github/workflows/gh-aw-estc-pr-buildkite-detective.md:250
  • .github/workflows/gh-aw-estc-pr-buildkite-detective.md:254
• This same request already exists in #647 and remains open.

Verification

I verified behavior with repository-local grep checks:

$ grep -nE "status:|check_run:|state == 'failure'|conclusion == 'failure'" gh-agent-workflows/estc-pr-buildkite-detective/example.yml
3:  status:
4:  check_run:
17:      (github.event_name == 'status' && github.event.state == 'failure') ||
18:      (github.event_name == 'check_run' && github.event.check_run.conclusion == 'failure')

$ grep -nE "concurrency:|group:|check_run.id \|\| github.run_id|Deduplication|add_comment" .github/workflows/gh-aw-estc-pr-buildkite-detective.md
53:concurrency:
54:  group: ${{ github.workflow }}-estc-pr-buildkite-detective-${{ github.event.check_run.id || github.run_id }}
250:**Deduplication**: Check the most recent prior detective comment on the PR. If the root cause and remediation are the same, call `noop` instead of posting a duplicate.
254:Call `add_comment` on the PR using this structure:

$ grep -nE "add-comment:|hide-older-comments|max: 1|target:" .github/workflows/gh-aw-fragments/safe-output-add-comment-pr-hide-older.md
3:  add-comment:
4:    max: 1
8:    hide-older-comments: true
9:    target: "triggering"

$ grep -nE "hide_older_comments|max\":1|target\":\"triggering\"" .github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml | head -n 5
604:          {"add_comment":{"hide_older_comments":true,"max":1,"target":"triggering"},...}
1421:          GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"hide_older_comments\":true,\"max\":1,\"target\":\"triggering\"},...}"

Detailed Action Plan

1. Add a stable detector marker in the ESTC prompt output (e.g., HTML comment workflow ID) in .github/workflows/gh-aw-estc-pr-buildkite-detective.md so previous detective comments can be found deterministically.
2. Replace add-only behavior for this workflow with upsert semantics in the safe-output fragment path currently imported by ESTC:
   • .github/workflows/gh-aw-fragments/safe-output-add-comment-pr-hide-older.md
3. Update ESTC instructions to explicitly update existing detective comment first, and only create one if no prior marker is found:
   • .github/workflows/gh-aw-estc-pr-buildkite-detective.md
4. Recompile lockfiles so generated config matches source changes:
   • .github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml
5. Update user-facing docs to reflect upsert behavior instead of “hide older” behavior:
   • gh-agent-workflows/estc-pr-buildkite-detective/README.md
6. Optional follow-up: reduce duplicate invocations by tightening event strategy (especially around status + check_run) after landing upsert.

Related Items

Type	Link	Relevance
Issue	#1098	Current report
Issue	#647	Same request (“reuse existing comment”)
PR	#596	Added hide-older-comments, but does not implement comment reuse/update
PR	#842	Open trigger-scoping work for Buildkite check_run; adjacent but not comment-upsert
File	.github/workflows/gh-aw-estc-pr-buildkite-detective.md	Current concurrency + prompt dedupe/add-comment behavior
File	.github/workflows/gh-aw-fragments/safe-output-add-comment-pr-hide-older.md	Add-comment + hide-older configuration
File	.github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml	Compiled safe-output config shows add-comment/hide-older
File	gh-agent-workflows/estc-pr-buildkite-detective/example.yml	Trigger conditions that can produce repeated runs

Note

🔒 Integrity filter blocked 4 items

The following items were blocked because they don't meet the GitHub integrity level.

• #937 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #763 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #764 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #594 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

---

What is this? | From workflow: Trigger Issue Triage

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[github-actions]

I can’t open a PR for this from this run because the fix requires editing files under .github/workflows/, and workflow-file changes can’t be pushed by this automation token.

Evidence in current sources:

• .github/workflows/gh-aw-estc-pr-buildkite-detective.md:250 only does semantic dedupe (noop only when diagnosis/remediation match).
• .github/workflows/gh-aw-estc-pr-buildkite-detective.md:254 still instructs add_comment for new output.
• .github/workflows/gh-aw-fragments/safe-output-add-comment-pr-hide-older.md:8 is configured to hide-older-comments: true (hide old comments), not update an existing one.

So this issue (#1098) is still the same open gap as #647: true comment upsert (update existing detective comment) is not implemented yet.

Maintainer action needed: apply the workflow-source change directly in .github/workflows/ (and recompile lock files) to switch this detector from add+hide behavior to deterministic update-in-place behavior.

---

What is this? | From workflow: Trigger Mention in Issue

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.