Skip to content

feat(signals): skip auto-start when a sibling report already names the same fix #90186

Open

Description

@andrewm4894

Problem

Two reports on the same team can reach auto-start with findings that name the same file and the same causative commit, and the pipeline opens a PR for each. Nothing after research compares a report with its siblings.

Each research run writes signal_finding artefacts (SignalFinding in products/signals/backend/artefact_schemas.py) with relevant_code_paths and relevant_commit_hashes, plus an actionability_judgment (ActionabilityAssessment.already_addressed). maybe_autostart_implementation_task (products/signals/backend/auto_start.py) gates on already_addressed but has no view of other reports. already_addressed is filled by the research agent's own gh pr list check, which is an instruction, not a mechanism (see the companion issue on feeding known PRs into research).

On one customer project, eight reports over five days all carried findings on the same providers.tsx path and the same commit hash (an unguarded analytics init letting local dev servers report into production). Three of them auto-started and opened three PRs changing that file. The signals were genuinely different exceptions, so grouping could not have joined them; the overlap only exists at the finding level, after research.

Fixes

  1. Sibling overlap check before auto-start. In maybe_autostart_implementation_task (or a small helper in auto_start.py it calls), load the signal_finding artefacts of the other ready reports on the team from the last N days (30 is a reasonable start) and compare with the new report's findings. Treat as overlapping when they share a causative commit hash, or share the primary code path (first entry, or any path present in every finding of the new report). Pure Postgres, no LLM call.
  2. On overlap, do not auto-start. Log a skip_reason of sibling report already targets the same fix: <report id> (the existing skip_reason pattern). Record it on the report as a log artefact so the inbox shows why no PR opened, and link the two reports (a related_report artefact or the existing linking mechanism if one fits).
  3. Prefer the report that already has a PR. If a sibling has an implementation task_run with a pr_url, point the new report's summary at it rather than at nothing.
  4. Do not merge reports automatically in this issue; linking plus no second PR is the outcome. Merging is a product decision for later.
  5. Tests: two reports, overlapping commit hash, second one must skip auto-start with the sibling reason; overlapping path only; no overlap (auto-starts as today).

Why

The pipeline already has, in its own artefacts, the fact that two reports name the same fix. Checking that is deterministic and cheap, and it is the check that would have prevented three PRs for one line of code. Relying on the research agent's gh lookup alone has already failed in production.

From this Slack thread; the investigation is written up in the self-driving-debugging skill in the skills store.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions