Skip to content

feat(agent): auto-apply high-confidence project/milestone matches - #4698

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
jeffrey701:feat/auto-apply-project-milestone-match
Jul 10, 2026
Merged

feat(agent): auto-apply high-confidence project/milestone matches#4698
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
jeffrey701:feat/auto-apply-project-milestone-match

Conversation

@jeffrey701

Copy link
Copy Markdown
Contributor

What & why

Closes #3185.

Suggest-mode (#3183/#3184) already ships the autoProjectMilestoneMatch tri-state config (off / suggest / auto) and posts an advisory comment when a PR fuzzy-matches an open Milestone or Projects v2 board. Until now "auto" behaved the same as "suggest" — it only commented. This wires the "auto" branch to actually attach the match.

What it does

maybeSuggestMilestoneMatchForPr now branches on the mode:

  • "auto" → new maybeAutoApplyProjectOrMilestoneMatch: resolves matches against the repo's configured backend (GitHub by default, Linear when opted in) and calls attachToMilestone / attachToProject for whichever match clears a confidence threshold — using the very same adapters suggest mode uses.
  • "suggest" → the existing comment-only path (unchanged).

Conservative by design. Auto-apply uses a higher bar than the suggest floor — DEFAULT_AUTO_APPLY_MIN_SCORE = 0.85 vs. the 0.65 suggest floor — because a wrong auto-attach silently mislabels a PR, while a wrong suggestion is only an advisory comment. A "native" confirmed link (score 1) always clears it; a below-threshold match is deliberately left untouched. The threshold is a parameter (defaulting to the constant) so it can be tightened to a repo's observed suggest-mode false-positive rate.

Best-effort / never blocks the gate. An attach failure is logged (milestone_auto_apply_failed) and swallowed by the webhook entry point — the same fail-open contract suggest mode already follows (src/queue/processors.ts maintenance step). Attaching is idempotent (re-PATCHing the same milestone / re-adding the same Projects v2 item is a no-op), so a repeated maintenance/webhook sweep never double-applies or spams.

Tests

test/unit/project-tracker-adapter.test.ts (all new cases pass; the module keeps 100% coverage on the added lines):

  • attaches a milestone that clears the default threshold;
  • does not attach a 0.75 fuzzy match (suggest-worthy, not auto-apply-worthy);
  • honors a lowered threshold override — the same 0.75 match attaches once the bar drops below its score;
  • attaches a matching Projects v2 item via GraphQL;
  • attaches nothing for a Linear backend (its attach is inert) without throwing;
  • routes mode "auto" through to an attach, never a suggestion comment;
  • best-effort — a failing attach in "auto" mode is logged and swallowed, never blocking the maintenance step.

Scope note

Per the issue's own prerequisite ("should not start until there's real false-positive data to set a default threshold from"), the threshold ships as a conservative code default + a function parameter rather than a persisted per-repo .gittensory.yml setting: the auto-apply mechanism and its confidence gate land here, and wiring the threshold to a per-repo setting (once production suggest-mode false-positive data exists to calibrate a default) is a clean, isolated follow-on that needs no change to this attach logic. No src/services/** or other guarded paths are touched; the change is confined to the integrations adapter and its unit test.

Wire the "auto" branch of autoProjectMilestoneMatch (the tri-state config
already shipped and parsed by JSONbored#3183/JSONbored#3184) to actually attach the matched
milestone/project instead of only commenting (JSONbored#3185).

maybeSuggestMilestoneMatchForPr now routes mode "auto" to a new
maybeAutoApplyProjectOrMilestoneMatch, which resolves matches against the
repo's configured backend (GitHub by default, Linear when opted in) and
attaches whichever milestone/project clears a confidence threshold, via
the very same adapters suggest mode uses. Auto-apply is deliberately more
conservative than suggest mode (DEFAULT_AUTO_APPLY_MIN_SCORE 0.85 vs the
0.65 suggest floor): a wrong auto-attach silently mislabels a PR, whereas
a wrong suggestion is only an advisory comment a maintainer can ignore. A
"native" confirmed link (score 1, e.g. Linear's GitHub integration)
always clears it; a below-threshold match is left untouched. The
threshold is a parameter (defaulting to the constant) so it can be tuned
to a repo's observed suggest-mode false-positive rate.

Best-effort: an attach failure is logged (milestone_auto_apply_failed)
and swallowed by the webhook entry point, never blocking the maintenance
step -- the same fail-open contract suggest mode already uses. Attaching
is idempotent, so a repeated sweep never double-applies.

Covered by test/unit/project-tracker-adapter.test.ts: attach on a
threshold-clearing match, no-attach below the default threshold, a
lowered-threshold override attaching the same match, a Projects v2
GraphQL attach, an inert Linear backend, the "auto" route attaching
instead of commenting, and a failing attach logged-and-swallowed.

Closes JSONbored#3185
@jeffrey701
jeffrey701 requested a review from JSONbored as a code owner July 10, 2026 13:53
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 10, 2026
@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.14%. Comparing base (8af214c) to head (a0a3d9b).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4698   +/-   ##
=======================================
  Coverage   94.14%   94.14%           
=======================================
  Files         437      437           
  Lines       38533    38551   +18     
  Branches    14049    14055    +6     
=======================================
+ Hits        36276    36294   +18     
  Misses       1599     1599           
  Partials      658      658           
Files with missing lines Coverage Δ
src/integrations/project-tracker-adapter.ts 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb

loopover-orb Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-10 14:06:30 UTC

2 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This PR wires the already-shipped "auto" config branch to actually attach a high-confidence project/milestone match instead of only commenting, via a new maybeAutoApplyProjectOrMilestoneMatch that reuses the exact same adapters and match-resolution logic suggest mode already exercises. The auto-apply threshold (0.85) is meaningfully higher than the suggest floor (0.65) and is a real, tested gate rather than dead code, and the webhook entry point correctly keeps the attach best-effort (logged and swallowed) so a tracker API failure never blocks the maintenance step. Test coverage is thorough: both sides of the threshold branch, the Linear inert-adapter path, the Projects v2 GraphQL attach path, and the exact best-effort failure/log shape are all exercised against the real call chain, not a fabricated one.

Nits — 5 non-blocking
  • src/integrations/project-tracker-adapter.ts: the milestone and project attach calls run sequentially rather than in parallel — fine at this call volume, but worth a Promise.all if this path ever needs to shave latency.
  • src/integrations/project-tracker-adapter.ts:486 (webhook catch handler): the auto-apply failure console.error(JSON.stringify(...)) block duplicates the shape used for suggest-mode failures elsewhere in the file — worth extracting to a shared logger helper if this pattern grows a third caller.
  • test/unit/project-tracker-adapter.test.ts: adapter-selection is only tested for github and linear backends explicitly — a switch with no default (or an exhaustiveness check) would catch a third backend being added without updating maybeAutoApplyProjectOrMilestoneMatch.
  • Consider exposing DEFAULT_AUTO_APPLY_MIN_SCORE as a .gittensory.yml overlay knob in a follow-up — the PR description anticipates repos tightening it per observed false-positive rate, but today it's only a function parameter, not an operator-facing config.
  • attachedMilestone/attachedProject aren't surfaced in any success log (only the failure path logs) — worth adding if operators want visibility into how often auto-apply fires versus silently no-ops below threshold.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #3185
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 122 registered-repo PR(s), 50 merged, 11 issue(s).
Contributor context ✅ Confirmed Gittensor contributor jeffrey701; Gittensor profile; 122 PR(s), 11 issue(s).
Gate result ✅ Passing No configured blocker found.
Linked issue satisfaction

Partially addressed
The PR wires the auto branch to call attachToMilestone/attachToProject with a best-effort, non-blocking pattern and includes solid tests for success/failure/threshold-override, but the threshold is only a function parameter with a hardcoded default rather than an actual per-repo config setting (no visible .gittensory.yml/autoProjectMilestoneMatchThreshold config plumbing or OpenAPI schema update),

Review context
  • Author: jeffrey701
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: TypeScript, Clojure, JavaScript
  • Official Gittensor activity: 122 PR(s), 11 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 99dd29b into JSONbored:main Jul 10, 2026
9 checks passed
@loopover-orb loopover-orb Bot added gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. and removed gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. labels Jul 10, 2026
@JSONbored JSONbored added gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. and removed gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. labels Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(agent): promote project/milestone matching to auto-apply mode

2 participants