Skip to content

fix: gate AI slop advisory to confirmed contributors - #727

Closed
JSONbored wants to merge 1 commit into
mainfrom
codex/fix-costly-ai-slop-calls-for-prs
Closed

fix: gate AI slop advisory to confirmed contributors#727
JSONbored wants to merge 1 commit into
mainfrom
codex/fix-costly-ai-slop-calls-for-prs

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Motivation

  • The AI slop advisory could be invoked for any PR author when slopGateMode != off and slopAiAdvisory was enabled, allowing untrusted PRs to consume shared Workers AI budget.
  • The budget estimate undercounted worst-case model usage because the runner may perform retries and a fallback model, amplifying actual calls versus the recorded estimate.

Description

  • Require a confirmedContributor check in runAiSlopForAdvisory and no-op when it is false or the advisory has no headSha so unconfirmed PR authors cannot trigger the AI call (src/queue/processors.ts).
  • Move contributor confirmation earlier in the PR processing flow and pass confirmedContributor through to the slop advisory helper so the gating is enforced before any AI invocation (src/queue/processors.ts).
  • Increase the neuron-budget estimate to account for the worst-case retry/fallback call count by adding SLOP_AI_MAX_MODEL_CALLS = 6 and using it in estimateNeurons (src/services/ai-slop.ts).
  • Add unit coverage that verifies unconfirmed contributors do not invoke the AI runner and update existing tests to pass the new confirmedContributor parameter (test/unit/ai-slop.test.ts).

Testing

  • Ran type checking with npm run typecheck and it completed successfully.
  • Ran the AI slop unit tests with npx vitest run test/unit/ai-slop.test.ts and the suite passed (28 tests, all passed).

Codex Task

@ghost

ghost commented Jun 14, 2026

Copy link
Copy Markdown

Note

Gittensory Gate skipped

PR closed before full evaluation. No late first comment was created.

Signal Result Evidence Action
Gate result ⚠️ Skipped #727 is no longer open. No action.

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

@ghost ghost added the gittensory:reviewed label Jun 14, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@ghost

ghost commented Jun 14, 2026

Copy link
Copy Markdown

gittensory · advisory review

Reviewed 3 changed file(s) — two independent AI reviewers.

Suggested action:Safe to merge — both reviewers found no blocking issues.

Reviewer A · gpt-oss-120b — recommends ✅ merge
The PR adds a confirmation check so AI slop advisory only runs for verified contributors, introduces a constant to bound model calls, and updates tests accordingly. The changes are well‑scoped, compile‑time safe, and include appropriate test coverage.

Suggestions

  • Consider documenting why the official contributor detection is only performed when gateEnabled is true, to avoid future confusion.
  • If runAiSlopForAdvisory could be called from other places in the future, make confirmedContributor an optional parameter with a default of false to keep the API flexible.

Worth double-checking

  • If there are other call sites for runAiSlopForAdvisory that were not updated, compilation will fail.
  • Skipping the contributor check when gateEnabled is false may unintentionally suppress AI advisory for confirmed contributors.

Reviewer B · nemotron-3-120b-a12b — recommends ✅ merge
The PR gates the AI slop advisory to confirmed contributors to prevent untrusted PRs from consuming the shared Workers AI budget. It adds a confirmedContributor check in runAiSlopForAdvisory, updates the neuron budget estimation to account for up to 6 model calls, and includes a new test verifying the advisory is skipped for unconfirmed contributors. The change aligns with the project's resource protection goals and maintains the public/private boundary.

Worth double-checking

  • The neuron budget estimation now uses a fixed multiplier of 6 (SLOP_AI_MAX_MODEL_CALLS); verify that runGittensoryAiSlopAdvisory does not exceed 6 actual AI calls to ensure accurate budget tracking.
  • Ensure the moved official detection logic in maybePublishPrPublicSurface correctly handles edge cases (e.g., missing author, disabled gate) after the refactor.

@ghost ghost added the gittensory-review label Jun 14, 2026
@JSONbored JSONbored self-assigned this Jun 14, 2026
JSONbored added a commit that referenced this pull request Jun 14, 2026
Completes the BYOK-only gating in this PR by closing the remaining cost
vector: the AI slop advisory was still invoking FREE Workers AI for any
PR author (the shared daily neuron budget), even though the AI review
path (runAiReviewForAdvisory) already no-ops for unconfirmed authors.

- runAiSlopForAdvisory now returns early when `!confirmedContributor`
  (matching the AI review path exactly): no AI spend — free OR BYOK — on
  a PR from an unconfirmed/untrusted author. The deterministic slop core
  still runs for everyone; only the paid AI layer is gated. Fail-closed:
  if the contributor can't be confirmed, no AI runs.
- The BYOK key lookup no longer needs its own `&& confirmedContributor`
  guard (the early return already guarantees it).

Adopts the full-gate approach from the sibling PR #727 (which also
spotted this vector) while preserving this branch's BYOK integration
(#728). Deliberately does NOT adopt #727's neuron-budget bump (1 -> 6):
the AI review path estimates by LOGICAL free-call count, not worst-case
retries, so a 6x reservation would be inconsistent and over-reserve,
causing premature quota_exceeded.
@JSONbored

Copy link
Copy Markdown
Owner Author

Superseded by #737. Your full-gate insight (no AI spend at all for unconfirmed authors, matching runAiReviewForAdvisory) was the right call and has been adopted in #737 — but on the current post-BYOK base, so it preserves the BYOK integration from #728 (this branch predated it and would have dropped settings/BYOK from runAiSlopForAdvisory).

One deliberate difference: I did not carry over the estimateNeurons bump from 1 → SLOP_AI_MAX_MODEL_CALLS=6. The number is accurate as a worst case (2 models × 3 retries), but the AI review path estimates by logical free-call count rather than worst-case retries, so a 6× reservation would be inconsistent with that convention and would over-reserve the daily neuron budget (premature quota_exceeded). Keeping the estimate at 1 logical call matches runAiReviewForAdvisory. Thanks for catching the vector.

@JSONbored JSONbored closed this Jun 14, 2026
@github-project-automation github-project-automation Bot moved this from Todo to Done in gittensory - v1 roadmap Jun 14, 2026
JSONbored added a commit that referenced this pull request Jun 14, 2026
…free + BYOK) (#737)

* fix: gate BYOK slop advisory by contributor

* fix(slop): fully gate the AI slop advisory on confirmed contributor

Completes the BYOK-only gating in this PR by closing the remaining cost
vector: the AI slop advisory was still invoking FREE Workers AI for any
PR author (the shared daily neuron budget), even though the AI review
path (runAiReviewForAdvisory) already no-ops for unconfirmed authors.

- runAiSlopForAdvisory now returns early when `!confirmedContributor`
  (matching the AI review path exactly): no AI spend — free OR BYOK — on
  a PR from an unconfirmed/untrusted author. The deterministic slop core
  still runs for everyone; only the paid AI layer is gated. Fail-closed:
  if the contributor can't be confirmed, no AI runs.
- The BYOK key lookup no longer needs its own `&& confirmedContributor`
  guard (the early return already guarantees it).

Adopts the full-gate approach from the sibling PR #727 (which also
spotted this vector) while preserving this branch's BYOK integration
(#728). Deliberately does NOT adopt #727's neuron-budget bump (1 -> 6):
the AI review path estimates by LOGICAL free-call count, not worst-case
retries, so a 6x reservation would be inconsistent and over-reserve,
causing premature quota_exceeded.
@JSONbored
JSONbored deleted the codex/fix-costly-ai-slop-calls-for-prs branch June 29, 2026 21:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant