Skip to content

feat(signals): drop the Signals credits quota gate from scout suggestion scans #94166

Closed

Description

@andrewm4894

Problem

The pre-computed scout suggestions runner (products/signals/backend/scout_harness/suggestions_runner.py, _gate_skip_reason) skips a project when its organization is over the Signals credits quota (is_team_signals_quota_limited). The manual refresh endpoint (scout_suggestions_api.py, refresh) throttles on the same check.

That gate was carried over from the scout scheduler, where it makes sense: a scout run exists to feed signals into the pipeline, and ingestion drops them while the limit binds. A suggestion scan is different. It is an internal run we do as part of self-driving. Its output is a stored batch the scouts tab reads, not signals, and it never bills the customer: Signals credits are charged only per report whose implementation ships a PR (SIGNALS_CREDITS_PER_REPORT_WITH_PR in products/signals/backend/billing.py). Nothing is lost by scanning an over-quota project, and the quota is the moment a project most needs a good "what to turn on next" list for when the limit lifts.

The skip also costs more than it looks. The coordinator stamps last_requested_at when it dispatches the child, and a skipped run never touches the row, so a quota-limited project is held for the full refresh_days (7 days by default) even when its billing period resets the next day. In the first tier-1 wave on 2026-09-03, 3 of 10 planned projects were skipped this way and are now held until 2026-09-10.

Proposal

Drop the Signals credits quota gate from the suggestions path. Keep the AI data processing consent gate and the acting-user gate as they are.

  • suggestions_runner.py: remove the quota_limited branch from _gate_skip_reason. The SuggestionRunResult.skip_reason value quota_limited stops being produced; the $scout_suggestions_generated telemetry keeps the field for the remaining reasons.
  • scout_suggestions_api.py refresh: remove the Throttled on is_team_signals_quota_limited. The daily refresh cap (SUGGESTIONS_REFRESH_DAILY_CAP) stays as the abuse guard, and the kill switch and blocklist checks stay.
  • products/signals/backend/scout_harness/AGENTS.md: the arun_scout_suggestions bullet lists "signals quota" among the gates. Update it.
  • Tests: add a case in products/signals/backend/test/test_scout_suggestions.py where the quota check returns limited and the scan still runs, and the matching refresh-endpoint case.

Out of scope here, worth a follow-up: a skipped run should not consume the refresh window. The cleanest shape is for a skip to clear last_requested_at (or to re-plan after failure_cooldown_hours instead of refresh_days), so the remaining skip reasons (ai_data_processing_not_approved, no_active_user) do not hold a project for a week after the condition clears.

Rollout

No flag work: the change ships dark until the next coordinator tick plans a project that would have been skipped. The rows already stamped by a quota skip need one manual nudge after deploy, either POST /api/projects/:id/signals/scout/suggestions/refresh/ or python manage.py run_scout_suggestions --team-id <id>, since the planner will otherwise revisit them only at the end of their refresh window.

Context

  • Producer shipped in feat(signals): pre-compute suggested scouts per project #85963. Flag payload knobs and the planner are described in products/signals/backend/scout_harness/AGENTS.md.
  • The scout scheduler's quota skip (temporal/agentic/scout_scheduler.py) and the pipeline gates in quota.py are unchanged by this issue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions