feat(signals): skip scout suggestion scans on low-activity projects

frontend/snapshots.yml

@@ -9204,6 +9204,10 @@ snapshots:
         hash: v1.k794b7964.de465356e1d0c3d5df4e27a38f222f1272cd482864e94f5c0926496b21585156.JrxNWbA8ll95SmETdYceyyUM7OXvaVFv4m2ZNqLTEc0
     scenes-app-inbox-scoutsuggestionsstrip--last-scan-failed--light:
         hash: v1.k794b7964.862bc68a11983494741c4d1522c7a0b5a9dbc62fa096994e7ca39fcee696dc45.Gqy-Dr8yrhe7v2EdrEZFwjferFE9JMtvLBRegJF3Ebw
+    scenes-app-inbox-scoutsuggestionsstrip--low-activity--dark:
+        hash: v1.k794b7964.732c41df9211ce01adee553f66133484b9e5ea18986cf871c44f27fa88846754.DoN92tzMp60OUzfZtXPqKJ9UiDTVJC9sLc5nQVmjHzU
+    scenes-app-inbox-scoutsuggestionsstrip--low-activity--light:
+        hash: v1.k794b7964.9073f092aa4a347930e8468bb2f212bbdbe96a636a9fbce691cffa6caa493b07.YTmjGLtgppZg6VzdbUe-5cMUkxvsWUPRELqQMczJIc8
     scenes-app-inbox-scoutsuggestionsstrip--narrow--dark:
         hash: v1.k794b7964.d16cb21c4d654c63d5f8b6047b3291b636b25e21b00a91b06bd8384967f8a4ef.h9itg-CPeBGLngbSfw7_TIYGEJHnd_Z_sQ4VYIS5u0k
     scenes-app-inbox-scoutsuggestionsstrip--narrow--light:

products/signals/backend/management/AGENTS.md

@@ -148,7 +148,7 @@ Output buckets per team: `created`, `updated`, `diverged` (team-edited or hand-a
 
 ### Pre-computed scout suggestions
 
-`run_scout_suggestions` drives the headless "Suggested for this project" scan (`../scout_harness/suggestions.py`) without waiting for its Temporal coordinator. `--team-id` runs the scan inline for one team (bypassing the planner and its cap; the org still needs AI data processing approved) and prints the stored batch; `--show` prints the stored batch without running; `--plan` prints what the coordinator would dispatch on the next tick under the current `signals-scout-suggestions` flag payload.
+`run_scout_suggestions` drives the headless "Suggested for this project" scan (`../scout_harness/suggestions.py`) without waiting for its Temporal coordinator. `--team-id` runs the scan inline for one team (bypassing the planner and its cap; the org still needs AI data processing approved) and prints the stored batch; `--show` prints the stored batch without running; `--plan` prints what the coordinator would dispatch on the next tick under the current `signals-scout-suggestions` flag payload, and runs the dispatch-time activity check too, so it stamps `low_activity` on the projects it rules out exactly as the tick would.
 
 ```bash
 # Generate (or regenerate) the batch for one team now, then print it

products/signals/backend/management/commands/run_scout_suggestions.py

@@ -2,7 +2,9 @@
 
 `--team-id` runs the headless scan inline (bypassing the planner and its cap) and prints the
 stored batch. `--plan` prints what the coordinator would dispatch on the next tick under the
-current `signals-scout-suggestions` flag payload, without dispatching anything.
+current `signals-scout-suggestions` flag payload, without dispatching anything. `--plan` runs
+the dispatch-time activity check too, so it stamps `low_activity` on the projects it rules out,
+exactly as the tick would.
 """
 
 from __future__ import annotations
@@ -21,10 +23,12 @@
     plan_suggestion_runs,
     read_suggestion_settings,
     reserved_scout_names,
+    select_teams_to_scan,
     stamp_requested,
     visible_items,
 )
 from products.signals.backend.scout_harness.suggestions_runner import arun_scout_suggestions
+from products.signals.backend.temporal.agentic.scout_suggestions import SUGGESTIONS_OVERSELECT_FACTOR
 
 
 class Command(BaseCommand):
@@ -38,11 +42,16 @@ def add_arguments(self, parser):
     def handle(self, *args, **options):
         settings = read_suggestion_settings()
         if options["plan"]:
-            planned = plan_suggestion_runs(settings)
+            cap = settings.max_children_per_tick
+            planned = plan_suggestion_runs(settings, limit=cap * SUGGESTIONS_OVERSELECT_FACTOR)
+            selection = select_teams_to_scan(planned, settings, limit=cap)
             self.stdout.write(f"settings: {settings}")
-            self.stdout.write(f"planned {len(planned)} team(s):")
-            for run in planned:
+            self.stdout.write(f"planned {len(selection.dispatch)} team(s):")
+            for run in selection.dispatch:
                 self.stdout.write(f"  team {run.team_id} (tier {run.tier})")
+            self.stdout.write(f"skipped {len(selection.skipped_team_ids)} team(s) as low_activity:")
+            for team_id in selection.skipped_team_ids:
+                self.stdout.write(f"  team {team_id}")
             return
 
         if options["team_id"] is None:

products/signals/backend/migrations/0152_alter_signalscoutsuggestionset_status.py

@@ -0,0 +1,25 @@
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+    dependencies = [("signals", "0151_scout_config_retired_pause_reason")]
+
+    operations = [
+        # Choices only: `status` stays a CharField of the same length, so this emits no SQL.
+        migrations.AlterField(
+            model_name="signalscoutsuggestionset",
+            name="status",
+            field=models.CharField(
+                choices=[
+                    ("fresh", "Fresh"),
+                    ("stale", "Stale"),
+                    ("failed", "Failed"),
+                    ("empty", "Empty"),
+                    ("low_activity", "Low activity"),
+                ],
+                db_default="empty",
+                default="empty",
+                max_length=16,
+            ),
+        ),
+    ]

products/signals/backend/migrations/max_migration.txt

@@ -1 +1 @@
-0151_scout_config_retired_pause_reason
+0152_alter_signalscoutsuggestionset_status

products/signals/backend/models.py

@@ -437,7 +437,7 @@
             return self.signals_researched
         return max(self.signals_at_run - SIGNALS_AT_RUN_INCREMENT, 0)
 
     def transition_to(
         self,
         new_status: "SignalReport.Status",
         *,
@@ -2508,7 +2508,7 @@
         inactivity: set[str] = set(cls.INACTIVITY_PAUSE_REASONS)
         return current in inactivity and incoming in inactivity
 
     def transition_status_by_system(
         self,
         new_status: "SignalScoutConfig.Status",
         *,
@@ -3128,6 +3128,8 @@
         FAILED = "failed", "Failed"
         # The last generation completed and found nothing worth suggesting.
         EMPTY = "empty", "Empty"
+        # The project was too quiet in the activity window to be worth a scan, so none ran.
+        LOW_ACTIVITY = "low_activity", "Low activity"
 
     # See SignalScoutConfig.all_teams for rationale.
     all_teams = models.Manager()  # noqa: DJ012

products/signals/backend/scout_harness/AGENTS.md

@@ -36,7 +36,7 @@ In production it is driven by `SignalsScoutCoordinatorWorkflow` (periodic tick e
   The scout's `scout-display-name` is stamped the same way, and additionally backfilled onto existing rows by `reconcile_canonical_display_names` — every canonical config predates the key, so nothing else would ever give them a label. That pass writes only where `display_name` is blank, so it can run on every tick without ever undoing a rename.
   An **operational** scout (`scout-role: operational`) is the one exception to all of that: it seeds enabled and `auto_pause_exempt`, past the `enabled_skills` allowlist and past `MAX_ENABLED_SCOUTS_PER_TEAM` (it still counts toward the cap, so its slot is visible), and `reconcile_operational_configs` runs on every tick to put rows seeded before the role existed back on those terms. The reconcile undoes only the harness's own silencing — a sweep warning or pause via `transition_status_by_system`, and a row the seed created disabled, which the model stores as `paused_by_user` and the reconcile tells apart by its null `status_changed_at`. A person's pause and a `repeated_failures` pause are both left standing. The holdback still gates the scout, whatever its role, and the config API refuses to delete it (`views.destroy`).
 - `suggestions.py` / `suggestions_runner.py`
-  Pre-computed scout suggestions: the push-side twin of the "Suggest a scout" chat button. `suggestions.py` (temporalio-free, cheap to import) holds the structured-output contract the headless run returns (`ScoutSuggestionBatch`: 3-5 items, each a `canonical` "turn this on" pick or a `custom` draft with a ready-to-create `draft_body`), the `signals-scout-suggestions` flag-payload settings (`enabled`, `eligibility_tier` 0-4 (0 = allowlist only), `refresh_days`, `max_children_per_tick`, allow/blocklist, failure breaker), the planner (`plan_suggestion_runs`: recomputes the priority queue from Postgres every tick; a project that has never ingested an event is no candidate at any tier, because the scan could only refuse it, and the gate is read from `Team.ingested_event` on the project or any of its environments at plan time rather than stamped on the row, so the project re-enters the queue on its first event and an allowlisted project is scanned either way; tier, then never-generated, then most overdue, then most recently engaged; engagement is a report view or rating, a scout a person turned on or off (`status_changed_by`), or a scout a person created (`created_by`), never a plain edit or a system touch; nothing is stored as a queue; engagement is read only for the teams already in the tier map, since `SignalReportAction.last_at` is deliberately unindexed and an unbounded filter on it scans the whole action history; a team past the failure breaker doubles its refresh interval per further failure, because the cooldown is shorter than the refresh window and so could never hold a scheduled retry back), draft validation (`validate_suggestion_items` drops anything the one-click Create could not apply as-is), and persistence on `SignalScoutSuggestionSet` (one row per team, items as JSON, every write under a row lock since each is a read-modify-write of the column; dismissals carry forward by `skill_name` and survive as hidden tombstones when a batch omits the name, compacted to the dismissal fields so a dropped draft body does not sit in the row forever; a batch persisted after the fleet moved is stored `stale`, and a batch that merely aged past `refresh_days`, whether it held items or concluded there was nothing to suggest, reads as `stale` through `effective_status`, since nothing writes that status on expiry; a failed generation anywhere after the gates keeps the prior items and counts toward the breaker; `mark_stale_if_fleet_changed` is wired to `SignalScoutConfig` saves in `../receivers.py`). Reads hide dismissed, created, and already-enabled items, plus custom drafts whose name a stored skill or config has since taken; the manual refresh endpoint honors the same kill switch and blocklist as the planner, runs the scan as its authenticated caller rather than the resolved team member, and a manual dispatch stamps `last_requested_at` so the coordinator does not double up. `suggestions_runner.py` is the one piece that needs the tasks agent facade: `arun_scout_suggestions` gates (AI-processing consent, an acting user; deliberately no self-driving credits gate, since a scan opens no pull request and so bills nothing, while a skip would still burn the team's whole refresh window), mints a `read_only` headless task with the reserved `SIGNALS_SCOUT_SUGGESTIONS` origin, no GitHub token and no MCP Store servers, because the scan reads project text any member can write, parses the batch with no prose salvage (an unparseable close-out is a failed generation), validates, persists, and emits `$scout_suggestions_generated`. Driven by `temporal/agentic/scout_suggestions.py` (coordinator + child, same shape as the scout coordinator, own schedule) and exposed read/dismiss/refresh at `signals/scout/suggestions/` (`../scout_suggestions_api.py`), where membership, token scope, and resource-level RBAC all anchor to the canonical parent team. The prompt shares `SCOUT_PROJECT_SCAN_GUIDANCE` (in `prompt.py`) with the chat template so the two voices never drift.
+  Pre-computed scout suggestions: the push-side twin of the "Suggest a scout" chat button. `suggestions.py` (temporalio-free, cheap to import) holds the structured-output contract the headless run returns (`ScoutSuggestionBatch`: 3-5 items, each a `canonical` "turn this on" pick or a `custom` draft with a ready-to-create `draft_body`), the `signals-scout-suggestions` flag-payload settings (`enabled`, `eligibility_tier` 0-4 (0 = allowlist only), `refresh_days`, `max_children_per_tick`, allow/blocklist, failure breaker), the planner (`plan_suggestion_runs`: recomputes the priority queue from Postgres every tick; a project that has never ingested an event is no candidate at any tier, because the scan could only refuse it, and the gate is read from `Team.ingested_event` on the project or any of its environments at plan time rather than stamped on the row, so the project re-enters the queue on its first event and an allowlisted project is scanned either way; tier, then never-generated, then most overdue, then most recently engaged; engagement is a report view or rating, a scout a person turned on or off (`status_changed_by`), or a scout a person created (`created_by`), never a plain edit or a system touch; nothing is stored as a queue; engagement is read only for the teams already in the tier map, since `SignalReportAction.last_at` is deliberately unindexed and an unbounded filter on it scans the whole action history; a team past the failure breaker doubles its refresh interval per further failure, because the cooldown is shorter than the refresh window and so could never hold a scheduled retry back), the dispatch-time activity check (`select_teams_to_scan`: after the plan is made and before a child is dispatched, one bounded ClickHouse read per picked candidate over the project and its child environments together, counting events and distinct active days in `activity_window_days`, on the offline cluster, since a scheduled fleet scan must not compete with interactive queries (a Temporal activity keeps `Workload.ONLINE` unless it says otherwise, and `readonly=True` would return before the offline routing rather than add to it); a project under `min_events_in_window` or under `min_active_days_in_window` is stamped `low_activity` instead of scanned, because the scan could only refuse it, and either knob at 0 turns that half of the check off; the read is capped at `ACTIVITY_READ_MAX_ROWS` rows and refuses at the cap rather than truncating, because the cap counts rows read while the query counts the rows inside the window, so a truncated read would answer with a small count that reads like a quiet project; a refused read counts as active, since a project whose window does not fit the cap is active whatever the refusal hid; it carries a wall-clock cap too (`ACTIVITY_READ_MAX_EXECUTION_S`), because `sync_execute` sets none and the pooled client's socket timeout is effectively infinite in production, so a stalled read would hold a worker thread past the planning activity's deadline; a read that fails dispatches anyway, since the coordinator stamps `last_requested_at` either way and a quiet minute of ClickHouse must not cost a project its refresh window; allowlisted projects are scanned whatever the read says, and the manual refresh endpoint never runs the check, because a person who pressed the button asked for the scan; the coordinator therefore overselects the plan by `SUGGESTIONS_OVERSELECT_FACTOR` so a tick still fills when several candidates are skipped, and the skipped ones are stamped `last_requested_at` alongside the dispatched ones so the planner leaves them for a full refresh window rather than re-reading them every tick), draft validation (`validate_suggestion_items` drops anything the one-click Create could not apply as-is), and persistence on `SignalScoutSuggestionSet` (one row per team, items as JSON, every write under a row lock since each is a read-modify-write of the column; dismissals carry forward by `skill_name` and survive as hidden tombstones when a batch omits the name, compacted to the dismissal fields so a dropped draft body does not sit in the row forever; a batch persisted after the fleet moved is stored `stale`, and a batch that merely aged past `refresh_days`, whether it held items or concluded there was nothing to suggest, reads as `stale` through `effective_status`, since nothing writes that status on expiry; a failed generation anywhere after the gates keeps the prior items and counts toward the breaker; a `low_activity` stamp keeps the prior items the same way but leaves the breaker alone, since a quiet project is not a broken one: the count is preserved but ignored for scheduling while the status holds, so the backoff cannot push the next activity check out past one refresh window, and it counts again as soon as a scan runs; a quiet row also does not expire to `stale`, so the project waits a full refresh window and is then re-checked; `mark_stale_if_fleet_changed` is wired to `SignalScoutConfig` saves in `../receivers.py`). Reads hide dismissed, created, and already-enabled items, plus custom drafts whose name a stored skill or config has since taken; the manual refresh endpoint honors the same kill switch and blocklist as the planner, runs the scan as its authenticated caller rather than the resolved team member, and a manual dispatch stamps `last_requested_at` so the coordinator does not double up. `suggestions_runner.py` is the one piece that needs the tasks agent facade: `arun_scout_suggestions` gates (AI-processing consent, an acting user; deliberately no self-driving credits gate, since a scan opens no pull request and so bills nothing, while a skip would still burn the team's whole refresh window), mints a `read_only` headless task with the reserved `SIGNALS_SCOUT_SUGGESTIONS` origin, no GitHub token and no MCP Store servers, because the scan reads project text any member can write, parses the batch with no prose salvage (an unparseable close-out is a failed generation), validates, and persists. Both it and the activity check emit `$scout_suggestions_generated` through the one emitter in `suggestions.py` (`capture_suggestions_generated`), so a skipped project reads as `status=skipped`, `skip_reason=low_activity` plus the two measured numbers on the same event the completed runs use, and the daily roll-up needs no new column. The skip carries a stable event id, keyed on the project and the refresh stamp the decision was made against, because the planning activity retries and would otherwise count one skipped scan several times; a completed run needs none, since its activity runs at most once. Driven by `temporal/agentic/scout_suggestions.py` (coordinator + child, same shape as the scout coordinator, own schedule) and exposed read/dismiss/refresh at `signals/scout/suggestions/` (`../scout_suggestions_api.py`), where membership, token scope, and resource-level RBAC all anchor to the canonical parent team. The prompt shares `SCOUT_PROJECT_SCAN_GUIDANCE` (in `prompt.py`) with the chat template so the two voices never drift.
 - `slack_delivery.py` / `slack_delivery_queue.py` / `slack_charts.py`
   Best-effort direct Slack delivery for configured scout outputs. Finding emissions and surfaced report emits/edits snapshot the run config's destination after their database write commits, then enqueue the shared retrying Celery worker. The destination targets a single channel or up to `MAX_SCOUT_SLACK_DM_TARGETS` members who each get an individual DM (one Celery task per recipient, so retries and permanent failures stay independent; a group DM would need the `mpim:write` scope the Slack app doesn't request). Integrations are project-scoped, so a workspace connected from any environment in the project can receive the canonical parent team's scout output.
   A report message renders its `charts` through the exports facade (`render_png_export`, attributed to the run's acting user, so the insight access check applies; a system render that expires with its delivery url, so it stays out of the user's export quota and does not outlive its only reference) and appends them as `image` blocks pointing at a signed delivery url; chart links in the prose still reduce to their label. A threaded report carries its charts on the lead message, since the replies only hold the summary's tail. Only `InsightVizNode` / `SavedInsightNode` charts render, capped per report and by a render-time budget (a render only starts if it can finish inside the budget), with rendered assets remembered per delivery so a Slack retry does not re-render, and every failure is skipped rather than failing the delivery — the whole chart build is best effort, not only the render. The Slack integration row is resolved again after the build, since a workspace reconnected during the render window replaces the token on the same row.