🐛 fix(watcher): suppress watcher snapshot when container enumeration fails (#362)

A transient docker / socket-proxy hiccup on a remote agent
(\`this.getContainers()\` throws) was logged as a warning but the
watch cycle continued and \`event.emitWatcherSnapshot\` was still
called with \`containers: []\`. On the controller side that
snapshot is authoritative — \`AgentClient.handleWatcherSnapshotEvent\`
calls \`pruneOldContainers([], watcherName)\`, which deleted every
container for that agent's watcher from the controller's store.
The agent's own store was untouched, so the agent's next cron
still logged "25 containers watched", but the controller UI
showed 0 for that agent until the next clean cron cycle landed
or the controller was restarted.

The watcher now records whether enumeration failed and skips the
snapshot emit in that case so the controller keeps its last-known
state across transient enumeration errors. Per-container events
still fire normally for any containers that were successfully
enumerated; only the empty-list-prune-everything path is closed.

Author: scttbnsn

CHANGELOG.md

@@ -142,6 +142,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - **GHCR token fallback treats whitespace-only tokens as missing ([commit `711d583c`](https://github.com/CodesWhat/drydock/commit/711d583c)).** `getGhcrTokenFallback` and the registry suppression logic were testing `token.length > 0`, which treated a token value of `"   "` as a valid credential. All token checks now call `.trim().length > 0`. The GitHub release-notes provider also logs a warn when `api.github.com` rejects a configured token or GHCR fallback, instead of silently swallowing the auth failure.
 
+- **[#362](https://github.com/CodesWhat/drydock/issues/362) — Agents intermittently showing 0 running containers in the controller UI.** When the Docker watcher's container enumeration on an agent (`this.getContainers()`) threw — transient socket-proxy hiccup, Docker daemon restart, momentary network blip — the warning was logged but execution still fell through to `event.emitWatcherSnapshot({ containers: [] })`. The controller treats the snapshot as authoritative and `pruneOldContainers([], watcherName)` then wiped every container for that agent's watcher from the controller's store. The agent's own store was untouched (which is why its next cron still logged "25 containers watched"), but the controller UI showed 0 until either the next clean cron cycle landed or the controller was restarted (re-handshake re-fetched the agent's container list). The watcher now tracks an `containerEnumerationFailed` flag and suppresses the snapshot emission on failure so the controller preserves last-known state across transient enumeration errors.
+
 ### i18n (rc.20)
 
 - **14 new locales bundled ([discussion #329](https://github.com/CodesWhat/drydock/discussions/329)).** Italian, Spanish, German, French, Brazilian Portuguese, Dutch, Polish, Turkish, Japanese, Korean, Russian, Vietnamese, Ukrainian, and Arabic now ship with the same namespace coverage as the existing English / Simplified Chinese / Traditional Chinese catalogs. Initial machine-translation seeds were authored against the rc.17 catalog snapshot and then run through Crowdin QA — 28 real bugs were fixed via the Crowdin API (Italian accent recovery, capitalization mismatches, one zh-TW stray symbol) and ~700 false-positive brand/tech terms (Drydock, Trivy, Cosign, SBOM, GHCR, etc.) were added to the per-language Crowdin dictionaries to silence spellcheck noise. 17 locales total in the picker.

app/watchers/providers/docker/Docker.ts

@@ -1074,6 +1074,7 @@ class Docker extends Watcher<DockerWatcherConfiguration> {
   async watch() {
     this.ensureLogger();
     let containers: Container[] = [];
+    let containerEnumerationFailed = false;
     startDigestCachePollCycleForRegistries();
 
     // Dispatch event to notify start watching
@@ -1086,6 +1087,7 @@ class Docker extends Watcher<DockerWatcherConfiguration> {
       this.log.warn(
         `Error when trying to get the list of the containers to watch (${getErrorMessage(e)})`,
       );
+      containerEnumerationFailed = true;
     }
     try {
       if (this.isCronWatchInProgress) {
@@ -1120,15 +1122,22 @@ class Docker extends Watcher<DockerWatcherConfiguration> {
       }
       await event.emitContainerReports(containerReports);
       this.lastRunAt = new Date().toISOString();
-      await event.emitWatcherSnapshot({
-        watcher: {
-          type: this.type,
-          name: this.name,
-          configuration: this.maskConfiguration() as Record<string, unknown>,
-          metadata: this.getMetadata(),
-        },
-        containers: containerReports.map((containerReport) => containerReport.container),
-      });
+      // Skip the snapshot emit when container enumeration itself failed.
+      // The snapshot is authoritative — downstream prunes everything not in
+      // `containers` — so emitting an empty list after a transient docker /
+      // socket-proxy hiccup would wipe the controller's view of this agent
+      // (issue #362). Preserve last-known state until the next clean cycle.
+      if (!containerEnumerationFailed) {
+        await event.emitWatcherSnapshot({
+          watcher: {
+            type: this.type,
+            name: this.name,
+            configuration: this.maskConfiguration() as Record<string, unknown>,
+            metadata: this.getMetadata(),
+          },
+          containers: containerReports.map((containerReport) => containerReport.container),
+        });
+      }
       return containerReports;
     } finally {
       endDigestCachePollCycleForRegistries();

app/watchers/providers/docker/Docker.watch.test.ts

@@ -359,6 +359,29 @@ describe('Docker Watcher', () => {
       expect(event.emitWatcherStop).toHaveBeenCalledWith(docker);
     });
 
+    test('should emit watcher snapshot when container enumeration succeeds', async () => {
+      docker.getContainers = vi.fn().mockResolvedValue([]);
+
+      await docker.watch();
+
+      expect(event.emitWatcherSnapshot).toHaveBeenCalledTimes(1);
+      expect(event.emitWatcherSnapshot).toHaveBeenCalledWith(
+        expect.objectContaining({ containers: [] }),
+      );
+    });
+
+    test('should NOT emit watcher snapshot when container enumeration fails (issue #362)', async () => {
+      docker.log = createMockLog(['warn']);
+      docker.getContainers = vi.fn().mockRejectedValue(new Error('Docker unavailable'));
+
+      await docker.watch();
+
+      // Empty snapshot would prune every container for this agent on the
+      // controller side; emission is suppressed so the controller keeps its
+      // last-known state until the next clean cycle.
+      expect(event.emitWatcherSnapshot).not.toHaveBeenCalled();
+    });
+
     test('should set lastRunAt after watch completes', async () => {
       docker.getContainers = vi.fn().mockResolvedValue([]);
       expect(docker.lastRunAt).toBeUndefined();