Skip to content

site(status): public service-status endpoint, sourced from the existing Alertmanager #9983

Description

@JSONbored

The backend slice of #9747, taking option C — source status from the existing PagerDuty-integrated alerting stack rather than adding new probes or a new schema.

It needs no new infrastructure

I checked the box rather than assuming. Three facts make C cheap:

  1. The app container already reaches Alertmanager. From inside loopover-loopover-1:
    fetch("http://alertmanager:9093/api/v2/alerts")[]. Grafana's /api/health answers too. Same compose network, no new egress.
  2. The tunnel already routes it. /etc/cloudflared/config.yml maps ^/v1/public/.*$ on shots.loopover.ai to localhost:8787, so any new /v1/public/* route is publicly reachable the moment it ships — no tunnel edit, no new hostname (which matters: there is no cert.pem on the box, so a new hostname could not be minted there anyway).
  3. The alerting is already PagerDuty-integrated. alertmanager.yml routes to oncall-pager and team-slack receivers, with rules managed in Grafana. That is exactly the "existing health/alerting instrumentation" site(status): public service-status surface — ORB and AMS fleet heartbeats and uptime #9747 says to reuse.

Scope

  • GET /v1/public/service-status — unauthenticated, public-safe: component name, status, and since-timestamp. No hostnames, no capacity, no alert internals, no instance ids.
  • Component status derived from active Alertmanager alerts mapped to components.
  • Unreachable Alertmanager reports unknown, never operational. A status page that shows green because it could not check is worse than no status page — the same false-green class as ci(verifiability): nightly anonymous walkthrough run — fail loudly on any public-path regression #9724's all-skip run.
  • Serves 404 where no Alertmanager is configured (the hosted Worker), rather than publishing a meaningless all-unknown board.

Explicitly NOT in this slice

Uptime percentages and incident history. Alertmanager's /api/v2/alerts returns active alerts only; there is no Prometheus on the box (metrics go through the otel-collector) and no status-sample table, so there is no source for "has it been healthy?" today. Fabricating an uptime figure from a single live sample would be the dishonest option.

The follow-up slice is a service_status_samples table written on the existing cron, with uptime and incidents derived from contiguous non-operational runs — so incidents appear without manual posting, and history accrues from deploy forward. Filed separately so this endpoint can ship and be useful now.

Acceptance

  • A public URL answers "is each component healthy right now?"
  • An unreachable or unconfigured alerting source is reported honestly, never as healthy.
  • Nothing in the payload identifies a host, an instance, or capacity.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions