Skip to content

fix(metrics): report concurrency from the semaphores that gate traffic - #1553

Open
devogechukwu wants to merge 2 commits into
CalloraOrg:mainfrom
devogechukwu:fix/admin-concurrency-metrics
Open

devogechukwu wants to merge 2 commits into
CalloraOrg:mainfrom
devogechukwu:fix/admin-concurrency-metrics

Conversation

@devogechukwu

@devogechukwu devogechukwu commented Oct 24, 2026 •

Copy link
Copy Markdown

Closes #1504

Summary

The admin concurrency endpoint previously read from sharedDeveloperSemaphore, but the actual billing middleware instantiated an isolated DeveloperSemaphore when constructed with config parameters. Additionally, billingConcurrencySemaphore in BillingService serialises deductions per user and was not visible.

This PR introduces a central ConcurrencyRegistry where limiters (billing.route, billing.service, etc.) register themselves, exposes them to Prometheus as bounded-cardinality gauges (concurrency_active_slots and concurrency_queued), surfaces them in GET /api/admin/metrics/concurrency, and ensures GET /api/admin/keys/concurrency reports { registered: false } when no key limiter is active.

Changes

  1. Concurrency Registry (src/services/concurrencyRegistry.ts):
    • Limiters register by name (billing.route, billing.service, etc.).
    • Observes slot acquire/release events and automatically updates Prometheus gauges concurrency_active_slots{limiter} and concurrency_queued{limiter}.
    • Provides a clear() method for test isolation.
  2. App Concurrency Wiring (src/app.ts):
    • createApp creates the billing.route semaphore once, registers it with concurrencyRegistry, and passes it into createPerDevConcurrencyMiddleware.
  3. Billing Service Registration (src/services/billing.ts):
    • Registers billingConcurrencySemaphore under billing.service.
  4. Admin Metrics Endpoint (src/routes/admin/metrics.ts):
    • Reports all registered limiters separately under data.limiters, including active, queued, limit, and per-developer breakdowns.
    • Retains legacy response fields (totalActive, devCounts, maxConcurrencyPerDeveloper) for backwards compatibility.
  5. Key Concurrency Endpoint (src/routes/admin/keys/concurrency.ts):
    • Returns { registered: false } when gateway.key is not registered.
  6. Documentation & Tests:
    • Updated docs/per-dev-concurrency.md and docs/per-key-concurrency.md.
    • Added unit and integration tests in src/routes/admin/metrics.test.ts, src/routes/admin/keys/concurrency.test.ts, and src/middleware/perDevConcurrency.test.ts.

Sample Responses & Metrics Scrape

GET /api/admin/metrics/concurrency during in-flight billing request:

{
  "data": {
    "devCounts": {
      "dev_123": 1
    },
    "totalActive": 1,
    "maxConcurrencyPerDeveloper": 1,
    "limiters": {
      "billing.route": {
        "active": 1,
        "queued": 0,
        "limit": 1,
        "type": "developer",
        "counts": {
          "dev_123": 1
        }
      },
      "billing.service": {
        "active": 0,
        "queued": 0,
        "limit": 1,
        "type": "developer",
        "counts": {}
      }
    },
    "campaign": "GrantFox FWC26"
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make /api/admin/metrics/concurrency read the semaphore that actually gates billing requests

1 participant