Skip to content

feat(observability): translate browserless's JSON metrics into Prometheus text - #5436

Merged
JSONbored merged 1 commit into
mainfrom
feat/browserless-metrics-translator
Jul 12, 2026
Merged

feat(observability): translate browserless's JSON metrics into Prometheus text#5436
JSONbored merged 1 commit into
mainfrom
feat/browserless-metrics-translator

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • browserless's own /metrics?token=<token> endpoint returns a rolling history of 5-minute-window JSON snapshots (confirmed live: 1099 entries spanning ~3.8 days), not Prometheus text format, so Prometheus can't scrape it directly.
  • Adds a browserless-exporter sidecar (scripts/browserless-metrics.sh) mirroring the existing backup-exporter "tiny BusyBox httpd wrapper around Prometheus text output" pattern already proven in this repo — polls the JSON endpoint, takes only the newest snapshot (the array's last element, since entries are oldest-first), and re-exposes its fields as Prometheus gauges (not counters — each field is a per-5-minute-window sample from browserless's own rolling buffer, not a monotonic total, so it can legitimately go up or down between scrapes).
  • Metrics: browserless_queued/browserless_running/browserless_max_concurrent (queue/concurrency), browserless_rejected/browserless_errors/browserless_timedout/browserless_unauthorized/browserless_unhealthy/browserless_successful (per-window outcomes), browserless_cpu_ratio/browserless_memory_ratio/browserless_session_mean_time_ms (resource/latency), plus browserless_exporter_last_scrape_success + browserless_sample_timestamp_seconds so an operator can distinguish "the exporter itself can't reach browserless" from "browserless has stopped producing new windows."
  • A failed poll doesn't corrupt the served metrics (atomic tmp+mv write) and doesn't crash the exporter — it degrades to last_scrape_success 0 gracefully, verified in isolation with a deliberately wrong token.
  • New grafana/dashboards/browserless-metrics.json: exporter health, queue depth/concurrency, session outcomes, and CPU/memory utilization.
  • Gated the same way as browserless itself: --profile visual-review, deploy.resources.limits.memory, .env.example documentation.

Verified live end to end on edge-nl-01, not just structural validation:

  1. Smoke-tested the script in isolation (both success and deliberate-bad-token failure paths) before touching the live compose stack.
  2. Deployed the real browserless-exporter service against the live browserless container — confirmed /metrics returns real data, including a genuine browserless_unauthorized count picked up from the isolated bad-token test above (proof the numbers are real, not just self-consistent).
  3. Applied the scrape config to live Prometheus — target shows up, zero scrape errors.
  4. Ran all 14 of the new dashboard's PromQL expressions directly against the live Prometheus API — all status: success.
  5. Confirmed the dashboard provisions into Grafana (10 panels) and that Grafana's own datasource proxy returns real query results.

Part of #5365. Closes #5368.

Test plan

  • npm run typecheck
  • npm run test:ci (full local gate: 15105 passed / 12 skipped)
  • npm audit --audit-level=moderate — 0 vulnerabilities
  • node scripts/validate-observability-configs.mjs
  • shellcheck -s sh scripts/browserless-metrics.sh — clean
  • Live verification on edge-nl-01 as described above

…heus text

browserless's /metrics?token=<token> returns a rolling history of 5-minute-
window JSON snapshots, not Prometheus text, so Prometheus can't scrape it
directly. Adds a browserless-exporter sidecar (mirrors the existing
backup-exporter "tiny BusyBox httpd wrapper" pattern) that polls it, takes the
newest snapshot, and re-exposes queue depth, concurrency, rejection/error/
timeout counts, and CPU/memory as real Prometheus gauges. Adds the scrape
target and a dedicated dashboard.

Verified live on edge-nl-01: rebuilt the sidecar against the running
browserless instance, confirmed /metrics returns real data (including a real
browserless_unauthorized count from an earlier deliberate bad-token test),
confirmed the Prometheus target shows up with zero scrape errors, and ran
every dashboard PromQL expression against the live Prometheus API.

Part of #5365, closes #5368
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jul 12, 2026
@codecov

codecov Bot commented Jul 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.46%. Comparing base (156b382) to head (084be6b).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5436   +/-   ##
=======================================
  Coverage   94.46%   94.46%           
=======================================
  Files         554      554           
  Lines       44426    44426           
  Branches    14663    14663           
=======================================
  Hits        41967    41967           
  Misses       1784     1784           
  Partials      675      675           
Flag Coverage Δ
shard-1 44.11% <ø> (ø)
shard-2 34.76% <ø> (ø)
shard-3 31.59% <ø> (ø)
shard-4 31.41% <ø> (ø)
shard-5 33.28% <ø> (+<0.01%) ⬆️
shard-6 43.37% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 12, 2026
@JSONbored
JSONbored merged commit 55f8fd1 into main Jul 12, 2026
18 checks passed
@JSONbored
JSONbored deleted the feat/browserless-metrics-translator branch July 12, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(observability): translate browserless's JSON /metrics into real Prometheus text format

1 participant