feat(observability): translate browserless's JSON metrics into Prometheus text - #5436
Merged
Merged
Conversation
…heus text browserless's /metrics?token=<token> returns a rolling history of 5-minute- window JSON snapshots, not Prometheus text, so Prometheus can't scrape it directly. Adds a browserless-exporter sidecar (mirrors the existing backup-exporter "tiny BusyBox httpd wrapper" pattern) that polls it, takes the newest snapshot, and re-exposes queue depth, concurrency, rejection/error/ timeout counts, and CPU/memory as real Prometheus gauges. Adds the scrape target and a dedicated dashboard. Verified live on edge-nl-01: rebuilt the sidecar against the running browserless instance, confirmed /metrics returns real data (including a real browserless_unauthorized count from an earlier deliberate bad-token test), confirmed the Prometheus target shows up with zero scrape errors, and ran every dashboard PromQL expression against the live Prometheus API. Part of #5365, closes #5368
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5436 +/- ##
=======================================
Coverage 94.46% 94.46%
=======================================
Files 554 554
Lines 44426 44426
Branches 14663 14663
=======================================
Hits 41967 41967
Misses 1784 1784
Partials 675 675
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/metrics?token=<token>endpoint returns a rolling history of 5-minute-window JSON snapshots (confirmed live: 1099 entries spanning ~3.8 days), not Prometheus text format, so Prometheus can't scrape it directly.browserless-exportersidecar (scripts/browserless-metrics.sh) mirroring the existingbackup-exporter"tiny BusyBox httpd wrapper around Prometheus text output" pattern already proven in this repo — polls the JSON endpoint, takes only the newest snapshot (the array's last element, since entries are oldest-first), and re-exposes its fields as Prometheus gauges (not counters — each field is a per-5-minute-window sample from browserless's own rolling buffer, not a monotonic total, so it can legitimately go up or down between scrapes).browserless_queued/browserless_running/browserless_max_concurrent(queue/concurrency),browserless_rejected/browserless_errors/browserless_timedout/browserless_unauthorized/browserless_unhealthy/browserless_successful(per-window outcomes),browserless_cpu_ratio/browserless_memory_ratio/browserless_session_mean_time_ms(resource/latency), plusbrowserless_exporter_last_scrape_success+browserless_sample_timestamp_secondsso an operator can distinguish "the exporter itself can't reach browserless" from "browserless has stopped producing new windows."last_scrape_success 0gracefully, verified in isolation with a deliberately wrong token.grafana/dashboards/browserless-metrics.json: exporter health, queue depth/concurrency, session outcomes, and CPU/memory utilization.--profile visual-review,deploy.resources.limits.memory,.env.exampledocumentation.Verified live end to end on edge-nl-01, not just structural validation:
browserless-exporterservice against the livebrowserlesscontainer — confirmed/metricsreturns real data, including a genuinebrowserless_unauthorizedcount picked up from the isolated bad-token test above (proof the numbers are real, not just self-consistent).up, zero scrape errors.status: success.Part of #5365. Closes #5368.
Test plan
npm run typechecknpm run test:ci(full local gate: 15105 passed / 12 skipped)npm audit --audit-level=moderate— 0 vulnerabilitiesnode scripts/validate-observability-configs.mjsshellcheck -s sh scripts/browserless-metrics.sh— clean