Skip to content

fix: spanner metrics regression#2329

Merged
miparnisari merged 1 commit intomainfrom
spanner-metrics-regression
Feb 20, 2026
Merged

fix: spanner metrics regression#2329
miparnisari merged 1 commit intomainfrom
spanner-metrics-regression

Conversation

@vroldanbet
Copy link
Copy Markdown
Contributor

@vroldanbet vroldanbet commented Apr 15, 2025

Description

When using --datastore-spanner-metrics=otel, none of the Spanner metrics were being emmitted. This PR fixes that.

The SDK has deprecated OpenCensus support, which was wired into the prometheus endpoint, in favour of OpenTelemetry. This was reflected in a recent SpiceDB change, which made the otel approach the default, and as a consequence all prometheus metrics disappeared.

This commit adds the prometheus exporter as a OpenTelemetry MetricsResolver, to add some the metrics back. The naming convention has changed, and I'm not aware of a way around that.

Also client-side latency metrics are not supported via the opentelemetry integration. See https://cloud.google.com/spanner/docs/capture-custom-metrics-opentelemetry#capture-client-round-trip-latency Instead they now have to be manually instrumented.

Testing

Try docker-compose -f docker-compose.spanner.yaml up --build and go to http://localhost:3000/. You should see the metrics:

spanner_gfe_header_missing_count_total
spanner_max_allowed_sessions_ratio
spanner_max_in_use_sessions_ratio
spanner_num_acquired_sessions_total
spanner_num_released_sessions_total
spanner_num_sessions_in_pool_ratio
spanner_open_session_count_ratio

If you set in the Docker-compose file --datastore-spanner-metrics=deprecated-prometheus, you get the same metrics but with the prefix spicedb_cloud_google_com_go_.

@github-actions github-actions bot added area/datastore Affects the storage system area/dependencies Affects dependencies labels Apr 15, 2025
@vroldanbet vroldanbet force-pushed the spanner-metrics-regression branch from 3c7cba0 to 8e8dc92 Compare April 15, 2025 15:45
@tstirrat15 tstirrat15 force-pushed the spanner-metrics-regression branch from 8e8dc92 to 5af1197 Compare November 21, 2025 17:16
@codecov
Copy link
Copy Markdown

codecov bot commented Nov 21, 2025

Codecov Report

❌ Patch coverage is 60.00000% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.59%. Comparing base (c3b1bdc) to head (cfc12dc).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
internal/datastore/spanner/spanner.go 58.34% 17 Missing and 3 partials ⚠️

❌ Your project status has failed because the head coverage (74.59%) is below the target coverage (75.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2329      +/-   ##
==========================================
+ Coverage   74.54%   74.59%   +0.05%     
==========================================
  Files         484      484              
  Lines       59201    59228      +27     
==========================================
+ Hits        44126    44173      +47     
+ Misses      11982    11969      -13     
+ Partials     3093     3086       -7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@miparnisari miparnisari force-pushed the spanner-metrics-regression branch from 5af1197 to 2a7dba4 Compare February 20, 2026 02:28
@github-actions github-actions bot added the area/cli Affects the command line label Feb 20, 2026
@miparnisari miparnisari force-pushed the spanner-metrics-regression branch from 2a7dba4 to 7600fd6 Compare February 20, 2026 02:28
@miparnisari miparnisari changed the title fix spanner metrics regression fix: spanner metrics regression Feb 20, 2026
@miparnisari miparnisari force-pushed the spanner-metrics-regression branch 3 times, most recently from 840fca8 to c6b094c Compare February 20, 2026 02:48
@github-actions github-actions bot added the area/tooling Affects the dev or user toolchain (e.g. tests, ci, build tools) label Feb 20, 2026
recent changes mean client-side spanner metrics
are no longer exposed the prometheus scrape endpoint.

The SDK has deprecated OpenCensus support, which was
wired into the prometheus endpoint, in favour of OpenTelemetry.
This was reflected in a recent SpiceDB change, which made the otel
approach the default, and as a consequence all prometheus metrics
disappeared.

This commit adds the prometheus exporter as a OpenTelemetry MetricsResolver,
to add some the metrics back. The naming convention has changed, and I'm not aware
of a way around that.

Also client-side latency metrics are not supported via the opentelemetry integration.
See https://cloud.google.com/spanner/docs/capture-custom-metrics-opentelemetry#capture-client-round-trip-latency
Instead they now have to be manually instrumented.
@miparnisari miparnisari force-pushed the spanner-metrics-regression branch from c6b094c to cfc12dc Compare February 20, 2026 03:01
@miparnisari miparnisari marked this pull request as ready for review February 20, 2026 03:09
@miparnisari miparnisari requested a review from a team as a code owner February 20, 2026 03:09
@miparnisari miparnisari enabled auto-merge (squash) February 20, 2026 14:51
Copy link
Copy Markdown
Contributor

@tstirrat15 tstirrat15 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@miparnisari miparnisari merged commit 1d65cbb into main Feb 20, 2026
42 of 44 checks passed
@miparnisari miparnisari deleted the spanner-metrics-regression branch February 20, 2026 16:29
@github-actions github-actions bot locked and limited conversation to collaborators Feb 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area/cli Affects the command line area/datastore Affects the storage system area/dependencies Affects dependencies area/tooling Affects the dev or user toolchain (e.g. tests, ci, build tools)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants