[nexus] Project-scoped OxQL endpoint

[david-crespo]

#7047 moves the existing /v1/timeseries/query endpoint (which requires the fleet viewer role) to /v1/system/timeseries/query. This PR is on top of #7047 and adds /v1/timeseries/query?project=my-project, using a sneaky trick to let us ensure that the user has read permissions on a project before we let them see metrics for that project. See #5298 (comment) for a discussion of the OxQL authz problem more broadly.

• Add /v1/timeseries/query with required project query param
• Integration tests showing the authz works in an expected scenarios
• Add a list schemas endpoint that only lists schemas with a project_id field
• Move schema filtering logic inside oximeter client
• Fully test schema list endpoint

The trick

1. Require the user to say (in a query param) what project they're interested in
2. Look up that project and make sure they can read it
3. Jam | filter silo_id == "<silo_id>" && project_id == "<project_id>" on the end of whatever query they passed in

It sounds silly, but I talked it over with @bnaecker and we couldn't find any holes. If the user tries to fetch metrics from another project inside their query, the query will end up with the filter project_id == "def" && project_id == "xyz" and the result set will always be empty. If they try to query a metric that doesn't have a project_id on it, it will error out. It works!

API design

Initially I had the endpoint as /v1/timeseries/query/project/{project}. This is really horrible. It should be /projects/, but that doesn't feel any better. I also considered /v1/projects/{project}/timeseries/query, which has some precedent:

omicron/nexus/external-api/output/nexus_tags.txt

Lines 59 to 64 in 45f5f1c

	instance_reboot POST /v1/instances/{instance}/reboot
	instance_serial_console GET /v1/instances/{instance}/serial-console
	instance_serial_console_stream GET /v1/instances/{instance}/serial-console/stream
	instance_ssh_public_key_list GET /v1/instances/{instance}/ssh-public-keys
	instance_start POST /v1/instances/{instance}/start
	instance_stop POST /v1/instances/{instance}/stop

But it also feels awful.

I like the query param approach. Right now, there are only fleet-scoped metrics and project-scoped metrics, so requiring the project ID makes sense, but the query params are nicely flexible in that we could make project optional or add other optional fields and just do different auth checks based on what's passed in. Neither path nor operation ID mention projects.

Examples of fleet- and project-scoped metrics

omicron/oximeter/oximeter/schema/virtual-machine.toml

Lines 3 to 18 in 45f5f1c

	[target]
	name = "virtual_machine"
	description = "A guest virtual machine instance"
	authz_scope = "project"
	versions = [
	{ version = 1, fields = [ "instance_id", "project_id", "silo_id", "sled_id", "sled_model", "sled_revision", "sled_serial" ] },
	]
	[[metrics]]
	name = "vcpu_usage"
	description = "Cumulative time each vCPU has spent in a state"
	units = "nanoseconds"
	datum_type = "cumulative_u64"
	versions = [
	{ added_in = 1, fields = [ "state", "vcpu_id" ] }
	]

omicron/oximeter/oximeter/schema/switch-data-link.toml

Lines 3 to 19 in 45f5f1c

	[target]
	name = "switch_data_link"
	description = "A network data link on an Oxide switch"
	authz_scope = "fleet"
	versions = [
	{ version = 1, fields = [ "rack_id", "sled_id", "sled_model", "sled_revision", "sled_serial", "switch_id", "switch_fab", "switch_lot", "switch_wafer", "switch_wafer_loc_x", "switch_wafer_loc_y", "switch_model", "switch_revision", "switch_serial", "switch_slot" ] },
	]
	[[metrics]]
	name = "bytes_sent"
	description = "Total number of bytes sent on the data link"
	units = "bytes"
	datum_type = "cumulative_u64"
	versions = [
	{ added_in = 1, fields = [ "port_id", "link_id" ] }
	]

No list endpoint yet

I backed out the schema list endpoint. We can't list project-scoped schemas because authz_scope is not in the database! See #5942. Currently the schema list endpoint hard-codes authz_scope: Fleet.

omicron/oximeter/db/src/model/from_block.rs

Line 142 in 69de8b6

authz_scope: AuthzScope::Fleet,

I am using the tag hidden on the query endpoint so that it goes in the OpenAPI definition (so I can use it in the console) but it will not show up on the docs site.

[david-crespo]

This is the crux of the PR.

[bnaecker]

Looks good, thanks for doing this! One small comment, but otherwise ready to go I think.

[bnaecker]

I think we might need to put the Connection error variant in here too. We get that when qorb fails to loan out a connection from its pool.

[david-crespo]

Thanks, I'll rebase on main and do that.