api: route per-call against unified hosts by simonfaltum · Pull Request #5137 · databricks/cli

simonfaltum · 2026-04-30T09:11:35Z

Why

databricks api always sent the workspace routing identifier (X-Databricks-Org-Id) when the profile had one, even when the path was an account API. On unified hosts (one host serving both workspace and account APIs) this misrouted account calls. There was also no way to explicitly route a call to the account API or override the identifier per call.

Changes

Before: routing was decided once from the profile and applied to every call.
Now: routing is decided per call from the path being requested.

Paths under /accounts/{id}/ are auto-detected as account-scope; the routing identifier is dropped.
A small hand-written list in cmd/api/paths.go carves out workspace-routed proxy APIs that happen to live under /accounts/, so they keep the identifier.
--account forces account-scope on a non-/accounts/ path.
--workspace-id <id> overrides the identifier per call. Mutually exclusive with --account.
?o=<id> on the path (the SPOG URL convention used by the Databricks UI) is recognized as a per-call workspace override, so URLs pasted from the browser route correctly.
The CLI-only workspace_id = none sentinel is stripped before the routing decision so the literal "none" never goes on the wire.

Routing logic lives in pure functions (hasAccountSegment, extractOrgIDFromQuery, resolveOrgID, normalizeWorkspaceID, isWorkspaceProxyPath) that take primitives. The cobra RunE is a thin adapter that resolves config and calls them.

Test plan

go test ./cmd/api covers the helpers with table-driven cases: deny-list hits and misses, query/fragment edge cases, mutual-exclusion errors, sentinel stripping, ?o= extraction.
go test ./acceptance -run TestAccept/cmd/api exercises seven variants end to end against terraform and direct engines: workspace path, account path, deny-listed proxy under /accounts/, --account, --workspace-id, ?o= query, workspace_id = none. Each test asserts header presence/absence explicitly via print_requests.py | contains.py.
make checks

The next PR teaches `databricks api` to detect workspace-vs-account scope per call. That decision needs a deny-list of paths under accounts/ that the SDK builds without an account-ID slot (workspace proxies). Hand-maintaining that list drifts from the SDK; this commit generates it. genpaths walks every service/*/impl.go in the pinned SDK with go/ast, classifies each `path :=` assignment, and emits cmd/api/paths_generated.go with a closed allowlist on account-ID source spellings. Refuses to emit prefixes that would over-match, fails loudly on idioms it doesn't recognize, handles var/define/assign forms and rejects compound assignments. Hooked into ./task generate-paths and the existing generated-files staleness gate in CI. The generated tables are not yet referenced at runtime; the next PR wires them in. Generated-file lint exclusions (lax rules) cover the unused declarations until then. Co-authored-by: Isaac

- Drop stdlib log in favor of fmt.Fprintf+os.Exit for the generator binary (depguard rule against the stdlib log package). - Make the path-class switch exhaustive by listing the no-op cases. - Lowercase the format-verb error string (staticcheck ST1005). Co-authored-by: Isaac

Keep the unified-host routing support small for the initial PR by hand-maintaining the current workspace-proxy exceptions instead of parsing generated SDK source.

`databricks api <verb> <path>` previously bypassed the generated SDK header logic and called client.Do directly, so on unified hosts where workspace-vs-account routing is decided per-call it had no way to distinguish the two. This wires up per-call detection using the deny-list from the prior PR, plus three explicit overrides: --account scope this call to the account API --workspace-id <id> override the workspace routing identifier {account_id} literal substituted from the active profile Detection runs on URL.Path so query strings and fragments can't false-match. The CLI-only WorkspaceIDNone sentinel (workspace_id = none in .databrickscfg) is normalized to empty before the SDK's idiomatic check sees it, so the literal "none" never goes on the wire. Behavior change for classic workspace profiles that have workspace_id set: the routing identifier is now sent. Classic gateways ignore the header so this should be benign; called out in the manual smoke plan in case it surfaces. Co-authored-by: Isaac

Keep the manual workspace-proxy list behind one helper so tests exercise the same path used by runtime account detection.

Add the standard OS and CI/cicd replacements to acceptance/cmd/api/test.toml and regenerate the recorded User-Agent strings to use os/[OS]. Without these, goldens generated locally on macOS contain os/darwin and no cicd/ segment, which fail on Linux + GitHub Actions where the SDK records os/linux ... cicd/github. Co-authored-by: Isaac

SPOG URLs from the Databricks UI carry the workspace ID as a query param (e.g. /api/2.2/jobs/list?o=7474644166319138). Recognize that param when present and use it as the routing identifier so pasted URLs route correctly without requiring --workspace-id. Precedence: --account > --workspace-id flag > ?o= > account-path auto-detect > profile workspace_id. Co-authored-by: Isaac

…' into simonfaltum/cli-api-spog-routing

…pog-routing

Pull this out of the per-call routing PR per review feedback. The routing change does not need account-ID interpolation to land; if we want it later it can be a focused follow-up alongside any other path-substitution we decide to support. Co-authored-by: Isaac

…citly Each test now uses print_requests.py | contains.py to assert the recorded request either does or does not carry the routing identifier. The intent (header sent or not, with what value) is now visible in the script and in output.txt, instead of being implied by the recorded out.requests.txt artifact alone. Co-authored-by: Isaac

pietern · 2026-05-04T13:46:09Z

@@ -0,0 +1 @@
+$CLI api get /api/2.0/clusters/list --account


This now dups the requests between out.requests.txt and output.txt. Can you keep one of them?

print_requests.py --keep was preserving out.requests.txt alongside the new inline assertion in output.txt, duplicating the recorded request. Drop --keep so the helper consumes out.requests.txt and the assertion stays in output.txt only. Co-authored-by: Isaac

…pog-routing

Setting MSYS_NO_PATHCONV=1 in the parent test.toml [Env] block fixed Git Bash rewriting /api/... to a Windows path, but it also broke the python helpers invoked on the next line (print_requests.py, contains.py): with the variable set globally, Git Bash passes script paths in MSYS form to python.exe, which then can't open them. Drop the [Env] block and prefix MSYS_NO_PATHCONV=1 inline on each \$CLI api ... call, matching the pattern already used by envsubst() in acceptance/script.prepare. Co-authored-by: Isaac

main removed AuthArguments.IsUnifiedHost in favor of DiscoveryURL-based routing (#5137). Update TestBuildLoginCommand_AppendsWorkspaceID's unified-host case to use a discovery URL containing /oidc/accounts/<id>/ so the test still exercises UnifiedOAuthArgument. Co-authored-by: Isaac

Drop the bespoke resolveWorkspaceID helper and the cached wsID field on lakeboxAPI. Match the minimal pattern that libs/telemetry, libs/filer, and SDK-generated workspace services already use: read cfg.WorkspaceID directly, send the X-Databricks-Org-Id header if set. Removes the '?o=<id>' fallback that parsed the host's query string. That behavior was unique to lakebox and inconsistent with how every other CLI surface handles SPOG hosts; the SDK's host-metadata discovery populates cfg.WorkspaceID for hosts that need it, and users who run into edge cases set workspace_id explicitly the same way they would for `bundle deploy` or `databricks api`. Adds the auth.WorkspaceIDNone ("none") sentinel strip so a profile created via `databricks auth login` for SPOG account-level access doesn't send the literal string "none" as the routing identifier. This fix matches what cmd/api/api.go (#5137) and libs/auth do; the four other orgIDHeaders helpers in the codebase still have the latent bug, which is a separate cleanup. Co-authored-by: Isaac

## Why `databricks api` always sent the workspace routing identifier (`X-Databricks-Org-Id`) when the profile had one, even when the path was an account API. On unified hosts (one host serving both workspace and account APIs) this misrouted account calls. There was also no way to explicitly route a call to the account API or override the identifier per call. ## Changes Before: routing was decided once from the profile and applied to every call. Now: routing is decided per call from the path being requested. - Paths under `/accounts/{id}/` are auto-detected as account-scope; the routing identifier is dropped. - A small hand-written list in `cmd/api/paths.go` carves out workspace-routed proxy APIs that happen to live under `/accounts/`, so they keep the identifier. - `--account` forces account-scope on a non-`/accounts/` path. - `--workspace-id <id>` overrides the identifier per call. Mutually exclusive with `--account`. - `?o=<id>` on the path (the SPOG URL convention used by the Databricks UI) is recognized as a per-call workspace override, so URLs pasted from the browser route correctly. - The CLI-only `workspace_id = none` sentinel is stripped before the routing decision so the literal "none" never goes on the wire. Routing logic lives in pure functions (`hasAccountSegment`, `extractOrgIDFromQuery`, `resolveOrgID`, `normalizeWorkspaceID`, `isWorkspaceProxyPath`) that take primitives. The cobra `RunE` is a thin adapter that resolves config and calls them. ## Test plan - [x] `go test ./cmd/api` covers the helpers with table-driven cases: deny-list hits and misses, query/fragment edge cases, mutual-exclusion errors, sentinel stripping, `?o=` extraction. - [x] `go test ./acceptance -run TestAccept/cmd/api` exercises seven variants end to end against terraform and direct engines: workspace path, account path, deny-listed proxy under `/accounts/`, `--account`, `--workspace-id`, `?o=` query, `workspace_id = none`. Each test asserts header presence/absence explicitly via `print_requests.py | contains.py`. - [x] `make checks`

## Why Pasting a SPOG URL from the Databricks UI (e.g. `https://acme.azuredatabricks.net/?o=12345`) into `DATABRICKS_HOST` drops the workspace identifier before any API call goes out. The SDK strips path and query from `Host` in `fixHostIfNeeded` without promoting `?o=` to `WorkspaceID`, so the request goes to the SPOG hostname without an `X-Databricks-Org-Id` header. The server can't route it and answers with the login HTML page, which surfaces as: ``` $ DATABRICKS_HOST=https://acme.azuredatabricks.net/?o=12345 databricks bundle validate Error: received HTML response instead of JSON ``` The bundle YAML `workspace.host` field is already normalized via `NormalizeHostURL` in `bundle/config/workspace.go`, and `databricks api` handles `?o=` per call (#5137). The env-var path was the remaining gap. ## Changes Before: `DATABRICKS_HOST=https://acme.databricks.net/?o=12345` reached the SDK with the query intact, the SDK dropped it, and `WorkspaceID` stayed empty. Now: `auth.NormalizeDatabricksHostEnv` runs once at the top of `root.Execute`, before the SDK reads anything. It uses the existing `ExtractHostQueryParams` helper to promote `?o=` / `?workspace_id=` to `DATABRICKS_WORKSPACE_ID` and `?a=` / `?account_id=` to `DATABRICKS_ACCOUNT_ID` (only when those env vars are unset), then rewrites `DATABRICKS_HOST` without the query string. A follow-up PR will push the same normalization into the SDK's `fixHostIfNeeded` so Python/Java/JS SDK users and any direct Go-SDK callers get the same fix without going through the CLI. ## Test plan - [x] `go test ./libs/auth/` covers the new helper with table-driven cases: `?o=` promotion, `?a=` + `?o=` together, existing `DATABRICKS_WORKSPACE_ID` is preserved, hosts without query are untouched, non-numeric `?o=` is dropped, unset `DATABRICKS_HOST` is a no-op. - [x] `go test ./cmd/root/` passes. - [x] `./task checks`. - [x] Manual end-to-end repro with a local SPOG-shaped test server: before the fix the SDK sent no `X-Databricks-Org-Id` header and got HTML back; after the fix the header is `258628866953061` and `bundle validate` proceeds to the workspace API.

## Why Pasting a SPOG URL from the Databricks UI (e.g. `https://acme.azuredatabricks.net/?o=12345`) into `DATABRICKS_HOST` drops the workspace identifier before any API call goes out. The SDK strips path and query from `Host` in `fixHostIfNeeded` without promoting `?o=` to `WorkspaceID`, so the request goes to the SPOG hostname without an `X-Databricks-Org-Id` header. The server can't route it and answers with the login HTML page, which surfaces as: ``` $ DATABRICKS_HOST=https://acme.azuredatabricks.net/?o=12345 databricks bundle validate Error: received HTML response instead of JSON ``` The bundle YAML `workspace.host` field is already normalized via `NormalizeHostURL` in `bundle/config/workspace.go`, and `databricks api` handles `?o=` per call (databricks#5137). The env-var path was the remaining gap. ## Changes Before: `DATABRICKS_HOST=https://acme.databricks.net/?o=12345` reached the SDK with the query intact, the SDK dropped it, and `WorkspaceID` stayed empty. Now: `auth.NormalizeDatabricksHostEnv` runs once at the top of `root.Execute`, before the SDK reads anything. It uses the existing `ExtractHostQueryParams` helper to promote `?o=` / `?workspace_id=` to `DATABRICKS_WORKSPACE_ID` and `?a=` / `?account_id=` to `DATABRICKS_ACCOUNT_ID` (only when those env vars are unset), then rewrites `DATABRICKS_HOST` without the query string. A follow-up PR will push the same normalization into the SDK's `fixHostIfNeeded` so Python/Java/JS SDK users and any direct Go-SDK callers get the same fix without going through the CLI. ## Test plan - [x] `go test ./libs/auth/` covers the new helper with table-driven cases: `?o=` promotion, `?a=` + `?o=` together, existing `DATABRICKS_WORKSPACE_ID` is preserved, hosts without query are untouched, non-numeric `?o=` is dropped, unset `DATABRICKS_HOST` is a no-op. - [x] `go test ./cmd/root/` passes. - [x] `./task checks`. - [x] Manual end-to-end repro with a local SPOG-shaped test server: before the fix the SDK sent no `X-Databricks-Org-Id` header and got HTML back; after the fix the header is `258628866953061` and `bundle validate` proceeds to the workspace API.

simonfaltum temporarily deployed to test-trigger-is April 30, 2026 09:12 — with GitHub Actions Inactive

simonfaltum added 2 commits April 30, 2026 11:28

simonfaltum force-pushed the simonfaltum/cli-api-spog-routing branch from 6db6f9e to a775d99 Compare April 30, 2026 09:38

simonfaltum temporarily deployed to test-trigger-is April 30, 2026 09:38 — with GitHub Actions Inactive

simonfaltum changed the title ~~api: make databricks api host-agnostic~~ api: generate workspace-proxy deny-list from SDK source Apr 30, 2026

simonfaltum mentioned this pull request Apr 30, 2026

api: route per-call against unified hosts #5140

Closed

4 tasks

simonfaltum added 3 commits April 30, 2026 12:15

api: replace generated proxy paths with manual list

28fbf39

Keep the unified-host routing support small for the initial PR by hand-maintaining the current workspace-proxy exceptions instead of parsing generated SDK source.

api: route account detection through proxy helper

4769fa7

Keep the manual workspace-proxy list behind one helper so tests exercise the same path used by runtime account detection.

simonfaltum temporarily deployed to test-trigger-is April 30, 2026 10:18 — with GitHub Actions Inactive

simonfaltum changed the title ~~api: generate workspace-proxy deny-list from SDK source~~ api: route per-call against unified hosts Apr 30, 2026

simonfaltum marked this pull request as ready for review April 30, 2026 10:32

simonfaltum temporarily deployed to test-trigger-is April 30, 2026 10:39 — with GitHub Actions Inactive

simonfaltum requested a review from pietern April 30, 2026 14:55

Merge branch 'main' into simonfaltum/cli-api-spog-routing

f548898

simonfaltum temporarily deployed to test-trigger-is May 1, 2026 06:59 — with GitHub Actions Inactive

Merge branch 'main' into simonfaltum/cli-api-spog-routing

9526263

simonfaltum temporarily deployed to test-trigger-is May 4, 2026 08:34 — with GitHub Actions Inactive

simonfaltum added 2 commits May 4, 2026 11:22

Merge remote-tracking branch 'origin/simonfaltum/cli-api-spog-routing…

b60ab18

…' into simonfaltum/cli-api-spog-routing

simonfaltum temporarily deployed to test-trigger-is May 4, 2026 09:25 — with GitHub Actions Inactive

simonfaltum added 3 commits May 4, 2026 15:31

Merge remote-tracking branch 'origin/main' into simonfaltum/cli-api-s…

55a83d5

…pog-routing

simonfaltum temporarily deployed to test-trigger-is May 4, 2026 13:39 — with GitHub Actions Inactive

pietern approved these changes May 4, 2026

View reviewed changes

Merge branch 'main' into simonfaltum/cli-api-spog-routing

f7793ce

simonfaltum temporarily deployed to test-trigger-is May 4, 2026 14:38 — with GitHub Actions Inactive

simonfaltum temporarily deployed to test-trigger-is May 4, 2026 14:41 — with GitHub Actions Inactive

Merge remote-tracking branch 'origin/main' into simonfaltum/cli-api-s…

1de3f95

…pog-routing

simonfaltum temporarily deployed to test-trigger-is May 5, 2026 09:01 — with GitHub Actions Inactive

simonfaltum temporarily deployed to test-trigger-is May 5, 2026 09:08 — with GitHub Actions Inactive

Merge branch 'main' into simonfaltum/cli-api-spog-routing

5f4e552

simonfaltum temporarily deployed to test-trigger-is May 5, 2026 11:11 — with GitHub Actions Inactive

simonfaltum merged commit 3c867c9 into main May 5, 2026
52 of 57 checks passed

simonfaltum deleted the simonfaltum/cli-api-spog-routing branch May 5, 2026 14:34

simonfaltum mentioned this pull request May 27, 2026

auth: extract ?o=/?a= from DATABRICKS_HOST env var #5337

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

api: route per-call against unified hosts#5137

api: route per-call against unified hosts#5137
simonfaltum merged 20 commits into
mainfrom
simonfaltum/cli-api-spog-routing

simonfaltum commented Apr 30, 2026 •

edited

Loading

Uh oh!

pietern May 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

simonfaltum commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

Changes

Test plan

Uh oh!

pietern May 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

simonfaltum commented Apr 30, 2026 •

edited

Loading