fix(kernel): Azure OAuth U2M — forward the in-house app/scopes uniformly on the kernel path

CHANGELOG.md

@@ -3,6 +3,7 @@
 ## Unreleased
 - Support `WithFederatedTokenProvider*` on the kernel backend by resolving its token once for kernel-side federation and forwarding the optional SP-wide client ID
 - Improve telemetry error reporting: driver failures are now categorized by cause instead of reported as a generic error (databricks/databricks-sql-go#414, #415, #417, #419, #424)
+- Fix OAuth U2M on the kernel backend (`WithUseKernel(true)`) against **Azure** workspaces: `resolveKernelAuth` now forwards the in-house OAuth app (`databricks-sql-connector`) + `sql offline_access` scopes **uniformly across all clouds**, instead of the Azure Entra-direct app id + `user_impersonation` scope that the Thrift path infers from the host. The kernel runs a single cloud-blind in-house workspace-federated U2M flow (it uses the workspace's OIDC-discovered authorize endpoint verbatim), so feeding it the Entra-direct app made the workspace federation route the browser to a broken AAD authorize URL. AWS/GCP behavior is unchanged (those already resolved to the in-house app + scopes). The Thrift path is unaffected.
 
 ## v1.14.0 (2026-07-13)
 - **Minimum Go version is now 1.25.0** (previously 1.20): the `go` directive was raised to 1.25.0 while clearing OSV-Scanner findings and updating dependencies. Consumers building with an older toolchain will need to upgrade Go (databricks/databricks-sql-go#368)

auth/oauth/u2m/authenticator.go

@@ -78,15 +78,17 @@ type u2mAuthenticator struct {
 	mx          sync.Mutex
 }
 
-// U2MClientID exposes the cloud-inferred OAuth client id so the SEA-via-kernel
-// backend uses the same client id for the kernel's browser/PKCE flow that the
-// Thrift path would, keeping cfg.Authenticator the single source of truth for auth
-// mode. It structurally satisfies the U2MCredentialsProvider interface the kernel
-// backend asserts (defined in internal/backend/kernel, off the public API).
+// U2MClientID exposes the cloud-inferred OAuth client id, keeping
+// cfg.Authenticator the single source of truth for auth mode. It structurally
+// satisfies the U2MCredentialsProvider interface the kernel backend asserts
+// (defined in internal/backend/kernel, off the public API).
 //
-// Only the client id is read here; resolveKernelAuth pairs it with the same
-// oauth.GetScopes set the Thrift path requests, so both backends authorize against
-// the built-in databricks-sql-connector client identically (not the kernel default).
+// NOTE: the kernel path asserts this interface only to DETECT that the U2M flow is
+// selected; it does NOT forward this cloud-inferred client id. The kernel runs one
+// in-house workspace-federated flow on every cloud, so resolveKernelAuth uses the
+// fixed built-in databricks-sql-connector client and offline_access + sql scopes
+// instead (see resolveKernelAuth's U2M case). On AWS/GCP that equals this value;
+// on Azure it deliberately differs.
 func (c *u2mAuthenticator) U2MClientID() string { return c.clientID }
 
 // Auth will start the OAuth Authorization Flow to authenticate the cli client

doc.go

@@ -228,11 +228,14 @@ OAuth U2M is interactive: on a cache miss, connecting launches the system browse
 blocks until login completes or the kernel's ~120s callback timeout expires. Because
 the C ABI can't interrupt session open mid-call, a connection-context deadline is not
 honored during that window. Use PAT or OAuth M2M for headless / deadline-bound
-connects. The kernel and Thrift backends use the same (cloud-inferred) U2M client id
-and request the same default scopes (offline_access + sql, or, on Azure,
-offline_access + <tenant>/user_impersonation): the kernel path forwards exactly the
-scopes the Thrift path computes, so both authorize identically against the built-in
-public client. Neither backend exposes a U2M-scopes option.
+connects. The kernel runs a single, cloud-agnostic in-house U2M flow (OIDC discovery
+against {host}/oidc, no Azure branching), so on every cloud the kernel path uses the
+built-in databricks-sql-connector client and requests offline_access + sql. It does
+NOT forward the cloud-inferred client id / scopes the Thrift path computes: on
+AWS/GCP those already match, but on Azure the Thrift path uses the Entra-direct app
+id and <tenant>/user_impersonation scope, which would break the kernel's
+workspace-federated flow. So the two backends authorize identically on AWS/GCP and
+deliberately diverge on Azure. Neither backend exposes a U2M-scopes option.
 
 Experimental kernel-only options (rejected by the default backend; the
 WithKernel* prefix marks them experimental):

internal/backend/kernel/auth.go

@@ -20,18 +20,20 @@ const (
 // validateKernelConfig); OpenSession maps it to exactly one
 // kernel_session_config_set_auth_* call.
 // Scopes and RedirectPort map to the optional args of set_auth_u2m and are wired
-// through to it by setAuth. resolveKernelAuth populates Scopes with the same
-// cloud-specific set the Thrift path requests (via oauth.GetScopes) so both
-// backends authorize identically; RedirectPort stays zero (no user option, kernel
-// default 8030) but is kept so kernel.Auth models the full set_auth_u2m surface —
-// a future WithOAuthRedirectPort becomes populating it, not re-plumbing the setter.
-// TestSetAuthByMode's "U2M full" case pins the marshalling of both.
+// through to it by setAuth. For U2M, resolveKernelAuth populates ClientID/Scopes
+// with the fixed in-house databricks-sql-connector client and offline_access + sql
+// on every cloud (NOT the cloud-inferred Thrift values), because the kernel runs one
+// in-house workspace-federated flow with no Azure branching. RedirectPort stays zero
+// (no user option, kernel default 8030) but is kept so kernel.Auth models the full
+// set_auth_u2m surface — a future WithOAuthRedirectPort becomes populating it, not
+// re-plumbing the setter. TestSetAuthByMode's "U2M full" case pins the marshalling
+// of both.
 type Auth struct {
 	Mode         AuthMode
 	Token        string   // PAT
-	ClientID     string   // M2M + U2M; federated PAT uses the optional SP-wide client id
+	ClientID     string   // M2M + U2M (U2M: fixed in-house client, cloud-agnostic); federated PAT uses the optional SP-wide client id
 	ClientSecret string   // M2M
-	Scopes       []string // U2M — Thrift-parity scopes from oauth.GetScopes; nil → kernel default
+	Scopes       []string // U2M — fixed offline_access + sql (cloud-agnostic); nil → kernel default
 	RedirectPort uint16   // U2M — no user option today; 0 → kernel default port (8030)
 }
 
@@ -69,12 +71,15 @@ func M2MScopesSupported(scopes []string) bool {
 	}
 }
 
-// U2MCredentialsProvider is implemented by the OAuth U2M authenticator to expose
-// the cloud-inferred client id the kernel should use for its browser/PKCE flow (so
-// the kernel path uses the same client id the Thrift path would). Internal for the
-// same reason as M2MCredentialsProvider; satisfied structurally by the unexported
-// u2m authenticator.
+// U2MCredentialsProvider is implemented by the OAuth U2M authenticator. The kernel
+// backend asserts this interface only to DETECT that the interactive U2M flow is
+// selected — it does NOT forward the returned (cloud-inferred) client id. The kernel
+// runs a single in-house workspace-federated flow on every cloud, so resolveKernelAuth
+// uses the fixed built-in databricks-sql-connector client and offline_access + sql
+// scopes instead. Internal for the same reason as M2MCredentialsProvider; satisfied
+// structurally by the unexported u2m authenticator.
 type U2MCredentialsProvider interface {
-	// U2MClientID returns the OAuth client id for the U2M browser flow.
+	// U2MClientID returns the OAuth client id for the U2M browser flow. Used by the
+	// kernel path only as a presence marker (see the interface doc above).
 	U2MClientID() string
 }

internal/backend/kernel/backend.go

@@ -400,8 +400,10 @@ func (k *KernelBackend) setAuth(cfg *C.KernelSessionConfig) error {
 		}
 	case AuthU2M:
 		// client id / scopes are optional: NULL when empty lets the kernel use its
-		// public client / default scopes. We pass Go's cloud-inferred client id when
-		// set, so the kernel uses the same client id the Thrift path would.
+		// public client / default scopes. resolveKernelAuth fills these with the
+		// fixed in-house databricks-sql-connector client and offline_access + sql
+		// scopes on every cloud (NOT the cloud-inferred Thrift values), so the
+		// kernel's single in-house workspace-federated flow works uniformly.
 		clientID := newCStrOrNull(k.cfg.Auth.ClientID)
 		defer clientID.free()
 		scopes := newCStrOrNull(joinScopes(k.cfg.Auth.Scopes))

kernel_auth_real_test.go

@@ -4,7 +4,6 @@ import (
 	"reflect"
 	"testing"
 
-	"github.com/databricks/databricks-sql-go/auth/oauth"
 	"github.com/databricks/databricks-sql-go/auth/oauth/m2m"
 	"github.com/databricks/databricks-sql-go/auth/oauth/u2m"
 	"github.com/databricks/databricks-sql-go/internal/backend/kernel"
@@ -63,12 +62,20 @@ func TestResolveKernelAuthRealAuthenticators(t *testing.T) {
 				"error here means *u2mAuthenticator no longer satisfies kernel.U2MCredentialsProvider "+
 				"(U2MClientID renamed?): %v", err)
 		}
-		if got.Mode != kernel.AuthU2M || got.ClientID == "" {
-			t.Errorf("auth = %+v, want mode=U2M with a non-empty (cloud-inferred) clientID", got)
+		if got.Mode != kernel.AuthU2M {
+			t.Errorf("auth = %+v, want mode=U2M", got)
 		}
-		// Scopes are forwarded from oauth.GetScopes(cfg.Host) for Thrift parity.
-		if want := oauth.GetScopes(c.Host, nil); !reflect.DeepEqual(got.Scopes, want) {
-			t.Errorf("U2M scopes = %v, want %v (Thrift parity)", got.Scopes, want)
+		// The kernel path forwards the in-house bundle UNIFORMLY across clouds —
+		// databricks-sql-connector + sql/offline_access — NOT the Azure
+		// Entra-direct app id / user_impersonation scope, even though this is an
+		// Azure host. The kernel runs one cloud-blind in-house workspace-federated
+		// U2M flow; feeding it the Entra-direct app breaks it. See
+		// resolveKernelAuth's U2M case.
+		if got.ClientID != u2mKernelClientID {
+			t.Errorf("U2M clientID = %q, want %s (in-house app, cloud-agnostic on the kernel path)", got.ClientID, u2mKernelClientID)
+		}
+		if want := []string{"offline_access", "sql"}; !reflect.DeepEqual(got.Scopes, want) {
+			t.Errorf("U2M scopes = %v, want %v (in-house, cloud-agnostic)", got.Scopes, want)
 		}
 	})
 }

kernel_config.go

@@ -9,7 +9,6 @@ import (
 	"time"
 
 	"github.com/databricks/databricks-sql-go/auth/noop"
-	"github.com/databricks/databricks-sql-go/auth/oauth"
 	"github.com/databricks/databricks-sql-go/auth/pat"
 	dbsqlerr "github.com/databricks/databricks-sql-go/errors"
 	"github.com/databricks/databricks-sql-go/internal/backend/kernel"
@@ -24,6 +23,14 @@ import (
 // Config field being silently dropped — run under CGO_ENABLED=0. The tagged
 // newKernelBackend calls validateKernelConfig, then assembles the cgo kernel.Config.
 
+// u2mKernelClientID is the in-house Databricks OAuth app the kernel path uses for
+// U2M on EVERY cloud. It is the built-in databricks-sql-connector client — the
+// AWS/GCP default and the kernel's own U2M default. Deliberately cloud-agnostic:
+// the kernel runs one in-house workspace-federated flow (no Azure branching), so
+// the kernel path must not send the Azure Entra-direct app id. See
+// resolveKernelAuth's U2M case.
+const u2mKernelClientID = "databricks-sql-connector"
+
 // validateKernelConfig enforces the kernel backend's "nothing silently ignored"
 // contract: it rejects every option the kernel path can't yet honor with a clear
 // error (rather than dropping it, which would behave differently than Thrift) and
@@ -329,15 +336,28 @@ func resolveKernelAuthContext(ctx context.Context, cfg *config.Config) (kernel.A
 		clientID, clientSecret := a.M2MCredentials()
 		return kernel.Auth{Mode: kernel.AuthM2M, ClientID: clientID, ClientSecret: clientSecret}, nil
 	case kernel.U2MCredentialsProvider:
-		// Forward the SAME cloud-specific scopes the Thrift path requests via
-		// oauth.GetScopes (offline_access + sql on AWS/GCP, offline_access +
-		// <tenant>/user_impersonation on Azure), so both backends authorize against
-		// the built-in databricks-sql-connector client identically. Without this the
-		// kernel applied its own default set (all-apis + offline_access), which a
-		// workspace whose public client isn't granted all-apis rejects with
-		// access_denied. RedirectPort is still left zero (no user option; kernel
-		// default 8030). Passing nil to GetScopes yields the pure cloud-default set.
-		return kernel.Auth{Mode: kernel.AuthU2M, ClientID: a.U2MClientID(), Scopes: oauth.GetScopes(cfg.Host, nil)}, nil
+		// The kernel runs a single, cloud-agnostic in-house U2M flow: it does OIDC
+		// discovery against {host}/oidc and uses that authorize endpoint verbatim —
+		// it has NO Azure/cloud branching. That in-house workspace-federated flow
+		// works on every cloud including Azure (the workspace federates the browser
+		// login to Entra server-side). So forward the in-house app +
+		// `sql offline_access` scopes UNIFORMLY across clouds; do NOT forward the
+		// Azure Entra-direct app id / `user_impersonation` scope that
+		// a.U2MClientID() / oauth.GetScopes pick for an Azure host. Handing those
+		// Entra-direct values to the kernel's in-house flow makes the workspace
+		// federation route the browser to AAD and yields a broken authorize URL
+		// (login.microsoftonline.com/{tenant}/v1/authorize). The Thrift path keeps
+		// the cloud-specific values (it needs them); only this kernel-path mapping
+		// is uniform. On AWS/GCP this is a no-op — a.U2MClientID() is already
+		// databricks-sql-connector and this scope slice is byte-for-byte the set
+		// oauth.GetScopes(host, nil) returns there (it appends offline_access,
+		// then sql), so the historical ordering is preserved too.
+		// RedirectPort stays zero (no user option; kernel default).
+		return kernel.Auth{
+			Mode:     kernel.AuthU2M,
+			ClientID: u2mKernelClientID,
+			Scopes:   []string{"offline_access", "sql"},
+		}, nil
 	case nil, *noop.NoopAuth, *pat.PATAuth:
 		// PAT (or no explicit authenticator). WithAccessToken sets both
 		// cfg.AccessToken and a *pat.PATAuth, but WithAuthenticator(&pat.PATAuth{...})

kernel_config_test.go

@@ -8,7 +8,6 @@ import (
 	"testing"
 	"time"
 
-	"github.com/databricks/databricks-sql-go/auth/oauth"
 	"github.com/databricks/databricks-sql-go/auth/pat"
 	"github.com/databricks/databricks-sql-go/auth/tokenprovider"
 	dbsqlerr "github.com/databricks/databricks-sql-go/errors"
@@ -19,7 +18,7 @@ import (
 
 // nonPATAuth stands in for any non-PAT, non-OAuth authenticator (token-provider /
 // external / static) — the kernel backend must reject it. It implements neither
-// auth.M2MCredentialsProvider nor auth.U2MCredentialsProvider.
+// kernel.M2MCredentialsProvider nor kernel.U2MCredentialsProvider.
 type nonPATAuth struct{}
 
 func (nonPATAuth) Authenticate(*http.Request) error { return nil }
@@ -174,21 +173,25 @@ func TestValidateKernelConfig(t *testing.T) {
 	t.Run("OAuth U2M resolves to a U2M descriptor", func(t *testing.T) {
 		c := baseKernelConfig()
 		c.AccessToken = ""
-		// A U2M authenticator is the single source of truth; resolveKernelAuth reads
-		// its (cloud-inferred) client id via the auth.U2MCredentialsProvider interface.
-		c.Authenticator = fakeU2MAuth{id: "databricks-sql-connector"}
+		// A U2M authenticator selects the U2M path via the kernel.U2MCredentialsProvider
+		// interface. On the kernel path the descriptor is CLOUD-AGNOSTIC: resolveKernelAuth
+		// forwards the in-house app + scopes uniformly (it deliberately does NOT read the
+		// authenticator's cloud-inferred client id), so an Azure host would resolve to the
+		// same values — see the U2M case in resolveKernelAuth. The fake's id here is
+		// intentionally NOT what the descriptor carries.
+		c.Authenticator = fakeU2MAuth{id: "ignored-cloud-inferred-id"}
 		a, err := validateKernelConfig(c)
 		if err != nil {
 			t.Fatalf("U2M should validate, got %v", err)
 		}
-		if a.Mode != kernel.AuthU2M || a.ClientID != "databricks-sql-connector" {
-			t.Errorf("auth = %+v, want mode=U2M clientID=databricks-sql-connector", a)
+		if a.Mode != kernel.AuthU2M || a.ClientID != u2mKernelClientID {
+			t.Errorf("auth = %+v, want mode=U2M clientID=%s (in-house, cloud-agnostic)", a, u2mKernelClientID)
 		}
-		// Scopes must match what the Thrift path requests for this host, so both
-		// backends authorize against the same client identically (not the kernel's
-		// all-apis default). baseKernelConfig's host is AWS → [offline_access, sql].
-		if want := oauth.GetScopes(c.Host, nil); !reflect.DeepEqual(a.Scopes, want) {
-			t.Errorf("U2M scopes = %v, want %v (Thrift parity)", a.Scopes, want)
+		// The kernel path forwards the fixed in-house scopes for every cloud, not the
+		// cloud-specific oauth.GetScopes set (which on Azure would add user_impersonation
+		// and derail the kernel's in-house flow to AAD).
+		if want := []string{"offline_access", "sql"}; !reflect.DeepEqual(a.Scopes, want) {
+			t.Errorf("U2M scopes = %v, want %v (in-house, cloud-agnostic)", a.Scopes, want)
 		}
 	})