diff --git a/docs/platform/enterprise-platform/configure-connector-gateway.mdx b/docs/platform/enterprise-platform/configure-connector-gateway.mdx index 381a0491..18172b63 100644 --- a/docs/platform/enterprise-platform/configure-connector-gateway.mdx +++ b/docs/platform/enterprise-platform/configure-connector-gateway.mdx @@ -2,43 +2,162 @@ title: Configure the Connector Gateway sidebar_label: Configure the Connector Gateway description: - Enable the per-user Connector Gateway in the platform chart, choose its - install identity, and understand how it scopes connector access per user. + Enable the Connector Gateway so each user gets one MCP endpoint that serves + only the connectors they're granted, signed in with their own identity. --- The Connector Gateway provides an identity-aware MCP endpoint and brokers upstream OAuth for connectors. Enable it through the Stacklok Enterprise -platform chart. +platform chart. Enabling it takes more than a single flag, because the gateway +signs users in and stores their connector credentials itself. + +This page uses the `stacklok-enterprise` release name and the `stacklok-system` +namespace from [Deploy the platform](./deployment.mdx). Several values below +embed those names, so adjust them if yours differ. ## Prerequisites - **A corporate identity provider** at `global.stacklok.primaryIdp`. The gateway resolves every caller to a directory user before it decides what they can see. See [Configure identity](./configure-identity.mdx). -- **The directory service**, which the gateway reaches over gRPC for identity, - connector configuration, and access policy. It is part of the platform chart. +- **The Enterprise Manager, with `enterprise-manager.directory.enabled: true`**. + The gateway gets each caller's identity, connector grants, and connector + configuration from it. It needs its PostgreSQL database, set up as described + in + [the Enterprise Manager database](./deployment.mdx#enterprise-manager-database). + To administer connectors across users in the console, also create the + `BYPASSRLS` admin database role described there. +- **The Enterprise Manager's key-encryption key, gRPC TLS, and NetworkPolicy**, + configured as described in + [Encrypting stored credentials](./deployment.mdx#encrypting-stored-credentials). + The Enterprise Manager encrypts connector credentials and the identity + provider's client secret with this key, and returns them to the gateway only + over TLS. That section's NetworkPolicy example already admits pods labeled + `app.kubernetes.io/name: connector-gateway`. +- **Redis or Valkey**, reachable from the cluster. The embedded authorization + server stores its sessions only in Redis, at every replica count, and the + chart doesn't deploy an instance. Configure it once under `global.redis`, as + described in + [Global Redis/Valkey defaults](./deployment.mdx#global-redisvalkey-defaults), + and the gateway inherits the host, port, and password Secret. + +## Prepare the gateway's secrets + +The gateway needs three Secrets in the release namespace. Create them before you +install or upgrade. + +**Key-encryption key (KEK).** The gateway seals the upstream OAuth tokens and +client registrations it stores with this key. It is separate from the Enterprise +Manager's KEK. The Secret holds 32 random bytes under the key `kek`: + +```bash +openssl rand 32 > kek +kubectl create secret generic connector-gateway-kek \ + --namespace stacklok-system \ + --from-file=kek=./kek +``` + +Keep a backup of the key. Losing or replacing it makes every credential the +gateway has stored unreadable. + +**Authorization server keys.** The signing key keeps issued tokens valid across +restarts and replicas, and the HMAC secret keeps authorization codes and refresh +tokens valid. The signing key must be a PEM-encoded ECDSA private key: + +```bash +openssl ecparam -name prime256v1 -genkey -noout -out signing-key.pem +openssl rand 32 > hmac-secret +kubectl create secret generic connector-gateway-auth-server-keys \ + --namespace stacklok-system \ + --from-file=signing-key.pem=./signing-key.pem \ + --from-file=hmac-secret=./hmac-secret +``` + +:::warning[Use existing Secrets with GitOps] + +`kek.generate: true` and `authServerKeys.generate: true` have the chart create +and preserve the key material itself, but only on a live `helm install` or +`helm upgrade`. `helm template`, Argo CD, and other render-and-apply workflows +can't look up the existing Secret, so they would mint new keys on every sync and +orphan everything sealed or signed with the old ones. Use `existingSecret` for +those workflows. + +::: + +**Enterprise Manager CA bundle.** The gateway verifies the Enterprise Manager's +gRPC serving certificate against a CA bundle. Put only the CA certificate in +this Secret. If cert-manager issues the Enterprise Manager's certificate, copy +`ca.crt` out of its Secret rather than referencing that Secret, which also holds +the private key: + +```bash +kubectl create secret generic enterprise-manager-grpc-ca \ + --namespace stacklok-system \ + --from-file=ca.crt=./ca.crt +``` ## Enable the Connector Gateway -Set the enable flag, gateway identifier, and public issuer URL: +Add the following to the values file you use for the platform. Merge the +`global` keys into your existing `global` block rather than adding a second one. ```yaml title="values.yaml" global: stacklok: + # primaryIdp from "Configure identity" stays here unchanged. connectorGateway: enabled: true - connectorGatewayId: - authServerIssuer: https:// + connectorGatewayId: '' + authServerIssuer: 'https://' + # Redis or Valkey from "Global Redis/Valkey defaults" + redis: + host: 'redis.example.com' + port: 6379 + existingSecret: 'redis-auth' + +connector-gateway: + vmcpConfig: + incomingAuth: + type: oidc + oidc: + issuer: 'https://' + audience: 'https:///gw/mcp' + enterpriseConfig: + authServer: + schema_version: 'v0.1.0' + allowed_audiences: + - 'https:///gw/mcp' + storage: + type: redis + directory: + addr: 'stacklok-enterprise-enterprise-manager.stacklok-system.svc.cluster.local:9443' + directoryTLS: + enabled: true + caBundle: + secretName: 'enterprise-manager-grpc-ca' + kek: + existingSecret: 'connector-gateway-kek' + authServerKeys: + existingSecret: 'connector-gateway-auth-server-keys' ``` -| Value | Purpose | -| -------------------------- | -------------------------------------------------- | -| `connectorGateway.enabled` | Deploy the Connector Gateway | -| `connectorGatewayId` | Identify this gateway to the directory and console | -| `authServerIssuer` | Set the gateway's public authorization server URL | +| Value | Purpose | +| ------------------------------------------ | ----------------------------------------------------------------------------------------- | +| `global.stacklok.connectorGateway.enabled` | Deploy the Connector Gateway | +| `global.stacklok.connectorGatewayId` | Identify this gateway to the Enterprise Manager and console | +| `global.stacklok.authServerIssuer` | Set the gateway's public authorization server URL | +| `vmcpConfig.incomingAuth` | Require tokens from the gateway's own authorization server on the MCP endpoint | +| `enterpriseConfig.authServer` | Configure the embedded authorization server and its Redis storage | +| `enterpriseConfig.directory.addr` | Point the gateway at the Enterprise Manager's TLS gRPC port | +| `directoryTLS` | Mount the Enterprise Manager CA bundle and the gateway's ServiceAccount token | +| `kek`, `authServerKeys` | Reference the Secrets from [Prepare the gateway's secrets](#prepare-the-gateways-secrets) | + +### Choose the gateway ID and issuer -Choose a stable `connectorGatewayId` such as `prod-eu` or `platform-staging`. -The value cannot be empty or contain a colon. Changing it creates a new gateway +Choose a stable `connectorGatewayId` such as `prod-eu` or `platform-staging`, +and give every install its own value. The gateway registers this ID with the +Enterprise Manager, so two installs sharing one ID act as a single gateway. The +value cannot be empty or contain a colon. Changing it creates a new gateway identity and changes the prefix used for stored per-user tokens. Set `authServerIssuer` to the HTTPS URL that clients use to reach this @@ -46,6 +165,12 @@ deployment. The gateway derives connector OAuth callbacks as `{issuer}/oauth/callback`. Configure the value under `global.stacklok`; the chart rejects equivalent settings under the component's internal configuration. +The MCP endpoint is `{issuer}/gw/mcp`, and that URL is the token audience. Set +`vmcpConfig.incomingAuth.oidc.issuer` to the same value as `authServerIssuer`, +and list the audience in `allowed_audiences` exactly as it appears in +`incomingAuth.oidc.audience`. The gateway compares these strings exactly and +refuses to start when they disagree. + :::note[Two different issuers] `global.stacklok.primaryIdp.issuer` identifies the corporate identity provider. @@ -55,13 +180,120 @@ Configure both values. See ::: +Omit `upstreams` from `enterpriseConfig.authServer`. The authorization server +loads its identity providers from the Enterprise Manager: your corporate +identity provider from `global.stacklok.primaryIdp`, plus the connector identity +providers that admins register in the Enterprise Manager. + +By default, the Enterprise Manager identifies users by the `sub` claim in your +corporate identity provider's tokens. If your provider identifies users with a +different claim, such as `uid` in Okta, add an +`enterprise-manager.directory.issuers` entry whose `issuer_url` matches +`global.stacklok.primaryIdp.issuer` exactly, and set `binding_claim`: + +```yaml title="values.yaml" +enterprise-manager: + directory: + issuers: + - id: 'corporate-idp' + issuer_url: 'https://idp.example.com' + audience: 'enterprise-manager' + binding_claim: 'uid' +``` + +The same entry configures SCIM provisioning for that provider. See +[SCIM identity providers](./deployment.mdx#scim-identity-providers). + +## Connect the gateway to the Enterprise Manager + +`enterpriseConfig.directory.addr` requires exactly one transport. With +`directoryTLS.enabled: true`, the gateway dials the Enterprise Manager's TLS +gRPC port (`9443`) and authenticates with a projected ServiceAccount token. The +Enterprise Manager side of that connection needs a TLS listener, caller +authentication, and the gateway's ServiceAccount on three allowlists: + +```yaml title="values.yaml" +enterprise-manager: + grpc: + tls: + enabled: true + existingSecret: 'enterprise-manager-grpc-tls' + callerAuth: + clusterOidcIssuer: '' + saSubjectAllowlist: + - 'system:serviceaccount:stacklok-system:stacklok-enterprise-connector-gateway' + gatewaySaSubjectAllowlist: + - 'system:serviceaccount:stacklok-system:stacklok-enterprise-connector-gateway' + credentialSaSubjectAllowlist: + - 'system:serviceaccount:stacklok-system:stacklok-enterprise-connector-gateway' +``` + +`grpc.tls` is the same setting you configured in +[Encrypting stored credentials](./deployment.mdx#encrypting-stored-credentials). +The chart refuses to render a TLS gateway transport while +`enterprise-manager.grpc.tls.enabled` is `false`. + +Set `clusterOidcIssuer` to your cluster's ServiceAccount token issuer, which +this command prints: + +```bash +kubectl get --raw /.well-known/openid-configuration | jq -r .issuer +``` + +Each allowlist covers different calls from the gateway. `saSubjectAllowlist` +covers identity resolution and per-user connector configuration. +`gatewaySaSubjectAllowlist` covers the gateway's startup registration and its +catalog reads. `credentialSaSubjectAllowlist` covers credential resolution, +which the gateway needs for the identity provider's client secret. The subject +is `system:serviceaccount::-connector-gateway`. If the +AI Gateway also calls the Enterprise Manager, keep its entries on +`saSubjectAllowlist` alongside this one. + +For a development cluster without TLS, set +`connector-gateway.enterpriseConfig.directory.insecure: true` in place of +`directoryTLS`, point `addr` at the cleartext port (`9091`), and set +`enterprise-manager.grpc.insecure: true`. The chart requires both settings +together. This transport carries credentials in cleartext with no caller +authentication, so reserve it for local testing. + +## Corporate identity provider settings + +The gateway reads these optional keys from `global.stacklok.primaryIdp`, in +addition to the issuer and client settings in +[Configure platform identity](./configure-identity.mdx): + +| Key | Purpose | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `clientClaim` | Access-token claim that names the OAuth client. Empty reads `cid` or `client_id`. Allowed values are `cid`, `client_id`, `azp`, and `appid`; set `azp` for Dex, or `azp` or `appid` for Microsoft Entra ID | +| `clientClaimAllowUnsafe` | Boolean. Accepts a `clientClaim` outside the allowed values. A weaker claim can match tokens minted for a different client, and the gateway logs a warning at startup | +| `allowPrivateIps` | Boolean. Lets discovery and token exchange reach an identity provider on a private or link-local address. HTTPS is still required for non-localhost hosts | +| `caBundleRef` | ConfigMap or Secret holding the PEM CA bundle for an identity provider served by a private CA. Set exactly one of `configMapRef` or `secretRef`, each with a `name`, plus an optional `key` (default `ca.crt`) | + +Set the boolean keys as unquoted `true` or `false`. The chart refuses a quoted +string, and `--set-string` produces one. + +:::warning[Include the full chain in `caBundleRef`] + +For sign-in, the gateway trusts `caBundleRef` in addition to the system roots. +Its control-plane token validator, which checks the corporate tokens the console +sends, trusts only this bundle. Include every intermediate and root certificate +in the chain that serves your identity provider's JWKS endpoint. A bundle that +omits a public intermediate or root makes every control-plane request fail with +`401`, even though sign-in works. + +::: + +The gateway reads the bundle once at startup. After you replace the contents of +the same ConfigMap or Secret, restart the gateway with +`kubectl rollout restart deployment/stacklok-enterprise-connector-gateway -n stacklok-system`. + ## How connector access works ```mermaid flowchart LR Client["MCP client
(editor, agent, CLI)"] MCPGW["Connector Gateway"] - Dir["Directory service"] + Dir["Enterprise Manager"] Backends["MCP servers
(ToolHive workloads, remote servers)"] Client -->|"one endpoint, user's own identity"| MCPGW @@ -69,26 +301,21 @@ flowchart LR MCPGW -->|"only the connectors they are entitled to"| Backends ``` -The gateway asks the directory to resolve each caller and return their connector -grants. Directory group membership determines the MCP servers exposed to the -client. - -:::note - -Cluster authorization policy uses OIDC claim groups. See +The gateway asks the Enterprise Manager to resolve each caller and return their +connector grants. Directory group membership determines the MCP servers exposed +to the client, and is tracked separately from the OIDC claim groups that cluster +authorization policy matches. See [Directory groups and OIDC claim groups](../concepts/two-group-models.mdx). -::: - ## After enabling -1. Confirm the gateway registered with the directory. Its pod becomes ready once - it has, and the admin **Connectors** view in the console appears for the id - you set. +1. Confirm the gateway pod is ready. It registers with the Enterprise Manager at + startup and becomes ready once its first connector catalog read succeeds. The + admin **Connectors** view in the console then appears for the ID you set. 2. Add connectors and grant them to groups, either in the console under - [Connectors](../../connector-gateway/connectors.mdx) or through the directory - API, which exposes the same operations under - `/v1/gateways/{gateway_id}/connectors`. + [Connectors](../../connector-gateway/connectors.mdx) or through the + [Enterprise Manager API](./api-reference.mdx#enterprise-manager), which + exposes the same operations under `/v1/gateways/{gateway_id}/connectors`. 3. Direct users to the deployment-specific instructions in the console. See [Roll out gateway clients](./roll-out-gateway-clients.mdx). @@ -98,3 +325,34 @@ Cluster authorization policy uses OIDC claim groups. See register connectors and grant access. - [Identity and directory](../enterprise-directory/index.mdx) for the users and groups that access is granted to. + +## Troubleshooting + +
+The gateway pod stays unready or restarts + +The readiness probe fails until the gateway's first connector catalog read from +the Enterprise Manager succeeds, so an unready pod means the gateway can't +complete its calls to the Enterprise Manager. Check the pod's logs: + +```bash +kubectl logs deployment/stacklok-enterprise-connector-gateway -n stacklok-system +``` + +- **`PermissionDenied` during registration, followed by a restart.** The + gateway's ServiceAccount subject is missing from + `enterprise-manager.grpc.callerAuth.gatewaySaSubjectAllowlist`. Add it as + shown in + [Connect the gateway to the Enterprise Manager](#connect-the-gateway-to-the-enterprise-manager). +- **`Unauthenticated` on every call to the Enterprise Manager.** The Enterprise + Manager rejected the gateway's ServiceAccount token. Check that + `clusterOidcIssuer` matches the issuer your cluster reports. +- **TLS handshake errors.** Check that `enterprise-manager-grpc-ca` holds the CA + that signed the Enterprise Manager's gRPC certificate, and that `addr` uses + port `9443` and a hostname on that certificate. After you rotate the CA, + restart the gateway; it reads the bundle only at startup. +- **Connection timeouts.** Check that the Enterprise Manager's gRPC + NetworkPolicy admits the gateway pods, and that the gateway can reach the + Redis or Valkey host. + +
diff --git a/docs/platform/enterprise-platform/deployment.mdx b/docs/platform/enterprise-platform/deployment.mdx index 338414da..cc01c683 100644 --- a/docs/platform/enterprise-platform/deployment.mdx +++ b/docs/platform/enterprise-platform/deployment.mdx @@ -57,8 +57,9 @@ identifiers retain their original names: the flag is `cloudUi` and its configuration key is `toolhive-cloud-ui`, so use those spellings in the values file. -The Connector Gateway requires its enable flag, gateway ID, and authorization -server issuer. See +Besides its enable flag, gateway ID, and authorization server issuer, the +Connector Gateway requires Redis or Valkey, key material, and a TLS connection +to the directory. See [Configure the Connector Gateway](./configure-connector-gateway.mdx) and [Configure the AI Gateway](./configure-ai-gateway.mdx). @@ -698,10 +699,11 @@ the components connect without credentials. The following components inherit `global.redis`: -| Enable flag | How it uses the global default | -| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `toolhiveOperator` | Default session storage for `MCPServer`, `MCPRemoteProxy`, and `VirtualMCPServer` (vMCP) workloads that have no explicit `spec.sessionStorage`. The per-resource setting always overrides. | -| `global.stacklok.aiGateway.enabled` | NER scan-result caching when an `AIGateway` enables the cache | +| Enable flag | How it uses the global default | +| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `toolhiveOperator` | Default session storage for `MCPServer`, `MCPRemoteProxy`, and `VirtualMCPServer` (vMCP) workloads that have no explicit `spec.sessionStorage`. The per-resource setting always overrides. | +| `global.stacklok.aiGateway.enabled` | NER scan-result caching when an `AIGateway` enables the cache | +| `global.stacklok.connectorGateway.enabled` | Required session storage for the gateway's embedded authorization server. See [Configure the Connector Gateway](./configure-connector-gateway.mdx) | The embedded auth server's token storage is configured separately through `MCPExternalAuthConfig`. See diff --git a/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-dark-desktop-linux.json b/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-dark-desktop-linux.json index 92fad2c4..e4dd7e7c 100644 --- a/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-dark-desktop-linux.json +++ b/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-dark-desktop-linux.json @@ -1,5 +1,8 @@ { - "titlePath": ["pages.spec.ts", "nav page - AI Gateway"], + "titlePath": [ + "pages.spec.ts", + "nav page - AI Gateway" + ], "testFile": "tests/visual/pages.spec.ts", "url": { "origin": "http://localhost:3005", diff --git a/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-dark-desktop-linux.png b/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-dark-desktop-linux.png index 89300328..5cbc16cb 100644 Binary files a/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-dark-desktop-linux.png and b/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-dark-desktop-linux.png differ diff --git a/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-light-desktop-linux.json b/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-light-desktop-linux.json index 92fad2c4..e4dd7e7c 100644 --- a/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-light-desktop-linux.json +++ b/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-light-desktop-linux.json @@ -1,5 +1,8 @@ { - "titlePath": ["pages.spec.ts", "nav page - AI Gateway"], + "titlePath": [ + "pages.spec.ts", + "nav page - AI Gateway" + ], "testFile": "tests/visual/pages.spec.ts", "url": { "origin": "http://localhost:3005", diff --git a/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-light-desktop-linux.png b/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-light-desktop-linux.png index be7a0630..f597eb29 100644 Binary files a/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-light-desktop-linux.png and b/tests/visual/pages.spec.ts-snapshots/nav-page-ai-gateway-light-desktop-linux.png differ