Epic: MCTL Unified Identity on ZITADEL and Agent Identity

Goal

Finish MCTL's canonical identity and authorization model on top of the already deployed ZITADEL human IdP, so users, agents and services are identified by stable MCTL principals rather than GitHub usernames, while MCP client identity and delegated agent identity remain explicit and auditable.

Architecture decision — current state

ZITADEL is already the platform human identity provider at auth.mctl.ai. This epic is no longer about choosing or deploying an IdP.

The intended boundary is:

ZITADEL authenticates humans and issues OIDC identities;
mctl-api canonical principals (prn_…) are the stable identity used by MCTL authorization, tenant membership, audit and policy;
external identities are linked by (provider, issuer, subject) to a canonical principal;
GitHub remains a linked identity and capability/integration provider, not the canonical authorization key;
MCP client identity (pre-registered / CIMD / DCR) is distinct from the authenticated human or service principal;
an agent action records both the agent actor and the server-derived on-behalf-of subject.

ZITADEL is not a broker in front of every credential type. Static service tokens, MCTL-issued MCP OAuth tokens and supported legacy GitHub credentials continue to terminate in mctl-api through the provider registry where appropriate.

Completed foundation

Required foundation already delivered:

mctl-api#373 — canonical human/agent/service principals and linked external identities;
mctl-gitops#1499 — self-hosted ZITADEL deployment;
mctl-gitops#1503 — platform hostname reservation protecting auth.mctl.ai;
mctl-api#434 — ZITADEL registered in the federation registry with issuer/audience validation;
mctl-api#435 — explicit verified linking of existing identities to ZITADEL identities;
mctl-api#374 — core provider-agnostic federation/authentication boundary;
mctl-api#572 / PR #622 — canonical actor/subject identity evidence for Slice D.

Also delivered, non-blocking for epic completion:

mctl-gitops#1500 — platform sign-in moved to ZITADEL.

Required work — final status 2026-10-18

All nine required work items in roadmap/epics/unified-identity.yaml are closed as completed. Roadmap publication run 39116819850, captured 2026-10-18T09:49:09Z, reports completion: complete, required 9/9, blocking: [], state healthy, drift 0 and error 0.

MCP client identity: `mctl-api#375`, done

CIMD was implemented in mctl-api#624 and merged as abc14bbd on 2026-10-18, after main was merged in and the review was re-run on the final head (Claude APPROVED, agy PASS, CI green).
It was released in mctl-api 4.100.0, and mctl-gitops 3a949776e bumped prod to it.
It ships dark: OAUTH_CIMD_ENABLED is unset, so it is off.
Live check after rollout:
- discovery does not advertise client_id_metadata_document_supported;
- an https:// URL client_id gets 400;
- a DCR client's authorize request still reaches the normal sign-in page, with no consent step;
- an existing OAuth token still validates (identity/self returns 200);
- no ERROR logs.
Turning CIMD on is a separate GitOps decision: it waits for a real directly connected client and an exact-host allowlist.

Delegated agent identity: `mctl-api#376`, done

mctl-agents#636 shipped in 1.76.0.
DevLoop 655 proved the chain wi_ → we_ → aar_ → art_ → mctl-agents-approve:
- the grant was consumed once;
- the decider was recorded as decided_by_principal_id=prn_….
mctl-gitops#1913 turned the legacy free-text relay off on 2026-10-16.
#376 was closed completed on 2026-10-18; the evidence is in its closing comment.

Canonical authorization cut-over: `mctl-api#377`, done

TENANT_AUTHZ_MODE=principal has been live since 2026-10-17 (mctl-gitops#1873, mctl-api 4.91.0).
The owner cut the shadow window short.
Live checks passed, including a ZITADEL customer sign-in to the portal.

Foundation, done

mctl-api#373, #374, #434, #435
mctl-gitops#1499, #1503

Non-blocking follow-ups

mctl-api#422: carry the numeric GitHub id in legacy local OAuth JWTs (closed, optional);
mctl-api#629: resolve surface binding and human-input respondents by principal (a ZITADEL-only follow-up to #377);
mctl-api#684: CIMD hardening P3s from the #624 review, and the rule for turning CIMD on;
mctl-gitops#1504 — separate tenant applications onto a PSL-listed registrable domain;
mctl-gitops#1501 — tenant-application SSO through per-tenant ZITADEL organizations.

Success criteria

Human authentication uses self-hosted ZITADEL, while MCTL authorization is keyed by stable canonical principals.
A user's authorization is unchanged when the same principal is reached through a different linked identity provider.
GitHub username/email is not a canonical authorization or tenant-membership key.
MCP OAuth keeps client identity distinct from the human/service principal; CIMD is supported additively and DCR remains available where required.
Agent actions distinguish the acting agent from the on-behalf-of subject, and the subject is derived only from server-held grants/decisions.
Tenant membership and policy checks resolve against canonical principals and fail closed when the required principal cannot be resolved.
Replacing the human IdP in the future would not require changing MCTL's authorization model.

Status

Complete. The canonical manifest moves to lifecycle: completed in a separate .github PR. The items under non-blocking follow-ups continue outside this epic.

Roadmap control-plane

Canonical graph: roadmap/epics/unified-identity.yaml (completion.mode: allRequired).

Human sign-in inventory and ZITADEL plan (2026-10-04)

Sources (all read-only):

every Ingress and IngressRoute in the cluster, and every Traefik Middleware, including forward-auth targets
mctl-gitops main
the app repositories, read where each host's login is implemented

Status before this inventory: ZITADEL (auth.mctl.ai) was the only sign-in for Argo CD and Forgejo. Since then, Grafana and Argo Workflows have also moved: mctl-gitops#1590, #1594, #1595, #1597, and #1598 (Secure cookie).

Host	Login today	Per-tenant scope	ZITADEL plan	Effort	Risk / blockers
`ops.mctl.ai` (Argo CD)	ZITADEL only	yes: `groups` maps to `argocd/rbac/tenants/<tenant>.csv`; opt-in `"argocd"`	done (#1500)	–	session cookie name is hard-coded (#1560 residual)
`git.mctl.ai` (Forgejo)	ZITADEL only	per account; registration is off	done	–	#1560: `__Host-` session cookie name
`secrets.mctl.ai` (Vault)	ZITADEL OIDC (`auth/oidc`); tokens remain	yes: `admins` maps to the admin policy; a tenant gets read on its own paths; opt-in `"vault"`	done (#1572–#1574, #1589)	–	–
`grafana.mctl.ai`	ZITADEL, admins only (#1597). Before: Dex. `admins` got Admin, every other Dex user got Viewer of the single, cluster-wide org	none. One org, emptyDir database	done for admins. Tenant orgs are designed in mctl-gitops#1601 (owner decision)	M (tenant orgs)	state is lost on every restart (#1601)
`workflows.mctl.ai`	ZITADEL (#1595). Before: Dex	yes: `admins` maps to `argo-workflows-admin` (all namespaces); `<tenant>` maps to `sso-team-<tenant>`, bound in its own namespace only; opt-in `"workflows"`	done; Secure cookies via server TLS (#1598)	–	`authorization` cookie name is hard-coded (#1560 residual)
`app.mctl.ai` (Backstage portal)	GitHub only. The portal's OIDC provider (`/api/oidc-provider`) issues tenant groups from `tenant_members`, keyed by GitHub login	yes (`tenant_members`)	add a ZITADEL OIDC sign-in provider plus a resolver from ZITADEL user to catalog user / GitHub login	M–L (code, mctl-portal)	everything is keyed by GitHub login, so it needs an identity mapping. The forward-auth hosts below depend on it
`temporal.mctl.ai`, `claude-remote.mctl.ai`, `admins-/labs-/ovk-openclaw.mctl.ai`	Traefik forwardAuth to the portal's `/api/oidc-provider/forward-auth?tenant=<t>&service=<s>`. It checks the host registration, a session (GitHub), and a role in the tenant. Tenants: `admins`, `labs`, `labs`, `admins`/`labs`/`ovk`	yes	(a) follows the portal switch, or (b) oauth2-proxy against ZITADEL plus a `groups` check	(a) free after the portal; (b) M, a new component	OpenClaw also uses api.mctl.ai OAuth and the portal's codex callback
`api.mctl.ai` (mctl-api)	bearer only: GitHub token, Dex JWT (`DEX_ISSUER_URL`), its own MCP OAuth JWT (upstream GitHub), static service tokens. A ZITADEL provider is already configured (`MCTL_OIDC_PROVIDERS`)	groups from Dex, or from GitHub org plus gitops tenants. ZITADEL users get no groups yet	tenant and admin access from ZITADEL	M	blocked on mctl-api#377 (groups) and #435 (linking)
`coolify.mctl.ai` (coolify-mcp)	its own OAuth 2.1 for MCP; upstream GitHub or Google; per-identity Coolify enrolment	per identity, not per tenant	add a generic-OIDC upstream	M (code)	–
`tg.mctl.ai` (mctl-telegram)	its own OAuth authorization server; upstream is Telegram OIDC	per Telegram user	at most a linked identity: the Telegram account is the identity	L	low value
`agent.mctl.ai`	machine-only bearer tokens; no human UI	–	none	–	–
`projects.mctl.ai`	Cloudflare Access JWT	–	ZITADEL as an Access IdP (Cloudflare track)	S	–
`academy.mctl.ai`	better-auth with GitHub	per user	better-auth `genericOAuth`	S (code)	end-user product; optional
`seerrsense.mctl.ai`	Google OIDC, fixed issuer	–	make the issuer configurable	S–M (code)	optional
`alice.mctl.ai`	its own OAuth with a consent screen; upstream not confirmed	–	to be checked	?	–
`mctl.ai` (web)	Cloudflare Worker, GitHub OAuth	–	optional	S	–
`rewards.mctl.ai`, pairdesk	Telegram auth	–	not applicable	–	–
`docs.mctl.ai`, `ui.mctl.ai`, tenant apps	public, or the app's own login (`erpact-*` already use ZITADEL, #1501)	–	out of scope	–	–
`auth.mctl.ai`	ZITADEL itself	–	–	–	login v2 `sessions` cookie name is hard-coded (#1560)

Removing Dex entirely

Dex (ops.mctl.ai/api/dex) now serves only one consumer: mctl-api, which uses it to verify tokens. Its mctl-api client has no callback route in mctl-api; it only provides an audience.

Order of removal. On HOLD until the owner has done the live checks on Grafana and Workflows, and mctl-api#377 has landed:

Grafana and Argo Workflows on ZITADEL: done (#1595, #1597).
mctl-api: set DEX_ISSUER_URL explicitly to empty. It defaults to the Dex URL. Do this only after mctl-api#377, or tenant access over tokens is lost.
Remove the Dex Deployment and config (argocd/templates/dex.yaml, dex-config.yaml) and the dex: / server.dex.* values in argocd/values.yaml. Clean up the leftover argocd-dex-server objects.
argocd/templates/argocd-github-oauth.yaml: keep webhook.github.secret (#970). Drop only argo-workflows-sso-client-secret and backstage-oidc-client-*.
Portal: drop the OIDC-provider client whose redirect is ops.mctl.ai/api/dex/callback.
Vault, done by the owner, not by the PR: retire platform/argo-workflows/sso, today shared by all three Dex clients and by Grafana. Then drop the oidc-client-secret key from grafana-admin-secret and the argo-workflows-sso ExternalSecret.

Owner live checks, 2026-10-04:

Argo Workflows (workflows.mctl.ai): sign-in through ZITADEL with the platform admin account works (mctl-gitops#1595). The Secure session cookies (#1598) are live.
Grafana (grafana.mctl.ai): sign-in through ZITADEL works and the account lands as Grafana Admin (mctl-gitops#1597).
Grafana sign-out now ends the ZITADEL session (mctl-gitops#1605, #1607). Re-opening grafana.mctl.ai asks for the account again instead of signing back in silently.
Grafana state has moved from an emptyDir SQLite to its own database on shared-pg (mctl-gitops#1604, #1608, slice 1 of mctl-gitops#1601). Grafana is still admins-only.

Grafana persistence (mctl-gitops#1601 slice 1) is done. Grafana now keeps its state in its own database on shared-pg (mctl-gitops#1604, #1608). Rows survived a pod restart, and the owner's sign-in after the switch persists as GrafanaAdmin. Grafana remains admins-only.

Vault human sign-in through ZITADEL (2026-10-04)

Humans now sign in to Vault (secrets.mctl.ai) through ZITADEL. Both sides are declarative: zitadel-iac configures ZITADEL, and a new in-cluster OpenTofu Job, vault-human-auth-iac, configures Vault.

PR	What
mctl-gitops#1572	Namespace `vault-human-auth-iac`, the hand-off Secret, and the Job's narrow Vault policy (bootstrapped once by the owner)
mctl-gitops#1573	ZITADEL project `Vault` (roles `admins`, `<tenant>`), opt-in tenant grants (`"vault": true`), the `vault` web client (code + PKCE). The shared `groups` action now covers Argo CD and Vault.
mctl-gitops#1574	The Vault OpenTofu root: `auth/oidc` ("MCTL login"), role `zitadel`, external groups `human-admins` (→ existing `admin`) and `human-tenant-<tenant>`. First rollout was plan-only.
mctl-gitops#1589	Guard widened to `create update`; human tokens capped at 1h
mctl-gitops#1596	Runbook: first login with the personal admin account
mctl-gitops#1623	Tenant humans read and write their own `secret/teams/<tenant>/` (no destroy, no metadata write; `<svc>/database` read-only)

Verified:

Locally, on ZITADEL v4.19.2 and Vault 1.17.2 with the real roots:
- a full code flow works for both the CLI and UI redirects;
- PKCE is S256;
- tenant isolation holds, proven both ways;
- each token's groups claim carries only the roles for its own client.
Live:
- the Job applied, and its hourly runs report "no changes";
- the owner's first UI login put their entity into human-admins → admin;
- grants for opted-in erpact users were created by zitadel-iac.

Open:

mctl-gitops#1578: guard fixture tests.
mctl-gitops#1624: CI policy matrix.
The auth/oidc → auth/mctl remount and the <username>-<sub> display name are approved and driven by a separate track.
The live tenant-login check is pending account activation.
The github auth method stays as the documented fallback until the owner decides otherwise.

Runbook: docs/runbooks/vault-human-auth.md.

Completed. All nine required work items are closed; the canonical manifest is lifecycle: completed (#198, 6f35424f). Roadmap publication run 39117400061 (captured 2026-10-18T10:00:48Z) reports unified-identity healthy, required 9/9, blocking [], drift 0, error 0. Non-blocking follow-ups continue outside the epic: mctlhq/mctl-api#629, mctlhq/mctl-api#684, mctlhq/mctl-gitops#1501, mctlhq/mctl-gitops#1504.

Post-completion evidence: mctl-api 4.100.1 in production (2026-10-18)

#685 shipped the CIMD hardening (Closes mctlhq/mctl-api#684). It was released as 4.100.1 rather than 4.101.0, because every commit in it is fix:, which makes this a patch under semver.

Release and rollout

Release PR chore(main): release 4.100.1 mctl-api#686 merged. Tag 4.100.1.
mctl-gitops release-deploy run 39125872339 succeeded. Bump commit 440a0b8a8 (chore: bump admins/mctl-api to 4.100.1).
Both mctl-api pods run ghcr.io/mctlhq/mctl-api:4.100.1 and are ready (started 12:08Z). No "level":"ERROR" lines in the first 15 minutes.

CIMD is still disabled

The deployment has no OAUTH_CIMD_* env var, so OAUTH_CIMD_ENABLED defaults to false.
/.well-known/oauth-authorization-server does not advertise client_id_metadata_document_supported.
/oauth/authorize with an HTTPS-URL client_id returns 400.

No regression in the OAuth and DCR flows

POST /oauth/register issues a dcr_… client.
/oauth/authorize for that client returns 200 with the sign-in page, offering both "Sign in with GitHub" and "Sign in with MCTL account".
An existing OAuth bearer token (the claude.ai MCP connector) still validates: mctl_whoami returns the owner with admin rights and the expected teams.

Roadmap projection

roadmap-publish run 39126814481, evaluator revision 6f35424f.
unified-identity: allRequired 9/9 complete, blocking: [], health healthy, drift 0, errors 0. Manifest lifecycle: completed.

Non-blocking follow-ups

CIMD: deferred P3 hardening, resolve before enabling in production mctl-api#687: the CIMD P3s deferred from #685, with an explicit gate to review and resolve them before enabling CIMD in production.
feat(unified-identity): offer tenant applications sign-in through a per-tenant ZITADEL organization mctl-gitops#1501: optional post-epic item, kept open.
feat(unified-identity): move tenant applications to a separate PSL-listed domain mctl-gitops#1504: tracked risk.
test(identity): live-check a ZITADEL-only human answering human input as principal:prn_ mctl-api#682: live check with a ZITADEL-only user.

Follow-up: on the owner's request the same line was re-released as mctl-api 4.101.0, a minor bump with no code change since 4.100.1.

Path: chore: release 4.101.0 mctl-api#688 (Release-As: 4.101.0), then release PR #689, then tag 4.101.0.
gitops release-deploy run 39130068735, bump 9c7787b42. Both pods run 4.101.0 (2/2 ready, 13:09Z) with 0 ERROR lines.
Re-checked on 4.101.0, CIMD still disabled: no OAUTH_CIMD_* env, no discovery field, an HTTPS client_id gets 400.
Re-checked on 4.101.0, OAuth and DCR flows still work: DCR registration, authorize returning the sign-in page, and an existing bearer token validating through mctl_whoami.

Full ZITADEL-callback sign-in verified live on mctl-api 4.101.0 (2026-10-18 13:11Z). It is the same OAuth code as 4.100.0 and 4.100.1.

The owner ran tools/mctl-token/mctl_token.py login, which registered a new DCR client dcr_7661a610… (log line OAuth client registered), and chose "Sign in with MCTL account".
Server log: oauth: ZITADEL sign-in principal_id=prn_01M39HRB7JRXZ2PHW5VVB695YZ login=mashkovd zitadel_subject=393619200037357269 result=issued.
The issued token resolves through /api/v1/identity/self to actor.principal_id=prn_01M39HRB7JRXZ2PHW5VVB695YZ, kind=human.
0 ERROR lines on both pods.

So the post-merge path of #624 (finishAuthorization from the ZITADEL callback) issues a code for a pre-registered DCR client without regression. This was not a required item; it is recorded as evidence.

mctl-api 4.101.1 is in production and carries the agreed #375/#687 hardening: #690 (netsafe 2001::/23 and 3fff::/20, plus the CIMD limits documentation), which closed mctlhq/mctl-api#687.

Release PR chore(main): release 4.101.1 mctl-api#691, tag 4.101.1. gitops release-deploy run 39131616174, bump 8601c8f97. Pods 2/2 ready on 4.101.1, 0 ERROR lines.
Smoke on 4.101.1: no OAUTH_CIMD_* env, no discovery field, an HTTPS client_id gets 400; DCR registration and authorize work; the ZITADEL-issued bearer token still resolves to prn_01M39HRB7JRXZ2PHW5VVB695YZ.
The roadmap evaluator still reads allRequired 9/9 complete. The epic stays closed as completed.

Navigation Menu

Repository navigation

Epic: MCTL Unified Identity on ZITADEL and Agent Identity #91

Goal

Architecture decision — current state

Completed foundation

Required work — final status 2026-10-18

MCP client identity: `mctl-api#375`, done

Delegated agent identity: `mctl-api#376`, done

Canonical authorization cut-over: `mctl-api#377`, done

Foundation, done

Non-blocking follow-ups

Success criteria

Status

Roadmap control-plane

Human sign-in inventory and ZITADEL plan (2026-10-04)

Removing Dex entirely

Vault human sign-in through ZITADEL (2026-10-04)

Post-completion evidence: mctl-api 4.100.1 in production (2026-10-18)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Epic: MCTL Unified Identity on ZITADEL and Agent Identity #91

Description

Goal

Architecture decision — current state

Completed foundation

Required work — final status 2026-10-18

MCP client identity: mctl-api#375, done

Delegated agent identity: mctl-api#376, done

Canonical authorization cut-over: mctl-api#377, done

Foundation, done

Non-blocking follow-ups

Success criteria

Status

Roadmap control-plane

Activity

mashkovd commented on Oct 4, 2026

Human sign-in inventory and ZITADEL plan (2026-10-04)

Removing Dex entirely

mashkovd commented on Oct 4, 2026

mashkovd commented on Oct 4, 2026

mashkovd commented on Oct 4, 2026

Vault human sign-in through ZITADEL (2026-10-04)

mashkovd commented on Oct 18, 2026

mashkovd commented on Oct 18, 2026

Post-completion evidence: mctl-api 4.100.1 in production (2026-10-18)

mashkovd commented on Oct 18, 2026

mashkovd commented on Oct 18, 2026

mashkovd commented on Oct 18, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

MCP client identity: `mctl-api#375`, done

Delegated agent identity: `mctl-api#376`, done

Canonical authorization cut-over: `mctl-api#377`, done