Okta dynamic SCEP challenges: Retry if Windows can't use it

[GrayW]

• @noahtalerman: Let's try check if it's a valid challenge and if not retry. Up to 10 times.

Goal

User story
As a Fleet administrator delivering Okta SCEP certificates to Windows hosts,
I want Fleet to obtain a challenge that Windows can actually use for each enrollment,
so that certificates issue reliably instead of failing whenever Okta generates a challenge containing a character Windows cannot encode.

Context

This is a completely Okta issue/bug, and the best we can do is implement a hacky workaround. Fleet 4.92 shipped a retry feature, which should improve the success rate with Okta, but not guarantee.

Windows encodes the SCEP challenge password as an ASN.1 PrintableString. The permitted set is letters, digits, space, and ' ( ) + , - . / : = ?. Okta's dynamic SCEP challenge uses the base64url alphabet, which adds - and _. The hyphen happens to be permitted; the underscore is not. A challenge containing _ installs as a profile, returns 200 to Fleet, and then fails on the device with CRYPT_E_INVALID_PRINTABLE_STRING in the SCEP exchange. This is why the customer reported the behavior as "erratic and will work occasionally": whether a given challenge works depends on whether Okta's generator happened to emit an underscore.

For the 32-character challenge shown on the bug (JURAzXStYElNpVi63B_ps6D0WxF7b3Gv), assuming a uniform draw from the 64-character base64url alphabet, roughly 40% of generated challenges will contain at least one underscore.

Open question to resolve first

Whether Okta exposes a challenge Fleet can fetch the way NDES does is unconfirmed, and it decides whether this story is buildable as written.

Okta's documentation does not list Windows under dynamic SCEP configuration

Proposed approach

Assuming Okta does expose a fetchable challenge:

1. Add an Okta dynamic SCEP CA type, or extend the custom SCEP CA with an optional dynamic-challenge configuration, so the challenge is retrieved per deployment rather than stored.
2. Fetch the challenge during Windows profile preprocessing, reusing the NDES branch's structure and its transient-versus-terminal error classification.
3. Validate the fetched challenge against windowsSCEPChallengeRegexp before substituting it. On a challenge Windows cannot encode, discard it and fetch another, up to a bounded number of attempts.
4. When the attempts are exhausted, fail the profile with a message that names the real cause, so the admin is not left guessing which side is at fault.

Product decision needed on the bounded retry count, since each attempt is an outbound call to Okta during profile deployment, and on what the admin-facing failure says when attempts run out.

Changes

Product

• UI changes: TODO. Depends on the spike. A new or extended CA type needs configuration UI under Settings > Integrations > Certificates. Insert the link to the relevant Figma "Ready" page.
• CLI (fleetctl) usage changes: TODO. Follows from the CA configuration shape.
• YAML changes: TODO. A new CA type or new fields need entries in the YAML files doc page. Specify as a PR to the reference docs release branch.
• REST API changes: TODO. The certificate authorities endpoints gain a type or fields. Specify as a PR to the reference docs release branch.
• Fleet's agent (fleetd) changes: No changes. The challenge is substituted server-side during profile preprocessing.
• Fleet server configuration changes: No changes
• Exposed, public API endpoint changes: No changes. Fleet makes an outbound call to Okta; no new inbound surface.
• fleetdm.com changes: No changes
• GitOps mode UI changes: TODO. Certificate authority configuration is GitOps-managed, so confirm the new fields follow the existing gate.
• GitOps generation changes: TODO. Confirm fleetctl generate-gitops emits the new CA configuration.
• Activity changes: TODO. Adding and editing a certificate authority already emits activities. Confirm whether a new CA type needs its own display name.
• Permissions changes: No changes. Follows existing certificate authority permissions.
• Changes to paid features or tiers: TODO. Custom SCEP and NDES are Fleet Premium, so this is Premium-only. Confirm no pricing page change is needed.
• My device and fleetdm.com/better changes: No changes
• Usage statistics: No changes
• Compatibility: No changes. Existing static custom SCEP configurations must keep working untouched.
• Other reference documentation changes: TODO. Document which challenge characters Windows supports, and the Okta guidance that results from the spike. This is worth writing regardless of which way the spike goes.
• First draft of test plan added
• Once shipped, requester has been notified (customer-rembrandt)
• Once shipped, dogfooding issue has been filed

Engineering

• Test plan is finalized
• Contributor API changes: No changes
• Feature guide changes: TODO. The certificate authority guide needs the Okta dynamic SCEP path and the Windows character constraint.
• Database schema migrations: TODO. A new CA type or new columns on the custom SCEP CA configuration. Confirm the challenge is stored encrypted, or not stored at all if it is fetched per deployment.

Risk assessment

• Requires testing in a hosted environment: yes. Verifying this end to end needs a real Okta tenant with dynamic SCEP configured plus a Windows host, since the failure only appears in the on-device SCEP exchange.
• Requires load testing: no. One or a few outbound calls per Windows profile deployment, the same shape as the existing NDES fetch, which is not load tested either.
• Risk level: Low
• Risk description: The change is additive and existing static custom SCEP configurations are unaffected. Two things to watch. First, an outbound fetch during profile preprocessing adds a dependency on Okta's availability to profile delivery, which is why the transient-versus-terminal classification must be reused rather than reinvented; a temporary Okta outage must not permanently fail the profile. Second, a retry loop inside preprocessing needs a hard bound, or a persistently bad challenge generator turns into repeated outbound calls on every deployment attempt.

Test plan

Core flow

• With Okta dynamic SCEP configured, deploy a certificate profile to a Windows host and confirm the certificate is installed and the profile reports Verified.
• Confirm the certificate appears in the expected store on the device, matching the InstallLocation the profile requests.
• Force Okta to return a challenge containing _, then confirm Fleet discards it, fetches another, and the enrollment still succeeds.
• Confirm the host's certificate details in Fleet reflect the issued certificate.

Edge cases

• Every retry returns an unusable challenge. Confirm Fleet fails the profile with a message naming the cause, and does not report Verified.
• Okta returns a challenge that is usable but expired or already consumed. Confirm the resulting failure is reported rather than silently retried forever.
• Okta is unreachable or returns 5xx during preprocessing. Confirm the profile is treated as transient and retried later, not permanently failed.
• Okta returns 4xx from bad credentials. Confirm this is terminal and the admin-facing detail says so.
• An existing static custom SCEP CA with a valid challenge. Confirm behavior is unchanged and no fetch is attempted.
• An existing static custom SCEP CA whose challenge contains _. Confirm the failure from PR Windows SCEP profiles now fail with non-printable chars #49887 still appears unchanged.
• A challenge containing a space, including leading and trailing. This is permitted and was previously verified to enroll on Windows 11, so confirm it is not rejected by the new path.
• A challenge containing - but no _. Permitted, must not be rejected.
• An NDES challenge containing a character outside the PrintableString set. Confirm the profile now fails with a clear message rather than reporting Verified with no certificate.
• The same Okta profile delivered to a macOS host, confirming the Apple path is unaffected and does not inherit the Windows character restriction.
• Two hosts receiving the same profile at once, confirming each gets its own challenge and neither reuses the other's.
• Verify the new CA configuration is readable and writable only by the roles that can already manage certificate authorities.

Supplemental testing

Testing notes

Confirmation

1. Engineer: Added comment to user story confirming successful completion of test plan (include any special setup, test data, or configuration used during development/testing if applicable).
2. QA: Added comment to user story confirming successful completion of test plan.
3. QA: Determined whether this story needs Playwright automation.
   • Needs automation: Yes / No
   • If yes, filed a follow-up issue in the :help-qa project with status "Needs automation":

[GrayW]

#47492 Re-opened with a P1

[sharon-fdm]

@getvictor , is this a duplication of something you already fixed?

(EDIT from @nulmete): maybe fixed by #49887?