Skip to content

Android certificate crud: validate variable replacement - #36648

Merged
getvictor merged 4 commits into
mainfrom
36533-android-certificate-crud-validate-variable-replacement
Dec 5, 2025
Merged

Android certificate crud: validate variable replacement#36648
getvictor merged 4 commits into
mainfrom
36533-android-certificate-crud-validate-variable-replacement

Conversation

@juan-fdz-hawa

@juan-fdz-hawa juan-fdz-hawa commented Dec 3, 2025

Copy link
Copy Markdown
Contributor

Related issue: Resolves #36533

If variables can't be interpolated return 400.

Checklist for submitter

If some of the following don't apply, delete the relevant line.

  • Changes file added for user-visible changes in changes/, orbit/changes/ or ee/fleetd-chrome/changes.
    See Changes files for more information.

Testing

  • Added/updated automated tests

Summary by CodeRabbit

  • Bug Fixes
    • Corrected HTTP status code returned when certificate template variable interpolation fails
    • Certificate delivery status now properly reflects failed interpolation, improving visibility into deployment issues

✏️ Tip: You can customize this high-level summary in your review settings.

If variables can't be interpolated return 400.
@codecov

codecov Bot commented Dec 3, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 59.37500% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.99%. Comparing base (068ffea) to head (7f13706).
⚠️ Report is 114 commits behind head on main.

Files with missing lines Patch % Lines
server/service/certificate_templates.go 0.00% 4 Missing and 2 partials ⚠️
server/service/certificates.go 50.00% 2 Missing and 2 partials ⚠️
...rver/datastore/mysql/host_certificate_templates.go 83.33% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #36648      +/-   ##
============================================
+ Coverage     65.95%   65.99%   +0.03%     
  Complexity        2        2              
============================================
  Files          2247     2248       +1     
  Lines        183631   183768     +137     
  Branches       7591     7591              
============================================
+ Hits         121121   121269     +148     
+ Misses        51436    51415      -21     
- Partials      11074    11084      +10     
Flag Coverage Δ
backend 67.70% <59.37%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

jacobshandling
jacobshandling previously approved these changes Dec 3, 2025

@getvictor getvictor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@juan-fdz-hawa Thank you for taking this task.

We need a bit of a customized flow here. This endpoint will only be hit by a specific Android agent before fetching a certificate.

Thus, if any fleet vars specified in the SubjectName are empty/missing, then we need to mark the HostCertificateTemplate with status fleet.MDMDeliveryFailed for the host and certificate template.

This way the certificate/profile will immediately show up as failed on the Host Details OS settings modal. And the Android agent will not try to use SCEP to get the cert.

After updating the DB, this endpoint should return OK with status=fleet.MDMDeliveryFailed in its regular payload (instead of Pending). We do not need to decrypt/return SCEPchallenge/fleetChallenge if status is not Pending.

This is a similar flow we do with MDM profiles when we can't replace variables:

profile.Status = &fleet.MDMDeliveryFailed

For reference, this is the updated flowchart for Android certs:
https://github.com/fleetdm/fleet/pull/36538/files?short_path=3ef7de3#diff-3ef7de32380db57632181ccc1ebdaf92304c00a42d6a9439350d34992d772479

@juan-fdz-hawa

Copy link
Copy Markdown
Contributor Author

@juan-fdz-hawa Thank you for taking this task.

We need a bit of a customized flow here. This endpoint will only be hit by a specific Android agent before fetching a certificate.

Thus, if any fleet vars specified in the SubjectName are empty/missing, then we need to mark the HostCertificateTemplate with status fleet.MDMDeliveryFailed for the host and certificate template.

This way the certificate/profile will immediately show up as failed on the Host Details OS settings modal. And the Android agent will not try to use SCEP to get the cert.

After updating the DB, this endpoint should return OK with status=fleet.MDMDeliveryFailed in its regular payload (instead of Pending). We do not need to decrypt/return SCEPchallenge/fleetChallenge if status is not Pending.

This is a similar flow we do with MDM profiles when we can't replace variables:

profile.Status = &fleet.MDMDeliveryFailed

For reference, this is the updated flowchart for Android certs: https://github.com/fleetdm/fleet/pull/36538/files?short_path=3ef7de3#diff-3ef7de32380db57632181ccc1ebdaf92304c00a42d6a9439350d34992d772479

Thank you for the context!

I have a couple of follow up questions:

  1. What should the behavior be for the GET /fleet/certificates/:id if variable interpolation fails?
  2. Why can't the Android client use the PUT fleetd/certificates/:id/status end-point to update the certificate status if interpolation fails (based on the HTTP status)? Doing any data mutation inside a GET feels a little bit out of place, and also the update behavior is already implemented in the PUT fleetd/certificates/:id/status end-point.

@getvictor

getvictor commented Dec 4, 2025

Copy link
Copy Markdown
Member

I have a couple of follow up questions:

  1. What should the behavior be for the GET /fleet/certificates/:id if variable interpolation fails?
  2. Why can't the Android client use the PUT fleetd/certificates/:id/status end-point to update the certificate status if interpolation fails (based on the HTTP status)? Doing any data mutation inside a GET feels a little bit out of place, and also the update behavior is already implemented in the PUT fleetd/certificates/:id/status end-point.
  1. By variable interpolation fail, you mean some sort of unexpected server error? FLEET_VAR_* should have been validated on certificate template save. So all FLEET_VARs should be valid in the subject name. So the only expected error is if a valid FLEET_VAR_* contains an empty/unknown value. If any other unexpected error occurs with variables, then it is ok to return server errror 5xx. (But in general we should avoid 5xx if we can.)

  2. That flow seems awkward. Server will send error to Android app, and Android app will send the same error back to server. Also, Android could go offline. It seems a better UX if we can surface the error to the IT admin sooner. cc: @marko-lisica

@juan-fdz-hawa

Copy link
Copy Markdown
Contributor Author

I have a couple of follow up questions:

  1. What should the behavior be for the GET /fleet/certificates/:id if variable interpolation fails?
  2. Why can't the Android client use the PUT fleetd/certificates/:id/status end-point to update the certificate status if interpolation fails (based on the HTTP status)? Doing any data mutation inside a GET feels a little bit out of place, and also the update behavior is already implemented in the PUT fleetd/certificates/:id/status end-point.
  1. By variable interpolation fail, you mean some sort of unexpected server error? FLEET_VAR_* should have been validated on certificate template save. So all FLEET_VARs should be valid in the subject name. So the only expected error is if a valid FLEET_VAR_* contains an empty/unknown value. If any other unexpected error occurs with variables, then it is ok to return server errror 5xx. (But in general we should avoid 5xx if we can.)

The implementation we currently have in place for GET /fleet/certificates/:id returns a 5xx on error (either if value can't be read or if empty), just wanted to double check if we wanted to change the current implementation, but seems to me like we want to keep things as they are on that end-point.

  1. That flow seems awkward. Server will send error to Android app, and Android app will send the same error back to server. Also, Android could go offline. It seems a better UX if we can surface the error to the IT admin sooner. cc: @marko-lisica

Ok, no problem. I'll update the host_certificate_templates status on GET /fleetd/certificates/:id on failure. Just a couple of more questions:

  1. Are we guaranteed that a host_certificate_templates row will exists at that point in time?
  2. If not, what should be the expected behavior? Do we create a new host_certificate_templates table entry? If so, what should the fleet_challenge value should be? An empty string?

@getvictor

Copy link
Copy Markdown
Member

The implementation we currently have in place for GET /fleet/certificates/:id returns a 5xx on error (either if value can't be read or if empty), just wanted to double check if we wanted to change the current implementation, but seems to me like we want to keep things as they are on that end-point.

I don't think we need the fleet/certificates/:id endpoint. No one uses it, right? We should only have fleetd/certificates/:id which is used by Android agent.

@marko-lisica

Copy link
Copy Markdown
Member

I don't think we need the fleet/certificates/:id endpoint. No one uses it, right? We should only have fleetd/certificates/:id which is used by Android agent.

@getvictor I think you're right. When I specified that one, I thought the agent would use it. I didn't add it to the API PR for #30876

@getvictor

Copy link
Copy Markdown
Member

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Dec 5, 2025

Copy link
Copy Markdown
Contributor
✅ Actions performed

Full review triggered.

@coderabbitai

coderabbitai Bot commented Dec 5, 2025

Copy link
Copy Markdown
Contributor

Walkthrough

This PR changes certificate template variable interpolation failure handling from returning errors to upserting failed delivery status records. It refactors the status update method from Update to Upsert pattern, supporting record insertion when needed, and updates integration tests to verify the new behavior.

Changes

Cohort / File(s) Summary
Changelog Entry
changes/36533-use-proper-status-code-on-failure-to-interpolate
Adds changelog entry documenting the fix for corrected status code on certificate template variable interpolation failures.
Datastore Interface & Mock
server/fleet/datastore.go, server/mock/datastore_mock.go
Renames public method UpdateCertificateStatus to UpsertCertificateStatus across interface and mock datastore, updating type aliases, fields, and method signatures while preserving control flow.
MySQL Datastore Implementation
server/datastore/mysql/host_certificate_templates.go
Implements upsert logic: attempts UPDATE on existing record; if no rows affected, checks certificate template existence; if exists, inserts new record; if not, returns tailored not-found error. Adds new SQL statements and enhanced error handling.
MySQL Datastore Tests
server/datastore/mysql/host_certificate_templates_test.go
Renames test to testUpsertHostCertificateTemplateStatus, updates method calls to UpsertCertificateStatus, introduces multiple certificate template scenarios, and aligns assertions with new upsert behavior.
Service Layer
server/service/certificates.go
Changes GetDeviceCertificateTemplate to upsert failed delivery status on variable interpolation failure and return the updated certificate instead of error; updates call site from UpdateCertificateStatus to UpsertCertificateStatus.
Integration Tests
server/service/integration_core_test.go
Adds test logic to fetch certificate templates for team, perform node-key authenticated GET to certificate endpoint, and assert delivery status is MDMDeliveryFailed; consolidates duplicate template retrieval.

Sequence Diagram(s)

sequenceDiagram
    participant Client as Android Device/Client
    participant Service as Fleet Service
    participant Datastore as MySQL Datastore
    
    Client->>Service: Request device certificate template
    activate Service
    
    Service->>Service: Attempt to interpolate variables<br/>(replace placeholders)
    
    alt Variable Interpolation Success
        Service->>Datastore: UpsertCertificateStatus<br/>(MDMDeliverySuccess)
        activate Datastore
        Datastore->>Datastore: UPDATE existing record
        Datastore-->>Service: Success
        deactivate Datastore
        Service-->>Client: Return certificate (200)
    else Variable Interpolation Fails
        rect rgb(255, 200, 200)
            Note over Service: New Behavior:<br/>Record failure instead of error
            Service->>Datastore: UpsertCertificateStatus<br/>(MDMDeliveryFailed, error detail)
            activate Datastore
            Datastore->>Datastore: UPDATE existing record
            alt No rows affected
                Datastore->>Datastore: SELECT template exists?
                alt Template exists
                    Datastore->>Datastore: INSERT new record
                else Template not found
                    Datastore-->>Service: Error (template not found)
                end
            end
            Datastore-->>Service: Status recorded
            deactivate Datastore
        end
        Service-->>Client: Return certificate with<br/>MDMDeliveryFailed status (200)
    end
    
    deactivate Service
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

  • MySQL upsert logic: New conditional insert pattern after update requires careful review of transaction semantics and error paths
  • Interface propagation: Method rename across interface, mock, service, and multiple call sites requires verification of all update sites
  • Service layer behavior change: Error handling restructuring from returning errors to recording failed status; verify certificate object state and client response handling
  • Test coverage: Substantive test updates and new integration test path for failure scenario need careful validation

Possibly related PRs

Suggested reviewers

  • sgress454

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title "Android certificate crud: validate variable replacement" is concise and directly related to the main change: handling variable interpolation failure for Android certificates by marking status as failed instead of returning an error.
Description check ✅ Passed The PR description includes the related issue (#36533), a clear summary of the change (return 400 when variables can't be interpolated), and completed checkboxes for changes file and tests.
Linked Issues check ✅ Passed The PR implements the core requirements from #36533: when variable interpolation fails, mark HostCertificateTemplate as MDMDeliveryFailed and return failed status instead of an error, mirroring MDM profile behavior.
Out of Scope Changes check ✅ Passed All changes are directly related to the linked issue objectives: renaming UpdateCertificateStatus to UpsertCertificateStatus to support upsert logic, updating certificate service to mark failed status on interpolation failure, and adding test coverage.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch 36533-android-certificate-crud-validate-variable-replacement

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
server/service/integration_core_test.go (1)

8034-8046: Make certificate template selection deterministic and add nil‑safety in assertions

Right now the test assumes savedCertificateTemplates[0] is the correct template and that Certificate/Status are always non‑nil:

  • savedCertificateTemplates[0] relies on datastore ordering and non‑empty results; later checks (including subject interpolation against hardware serial) implicitly expect this to be “Template 1”.
  • *getCertResp.Certificate.Status will panic if either Certificate or Status is unexpectedly nil; a require.NotNil would yield a clearer test failure.

Consider tightening this to assert size and pick the intended template by name, and to guard the dereference:

-	savedCertificateTemplates, _, err := s.ds.GetCertificateTemplatesByTeamID(ctx, team.ID, fleet.ListOptions{Page: 0, PerPage: 10})
-	require.NoError(t, err)
-	certID := savedCertificateTemplates[0].ID
+	savedCertificateTemplates, _, err := s.ds.GetCertificateTemplatesByTeamID(ctx, team.ID, fleet.ListOptions{Page: 0, PerPage: 10})
+	require.NoError(t, err)
+	require.NotEmpty(t, savedCertificateTemplates)
+
+	var certID uint
+	for _, tmpl := range savedCertificateTemplates {
+		if tmpl.Name == "Template 1" {
+			certID = tmpl.ID
+			break
+		}
+	}
+	require.NotZero(t, certID, "expected to find certificate template %q for this test", "Template 1")
@@
-	require.NoError(t, json.NewDecoder(resp.Body).Decode(&getCertResp))
-	require.NoError(t, resp.Body.Close())
-	require.Equal(t, *getCertResp.Certificate.Status, fleet.MDMDeliveryFailed)
+	require.NoError(t, json.NewDecoder(resp.Body).Decode(&getCertResp))
+	require.NoError(t, resp.Body.Close())
+	require.NotNil(t, getCertResp.Certificate)
+	require.NotNil(t, getCertResp.Certificate.Status)
+	require.Equal(t, fleet.MDMDeliveryFailed, *getCertResp.Certificate.Status)

This removes any dependency on implicit ordering and makes failures around the returned payload easier to diagnose.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3bb29ec and 7f4e5ba.

📒 Files selected for processing (7)
  • changes/36533-use-proper-status-code-on-failure-to-interpolate (1 hunks)
  • server/datastore/mysql/host_certificate_templates.go (3 hunks)
  • server/datastore/mysql/host_certificate_templates_test.go (5 hunks)
  • server/fleet/datastore.go (1 hunks)
  • server/mock/datastore_mock.go (3 hunks)
  • server/service/certificates.go (2 hunks)
  • server/service/integration_core_test.go (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.go

⚙️ CodeRabbit configuration file

When reviewing SQL queries that are added or modified, ensure that appropriate filtering criteria are applied—especially when a query is intended to return data for a specific entity (e.g., a single host). Check for missing WHERE clauses or incorrect filtering that could lead to incorrect or non-deterministic results (e.g., returning the first row instead of the correct one). Flag any queries that may return unintended results due to lack of precise scoping.

Files:

  • server/datastore/mysql/host_certificate_templates.go
  • server/service/certificates.go
  • server/fleet/datastore.go
  • server/mock/datastore_mock.go
  • server/service/integration_core_test.go
  • server/datastore/mysql/host_certificate_templates_test.go
🧠 Learnings (3)
📓 Common learnings
Learnt from: getvictor
Repo: fleetdm/fleet PR: 36139
File: android/app/src/main/java/com/fleetdm/agent/scep/ScepClientImpl.kt:75-76
Timestamp: 2025-11-26T18:58:18.865Z
Learning: In Fleet's Android MDM agent SCEP implementation (android/app/src/main/java/com/fleetdm/agent/scep/ScepClientImpl.kt), OptimisticCertificateVerifier is intentionally used because: (1) SCEP URL is provided by authenticated MDM server, (2) challenge password authenticates enrollment, (3) enterprise SCEP servers use internal CAs not in system trust stores, (4) enrolled certificate is validated when used.
📚 Learning: 2025-08-08T07:40:05.301Z
Learnt from: getvictor
Repo: fleetdm/fleet PR: 31726
File: server/datastore/mysql/labels_test.go:2031-2031
Timestamp: 2025-08-08T07:40:05.301Z
Learning: In fleetdm/fleet repository tests (server/datastore/mysql/labels_test.go and similar), using testing.T.Context() is valid because the project targets a recent Go version where testing.T.Context() exists. Do not suggest replacing t.Context() with context.Background() in this codebase.

Applied to files:

  • server/datastore/mysql/host_certificate_templates.go
📚 Learning: 2025-07-08T16:12:48.797Z
Learnt from: getvictor
Repo: fleetdm/fleet PR: 30589
File: ee/server/service/hostidentity/depot/depot.go:108-111
Timestamp: 2025-07-08T16:12:48.797Z
Learning: In ee/server/service/hostidentity/depot/depot.go, the SCEP depot interface methods like Put() do not accept context parameters, and the common_mysql.WithRetryTxx callback function type TxFn only receives a transaction parameter, not a context. Therefore, using context.Background() in tx.ExecContext calls within the transaction callback is the correct approach.

Applied to files:

  • server/datastore/mysql/host_certificate_templates.go
  • server/datastore/mysql/host_certificate_templates_test.go
🧬 Code graph analysis (2)
server/fleet/datastore.go (1)
server/fleet/mdm.go (1)
  • MDMDeliveryStatus (462-462)
server/service/integration_core_test.go (2)
server/fleet/app.go (1)
  • ListOptions (1251-1275)
server/fleet/mdm.go (1)
  • MDMDeliveryFailed (500-500)
🔇 Additional comments (9)
changes/36533-use-proper-status-code-on-failure-to-interpolate (1)

1-1: LGTM! Clear and concise changelog entry.

The changelog entry accurately describes the fix for handling certificate template variable interpolation failures.

server/service/certificates.go (2)

405-405: LGTM! Method rename reflects upsert semantics.

The change from UpdateCertificateStatus to UpsertCertificateStatus correctly reflects the new semantics where the status record is inserted if it doesn't exist. This aligns with the PR objectives to handle cases where the host_certificate_templates row may not yet exist.


157-170: Good implementation of failed status handling with proper SQL filtering.

The implementation correctly aligns with PR objectives by upserting failed status and returning the certificate instead of an error. The underlying SQL query properly filters by both host_uuid AND certificate_template_id in the UPDATE clause (line 178 of host_certificate_templates.go), ensuring updates target the correct record.

Note that if UpsertCertificateStatus itself fails, an error is still returned—this breaks the "always return OK" pattern but is necessary since we couldn't record the failure.

server/service/integration_core_test.go (1)

8058-8058: Good negative‑path coverage for missing node key on fleetd certificate fetch

The unauthorized GET /api/fleetd/certificates/{id} case correctly verifies that a missing Authorization/node key yields 401 and ensures the response body is closed; no changes needed here.

server/fleet/datastore.go (1)

2516-2516: LGTM!

The rename from UpdateCertificateStatus to UpsertCertificateStatus accurately reflects the new behavior where the method inserts a record when none exists for the host/template pair.

server/datastore/mysql/host_certificate_templates.go (1)

168-219: Upsert implementation looks correct.

The update-first, then check-and-insert pattern correctly handles both scenarios:

  1. Updating existing host_certificate_templates records
  2. Creating new records when variable interpolation fails (per PR objectives)

The empty fleet_challenge on insert aligns with the requirement to not return/decrypt SCEP challenge when marking status as failed.

server/datastore/mysql/host_certificate_templates_test.go (1)

615-735: Test coverage for upsert behavior is comprehensive.

The test correctly covers:

  1. Updating an existing record (ct1 with pre-inserted host_certificate_templates row)
  2. Creating a new record when none exists (ct2 without pre-existing row)
  3. Invalid status validation
  4. Detail field propagation

The use of CreateCertificateTemplate API instead of ad-hoc SQL is a good improvement for test maintainability.

server/mock/datastore_mock.go (2)

9784-9788: UpsertCertificateStatus mock method correctly wires to function field

The new UpsertCertificateStatus method uses the standard pattern (lock, set ...Invoked, unlock, delegate to func) and passes all arguments through unchanged. This cleanly maintains behavior with the updated name.


1640-1640: UpsertCertificateStatus mock type and fields are correctly implemented

The UpsertCertificateStatusFunc type at line 1640 and corresponding DataStore fields (lines 4090-4091) properly align with the method implementation (lines 9784-9788), following the standard mock pattern. Parameter signatures match and the invocation flag convention is consistent with the rest of the mock.

Note: Searches reveal UpdateCertificateStatus references remain in the service layer (./server/mock/service/service_mock.go, ./server/service/certificates.go, ./server/fleet/service.go). This is expected—the datastore layer uses UpsertCertificateStatus while the service layer retains UpdateCertificateStatus. No action needed.

Comment thread server/datastore/mysql/host_certificate_templates.go
@getvictor
getvictor merged commit a098a6c into main Dec 5, 2025
45 checks passed
@getvictor
getvictor deleted the 36533-android-certificate-crud-validate-variable-replacement branch December 5, 2025 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

android certificate crud: validate variable replacement

4 participants