Skip to content

Add macOS up-to-date policies driven by Apple GDMF + grace days - #50383

Draft
cacaosteve wants to merge 1 commit into
fleetdm:mainfrom
cacaosteve:issue-3934-macos-up-to-date
Draft

Add macOS up-to-date policies driven by Apple GDMF + grace days#50383
cacaosteve wants to merge 1 commit into
fleetdm:mainfrom
cacaosteve:issue-3934-macos-up-to-date

Conversation

@cacaosteve

@cacaosteve cacaosteve commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Related issue: Resolves #3934

Checklist for submitter

  • Changes file added for user-visible changes in changes/
  • Input data is properly validated; SQL uses placeholders
  • Timeouts / retries are bounded (GDMF HTTP client uses a 10s timeout; fetch honors context)

Testing

  • Added/updated automated tests (server/mdm/apple/gdmf, migration test, PolicySpec validation, MySQL upsert/unclaim tests, GitOps client managed-key batching)
  • Where appropriate, automated tests simulate the sync path
  • QA'd all new/changed functionality manually (see notes)

Summary

Fleet keeps macOS OS-currency policies current from Apple's GDMF software update catalog (https://gdmf.apple.com/v2/pmv), so admins don't hardcode version pins that go stale.

  • Up to date (fleet_managed_key: macos_os_up_to_date, grace_days = 0): latest ProductVersion per major track (current + previous major)
  • Acceptable (fleet_managed_key: macos_os_acceptable, grace_days = 30): previous point release allowed until the newest release is 30 days old
  • Hourly cron fetches GDMF, upserts apple_software_update_assets, and rewrites policy queries only where fleet_managed_key is set explicitly
  • Standard-query-library + GitOps YAML docs include fleet_managed_key (the up-to-date entry is no longer a user-editable template; Fleet owns the query text)
  • Overlaps with OS updates / AULD (shared GDMF cache); this PR is compliance reporting, not MDM enforcement

Out of scope: dogfood it-and-security/ and internal dogfood updater scripts.

Test plan

  • go test ./server/mdm/apple/gdmf/
  • go test ./server/fleet/ + ApplyPolicySpecs duplicate-key / modify-platform service tests
  • MySQL: UpdateFleetManagedPolicyQueries, unclaim, ReplaceAppleSoftwareUpdateAssets upsert/prune, unique team-key locking
  • go build ./cmd/fleet
  • Deployed and confirmed hourly GDMF refresh; automatic rewrite on changed GDMF data is covered by sync + MySQL tests
  • Grace scenario: host one point behind fails up-to-date and passes acceptable while release age < 30 days
  • Host on latest passes both

Notes for reviewers

  • Single commit on current main; design notes stay local/uncommitted.
  • Scope narrowed per maintainer feedback: no it-and-security/ or dogfood updater script changes.
  • Manual QA covered the grace scenario and hourly GDMF refresh on a dogfood deploy of this branch.

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds Fleet-maintained macOS currency policies from Apple’s GDMF catalog. The change stores update assets, computes version floors with a 30-day grace period, updates Fleet-managed policy queries, and refreshes them hourly. It adds Fleet-managed policy ownership validation, database uniqueness, GitOps reconciliation, and context-aware GDMF retry cancellation.

Possibly related PRs

  • fleetdm/fleet#50036: Adds the Apple software update assets schema used by the synchronization flow.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 28.99% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes satisfy issue #3934 by providing GDMF-driven current and previous major-version checks with configurable grace-period behavior.
Out of Scope Changes check ✅ Passed The supporting datastore, policy ownership, GitOps, migration, and synchronization changes directly support the stated macOS currency-policy objectives.
Title check ✅ Passed The title clearly describes the primary change: macOS currency policies driven by Apple GDMF with grace periods.
Description check ✅ Passed The description covers the issue, user-visible changes, validation, testing, manual QA, scope, and implementation details.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🧹 Nitpick comments (5)
server/mdm/apple/gdmf/sync_test.go (1)

54-69: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add error-path and empty-asset coverage.

The test covers only the happy path. Three uncovered branches carry real behavior:

  • getAssetMetadataFn returns an error, so no datastore call must happen.
  • UpdatePolicyQueriesByName returns an error, so the sync must stop and wrap the error.
  • The GDMF response contains no macOS assets, so SyncMacOSCurrencyPolicies must return nil without calling UpdatePolicyQueriesByName.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/mdm/apple/gdmf/sync_test.go` around lines 54 - 69, Extend the tests
for SyncMacOSCurrencyPolicies with cases where getAssetMetadataFn returns an
error and verify no datastore update occurs, UpdatePolicyQueriesByName returns
an error and the sync stops with a wrapped error, and the GDMF response has no
macOS assets and returns nil without updating policy queries.
server/datastore/mysql/apple_software_update_assets.go (1)

29-44: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value

Consider one batched multi-row INSERT instead of a per-asset round trip.

The loop issues one ExecContext per asset. The GDMF macOS list is small today, so the impact is limited. A single statement with repeated VALUES tuples still reduces round trips and keeps the transaction short.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/datastore/mysql/apple_software_update_assets.go` around lines 29 - 44,
Replace the per-asset ExecContext calls in the asset insertion loop with one
batched multi-row INSERT using repeated VALUES tuples and the existing asset
fields, while preserving the empty/invalid SupportedDevices fallback and
ctxerr.Wrap error handling.
server/mdm/apple/gdmf/sync.go (2)

70-75: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Reuse MacOSAssetsForCurrencyPolicies for the asset-set fallback.

Lines 72-75 repeat the AssetSets then PublicAssetSets fallback that MacOSAssetsForCurrencyPolicies already implements in server/mdm/apple/gdmf/macos_versions.go lines 154-162. Two copies of the same precedence rule can drift, and the persisted rows would then disagree with the rows used for the policy floors.

♻️ Proposed deduplication
-	// Prefer AssetSets (fuller history); fall back to PublicAssetSets.
-	src := meta.AssetSets.MacOS
-	if len(src) == 0 {
-		src = meta.PublicAssetSets.MacOS
-	}
+	// Prefer AssetSets (fuller history); fall back to PublicAssetSets.
+	src := MacOSAssetsForCurrencyPolicies(meta)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/mdm/apple/gdmf/sync.go` around lines 70 - 75, Update
replaceMacOSAssets to obtain its macOS asset source through
MacOSAssetsForCurrencyPolicies instead of duplicating the
AssetSets-then-PublicAssetSets fallback; preserve the existing precedence and
use the shared helper’s result when persisting asset rows.

90-102: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Rename the date helper and skip the marshal for the empty case.

Two small points:

  • parsePostingDate now parses ExpirationDate as well. Rename it to parseGDMFDate so the name matches both call sites.
  • json.Marshal runs before the emptiness check, so the "null" result it produces for a nil slice is computed and then discarded. Check the length first.
♻️ Proposed reordering
-		devices, err := json.Marshal(a.SupportedDevices)
-		if err != nil {
-			return ctxerr.Wrap(ctx, err, "marshal supported devices")
-		}
-		if len(a.SupportedDevices) == 0 {
-			devices = []byte("[]")
-		}
-		row.SupportedDevices = devices
+		devices := []byte("[]")
+		if len(a.SupportedDevices) > 0 {
+			devices, err = json.Marshal(a.SupportedDevices)
+			if err != nil {
+				return ctxerr.Wrap(ctx, err, "marshal supported devices")
+			}
+		}
+		row.SupportedDevices = devices
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/mdm/apple/gdmf/sync.go` around lines 90 - 102, Rename the date helper
parsePostingDate to parseGDMFDate and update both PostingDate and ExpirationDate
call sites. In the supported-devices handling, check len(a.SupportedDevices)
before calling json.Marshal, use [] for the empty case, and marshal only when
the slice is non-empty.
server/mdm/apple/gdmf/macos_versions_test.go (1)

25-35: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add a two-digit minor version case to the query assertions.

This assertion pins the current query text, so the string-comparison defect flagged on server/mdm/apple/gdmf/macos_versions.go lines 135-149 stays invisible. After the predicate becomes version-aware, add a case with a floor such as 26.10.0 and assert that a host on 26.10.0 satisfies the generated predicate while 26.9.0 does not.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/mdm/apple/gdmf/macos_versions_test.go` around lines 25 - 35, Add a
two-digit minor-version test case in the “grace 0 requires latest per major”
coverage around RequiredMacOSVersions and PolicyQuery, using a floor such as
26.10.0; assert the generated predicate accepts 26.10.0 but rejects 26.9.0, so
version-aware comparison behavior is covered beyond exact query-text assertions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/scripts/dogfood-policy-updater-latest-macos.sh:
- Around line 109-113: Scope each floor generated by query() to its own
major-version track by combining a version LIKE '{major}.%' guard with the
existing version comparison. Regenerate or manually update the queries in
.github/scripts/dogfood-policy-updater-latest-macos.sh#L109-L113,
it-and-security/lib/macos/policies/latest-macos.yml#L2-L4,
it-and-security/lib/macos/policies/acceptable-macos.yml#L1-L4, and both relevant
queries in
docs/01-Using-Fleet/standard-query-library/standard-query-library.yml#L1265-L1279;
also apply the same scoped logic in server/mdm/apple/gdmf/macos_versions.go.
- Around line 32-122: Replace the eval-based handoff around the Python generator
with non-executable data transfer: have the Python block emit structured JSON or
plain NAME=value records, then assign UP_TO_DATE_QUERY, ACCEPTABLE_QUERY,
LATEST_FLOORS, and ACCEPTABLE_FLOORS using jq or shell read operations. Preserve
the existing floors and query generation logic while ensuring network-derived
output is never re-parsed as shell code.
- Around line 163-186: Update replace_policy_query in the embedded Python
updater to terminate with a non-zero status when pattern.subn returns anything
other than exactly one match, instead of returning the original text after only
warning. Preserve the existing successful replacement behavior and ensure either
unmatched policy name causes the Python process, and therefore the workflow, to
fail.

In `@server/datastore/mysql/apple_software_update_assets.go`:
- Around line 14-47: Replace the delete-and-reinsert flow in
ReplaceAppleSoftwareUpdateAssets with an upsert keyed by (class,
product_version, build), preserving first_seen_at and changing updated_at only
when persisted asset values differ. After upserting, delete only rows for the
class whose product_version/build pairs are absent from assets; when assets is
empty, delete all rows for the class. Use the migration’s existing unique key
and maintain the current JSON normalization and transaction retry behavior.

In `@server/datastore/mysql/policies.go`:
- Around line 400-411: Update UpdatePolicyQueriesByName to include a platform
predicate restricting matches to macOS policies, while preserving the existing
normalized name and query-difference conditions. Ensure the corresponding query
arguments use the platform value Fleet stores for macOS so
SyncMacOSCurrencyPolicies cannot modify unrelated global or team policies with
the same name.

In `@server/mdm/apple/gdmf/macos_versions.go`:
- Around line 135-149: Update PolicyQuery to generate numeric component
comparisons for each VersionFloor instead of comparing os_version.version as
text. Build each track predicate with the major equality and minor/patch
boundary logic described, then combine the per-floor groups with OR while
preserving the empty-floor and terminating-semicolon behavior.

In `@server/mdm/apple/gdmf/sync.go`:
- Around line 25-31: Update the GDMF metadata-fetch indirection around
getAssetMetadataFn so it accepts context.Context, and pass ctx from the sync
flow into the underlying GetAssetMetadata request. Preserve the existing error
wrapping and replaceMacOSAssets behavior while ensuring the outbound request
uses the cron context for cancellation and deadlines.

---

Nitpick comments:
In `@server/datastore/mysql/apple_software_update_assets.go`:
- Around line 29-44: Replace the per-asset ExecContext calls in the asset
insertion loop with one batched multi-row INSERT using repeated VALUES tuples
and the existing asset fields, while preserving the empty/invalid
SupportedDevices fallback and ctxerr.Wrap error handling.

In `@server/mdm/apple/gdmf/macos_versions_test.go`:
- Around line 25-35: Add a two-digit minor-version test case in the “grace 0
requires latest per major” coverage around RequiredMacOSVersions and
PolicyQuery, using a floor such as 26.10.0; assert the generated predicate
accepts 26.10.0 but rejects 26.9.0, so version-aware comparison behavior is
covered beyond exact query-text assertions.

In `@server/mdm/apple/gdmf/sync_test.go`:
- Around line 54-69: Extend the tests for SyncMacOSCurrencyPolicies with cases
where getAssetMetadataFn returns an error and verify no datastore update occurs,
UpdatePolicyQueriesByName returns an error and the sync stops with a wrapped
error, and the GDMF response has no macOS assets and returns nil without
updating policy queries.

In `@server/mdm/apple/gdmf/sync.go`:
- Around line 70-75: Update replaceMacOSAssets to obtain its macOS asset source
through MacOSAssetsForCurrencyPolicies instead of duplicating the
AssetSets-then-PublicAssetSets fallback; preserve the existing precedence and
use the shared helper’s result when persisting asset rows.
- Around line 90-102: Rename the date helper parsePostingDate to parseGDMFDate
and update both PostingDate and ExpirationDate call sites. In the
supported-devices handling, check len(a.SupportedDevices) before calling
json.Marshal, use [] for the empty case, and marshal only when the slice is
non-empty.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0842257d-cfd6-4bf7-9811-71bddfbab2a1

📥 Commits

Reviewing files that changed from the base of the PR and between 1dafb29 and 8442ea4.

📒 Files selected for processing (18)
  • .github/scripts/dogfood-policy-updater-latest-macos.sh
  • changes/3934-macos-up-to-date-gdmf-policies
  • cmd/fleet/cron.go
  • cmd/fleet/cron_registration.go
  • docs/01-Using-Fleet/standard-query-library/standard-query-library.yml
  • it-and-security/fleets/workstations.yml
  • it-and-security/lib/macos/policies/acceptable-macos.yml
  • it-and-security/lib/macos/policies/latest-macos.yml
  • server/datastore/mysql/apple_software_update_assets.go
  • server/datastore/mysql/policies.go
  • server/fleet/apple_software_update_assets.go
  • server/fleet/cron_schedules.go
  • server/fleet/datastore.go
  • server/mdm/apple/gdmf/macos_versions.go
  • server/mdm/apple/gdmf/macos_versions_test.go
  • server/mdm/apple/gdmf/sync.go
  • server/mdm/apple/gdmf/sync_test.go
  • server/mock/datastore_mock.go

Comment thread .github/scripts/dogfood-policy-updater-latest-macos.sh Outdated
Comment thread .github/scripts/dogfood-policy-updater-latest-macos.sh Outdated
Comment thread .github/scripts/dogfood-policy-updater-latest-macos.sh Outdated
Comment thread server/datastore/mysql/apple_software_update_assets.go
Comment thread server/datastore/mysql/policies.go Outdated
Comment thread server/mdm/apple/gdmf/macos_versions.go
Comment thread server/mdm/apple/gdmf/sync.go Outdated
@codecov

codecov Bot commented Aug 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.59712% with 119 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.26%. Comparing base (85692f8) to head (1f40849).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
server/datastore/mysql/policies.go 69.09% 30 Missing and 21 partials ⚠️
server/mdm/apple/gdmf/sync.go 69.62% 12 Missing and 12 partials ⚠️
server/mdm/apple/gdmf/api.go 42.85% 9 Missing and 3 partials ⚠️
server/mdm/apple/gdmf/macos_versions.go 89.18% 7 Missing and 5 partials ⚠️
...er/datastore/mysql/apple_software_update_assets.go 76.74% 5 Missing and 5 partials ⚠️
...les/20260805011013_AddFleetManagedKeyToPolicies.go 78.94% 3 Missing and 1 partial ⚠️
cmd/fleet/cron.go 76.92% 2 Missing and 1 partial ⚠️
server/service/client.go 96.72% 1 Missing and 1 partial ⚠️
server/fleet/policies.go 96.77% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #50383      +/-   ##
==========================================
+ Coverage   68.24%   68.26%   +0.01%     
==========================================
  Files        3949     3953       +4     
  Lines      252226   252731     +505     
  Branches    13328    13328              
==========================================
+ Hits       172140   172524     +384     
- Misses      64671    64743      +72     
- Partials    15415    15464      +49     
Flag Coverage Δ
backend 69.53% <78.59%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (2)
server/fleet/datastore.go (1)

1451-1454: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Document the empty-set contract.

The MySQL implementation rejects an empty assets slice with an error (server/datastore/mysql/apple_software_update_assets.go lines 16-18). The interface comment does not state this. Add it so callers do not treat an empty feed as a valid "delete all" request.

📝 Proposed doc addition
 	// ReplaceAppleSoftwareUpdateAssets upserts the provided GDMF assets for the
 	// class (preserving first_seen_at on existing rows) and deletes rows absent
-	// from the new set. Used by the apple_software_update_assets cron.
+	// from the new set. It returns an error if assets is empty, so an empty
+	// upstream feed cannot erase the cache. Used by the
+	// apple_software_update_assets cron.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/fleet/datastore.go` around lines 1451 - 1454, Update the
ReplaceAppleSoftwareUpdateAssets interface comment to explicitly state that an
empty assets slice is rejected and does not delete all existing rows, matching
the MySQL implementation’s contract.
server/fleet/policies.go (1)

793-804: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Reference name constants instead of repeating the literals.

server/mdm/apple/gdmf/macos_versions.go declares PolicyNameUpToDate, DogfoodPolicyNameUpToDate, PolicyNameAcceptable, and DogfoodPolicyNameAcceptable with these same four strings. The literals now exist in two packages, so a rename in one place silently breaks the mapping in the other. Declare the four names in this file next to the key constants and let the gdmf package alias them, as it already does for the keys.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/fleet/policies.go` around lines 793 - 804, Define shared constants for
the four macOS policy names alongside the Fleet-managed key constants, then
update FleetManagedKeyForPolicyName to switch on those constants instead of
repeating string literals. Expose these constants for the gdmf package to alias,
replacing its duplicate name declarations while preserving the existing key
alias pattern and mappings.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@server/datastore/mysql/migrations/tables/20260801062925_AddFleetManagedKeyToPolicies.go`:
- Around line 34-60: The migration’s macOS backfills can assign the same
fleet-managed key to multiple policies in one team scope. In
20260801062925_AddFleetManagedKeyToPolicies.go, restrict each UPDATE to the
single lowest-id matching policy per COALESCE(team_id, 0) scope; in
20260801062925_AddFleetManagedKeyToPolicies_test.go, add both supported policy
names in one Darwin scope before applyNext and assert migration success with
exactly one row receiving each key.

In `@server/datastore/mysql/policies.go`:
- Around line 430-437: Move the cleanupPolicy calls out of the withRetryTxx
transaction that processes the GDMF query update. Commit the transaction after
resetPolicyAutomationAttempts succeeds for all IDs, then iterate over the
returned IDs and invoke cleanupPolicy with the same full-membership cleanup
arguments outside the transaction, preserving error wrapping and relying on
needs_full_membership_cleanup for interrupted cleanup recovery.

In `@server/fleet/policies.go`:
- Around line 667-672: Validate FleetManagedKey at the ApplyPolicySpecs
policy-spec validation boundary before persistence, accepting only known
Fleet-managed keys and rejecting or ignoring user-supplied unknown values;
ensure arbitrary policy types cannot receive Fleet-managed keys and
duplicate-key database errors are prevented. Anchor the change around
FleetManagedKey handling in ApplyPolicySpecs and preserve Fleet’s existing
policy-name-based derivation behavior.

---

Nitpick comments:
In `@server/fleet/datastore.go`:
- Around line 1451-1454: Update the ReplaceAppleSoftwareUpdateAssets interface
comment to explicitly state that an empty assets slice is rejected and does not
delete all existing rows, matching the MySQL implementation’s contract.

In `@server/fleet/policies.go`:
- Around line 793-804: Define shared constants for the four macOS policy names
alongside the Fleet-managed key constants, then update
FleetManagedKeyForPolicyName to switch on those constants instead of repeating
string literals. Expose these constants for the gdmf package to alias, replacing
its duplicate name declarations while preserving the existing key alias pattern
and mappings.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 63bc38c9-f0ef-4f88-aa34-da5387352c88

📥 Commits

Reviewing files that changed from the base of the PR and between 0dcd04b and 9a6e811.

📒 Files selected for processing (16)
  • .github/scripts/dogfood-policy-updater-latest-macos.sh
  • docs/01-Using-Fleet/standard-query-library/standard-query-library.yml
  • it-and-security/lib/macos/policies/acceptable-macos.yml
  • it-and-security/lib/macos/policies/latest-macos.yml
  • server/datastore/mysql/apple_software_update_assets.go
  • server/datastore/mysql/migrations/tables/20260801062925_AddFleetManagedKeyToPolicies.go
  • server/datastore/mysql/migrations/tables/20260801062925_AddFleetManagedKeyToPolicies_test.go
  • server/datastore/mysql/policies.go
  • server/fleet/apple_software_update_assets.go
  • server/fleet/datastore.go
  • server/fleet/policies.go
  • server/mdm/apple/gdmf/macos_versions.go
  • server/mdm/apple/gdmf/macos_versions_test.go
  • server/mdm/apple/gdmf/sync.go
  • server/mdm/apple/gdmf/sync_test.go
  • server/mock/datastore_mock.go
🚧 Files skipped from review as they are similar to previous changes (7)
  • server/fleet/apple_software_update_assets.go
  • server/mdm/apple/gdmf/sync.go
  • server/mock/datastore_mock.go
  • server/mdm/apple/gdmf/macos_versions.go
  • docs/01-Using-Fleet/standard-query-library/standard-query-library.yml
  • it-and-security/lib/macos/policies/latest-macos.yml
  • it-and-security/lib/macos/policies/acceptable-macos.yml

Comment thread server/datastore/mysql/policies.go
Comment thread server/fleet/policies.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
server/datastore/mysql/policies.go (1)

1793-1794: 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Make ownership updates presence-aware.

Line 1823 converts both an omitted field and an explicit empty value to nil. Line 1794 then preserves the existing database value for nil. An existing Fleet-managed policy therefore cannot become user-owned through ApplyPolicySpecs, although PolicySpec documents empty as user-owned. Use a presence-aware input and write NULL for an explicit clear while preserving the value only when the field is omitted.

Also applies to: 1820-1826

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/datastore/mysql/policies.go` around lines 1793 - 1794, The policy
ownership update in ApplyPolicySpecs must distinguish an omitted
fleet-managed-key field from an explicitly empty value. Preserve the existing
database value only when the field is absent, but write NULL when the caller
explicitly clears it; update the input/presence handling around the policy spec
conversion and the fleet_managed_key SQL assignment (including the corresponding
logic near the referenced later lines).
♻️ Duplicate comments (2)
server/fleet/policies.go (1)

668-672: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Validate FleetManagedKey before persistence.

PolicySpec.FleetManagedKey accepts any non-empty string, and ApplyPolicySpecs writes it to policies.fleet_managed_key. A caller can mark an unrelated policy as a macOS Fleet-managed policy, so the GDMF refresh can rewrite its query. A duplicate scoped key instead fails at the database constraint. Validate the key against the allowlist and enforce the compatible policy scope before insertion.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/fleet/policies.go` around lines 668 - 672, Validate
PolicySpec.FleetManagedKey in ApplyPolicySpecs before inserting or updating
policies: accept only supported Fleet-managed keys and require each key’s
compatible policy scope, rejecting unknown keys or mismatched scopes before
persistence. Preserve the existing duplicate-key database constraint behavior
and leave empty FleetManagedKey values treated as user-owned.
server/datastore/mysql/policies.go (1)

400-445: 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

Commit policy changes before deleting memberships.

UpdateFleetManagedPolicyQueries performs full membership and statistics cleanup in the retryable transaction. cleanupPolicy with full cleanup enabled deletes all memberships and can hold those locks until commit. This can block host result writes during a large refresh. Commit the query update first, set needs_full_membership_cleanup, then run cleanup after commit and clear the flag only after successful cleanup, as ApplyPolicySpecs does at Lines 1964-2009.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/datastore/mysql/policies.go` around lines 400 - 445, Refactor
UpdateFleetManagedPolicyQueries to commit the policy query/checksum update and
mark each changed policy with needs_full_membership_cleanup before deleting
memberships. Run cleanupPolicy after the transaction commits, and clear the flag
only after each cleanup succeeds, following the ApplyPolicySpecs pattern;
preserve retry behavior and return the updated policy IDs while ensuring failed
cleanup leaves the flag set for retry.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@server/datastore/mysql/policies.go`:
- Around line 1793-1794: The policy ownership update in ApplyPolicySpecs must
distinguish an omitted fleet-managed-key field from an explicitly empty value.
Preserve the existing database value only when the field is absent, but write
NULL when the caller explicitly clears it; update the input/presence handling
around the policy spec conversion and the fleet_managed_key SQL assignment
(including the corresponding logic near the referenced later lines).

---

Duplicate comments:
In `@server/datastore/mysql/policies.go`:
- Around line 400-445: Refactor UpdateFleetManagedPolicyQueries to commit the
policy query/checksum update and mark each changed policy with
needs_full_membership_cleanup before deleting memberships. Run cleanupPolicy
after the transaction commits, and clear the flag only after each cleanup
succeeds, following the ApplyPolicySpecs pattern; preserve retry behavior and
return the updated policy IDs while ensuring failed cleanup leaves the flag set
for retry.

In `@server/fleet/policies.go`:
- Around line 668-672: Validate PolicySpec.FleetManagedKey in ApplyPolicySpecs
before inserting or updating policies: accept only supported Fleet-managed keys
and require each key’s compatible policy scope, rejecting unknown keys or
mismatched scopes before persistence. Preserve the existing duplicate-key
database constraint behavior and leave empty FleetManagedKey values treated as
user-owned.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: a5af2b66-62c7-488d-842d-dd1f00f084b8

📥 Commits

Reviewing files that changed from the base of the PR and between 9a6e811 and 37ed986.

📒 Files selected for processing (7)
  • server/datastore/mysql/migrations/tables/20260801062925_AddFleetManagedKeyToPolicies.go
  • server/datastore/mysql/migrations/tables/20260801062925_AddFleetManagedKeyToPolicies_test.go
  • server/datastore/mysql/policies.go
  • server/datastore/mysql/schema.sql
  • server/fleet/policies.go
  • server/mdm/apple/gdmf/macos_versions.go
  • server/mdm/apple/gdmf/macos_versions_test.go
💤 Files with no reviewable changes (1)
  • server/mdm/apple/gdmf/macos_versions_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • server/datastore/mysql/migrations/tables/20260801062925_AddFleetManagedKeyToPolicies_test.go
  • server/datastore/mysql/migrations/tables/20260801062925_AddFleetManagedKeyToPolicies.go

@allenhouchins

Copy link
Copy Markdown
Member

@cacaosteve Thanks for the contribution. I am moving this to draft. There are a number of code review comments that need to be addressed. This PR is also modifying a number of files that it shouldn't and are unrelated to the feature request. For example, editing scripts that we use for managing our internal instance of Fleet.

Please move this back to ready for review after the code review comments have been addressed, the scope of code changes has been addressed, and manual QA has been completed. Thanks!

@allenhouchins
allenhouchins marked this pull request as draft August 1, 2026 19:35
@cacaosteve

Copy link
Copy Markdown
Contributor Author

@allenhouchins Thanks for the feedback — keeping this in draft.

Addressed so far:

  1. Scope — reverted all it-and-security/ and dogfood updater script changes so this PR is limited to the product feature (server sync/cron, fleet_managed_key, standard-query-library, schema/migration).
  2. Review comments — ownership by explicit fleet_managed_key only; empty-feed cache preserve; upsert (keeps first_seen_at); deferred membership cleanup via needs_full_membership_cleanup; key allowlist + darwin/dynamic validation; clearable ownership on apply; GDMF fetch now takes context (GetAssetMetadataWithContext).

Still open before ready-for-review:

  • Finish documenting/completing manual QA on a real server
  • Any remaining CODEOWNER comments after the narrowed diff

Will move out of draft only after those are done.

@cacaosteve

Copy link
Copy Markdown
Contributor Author

Follow-up on latest review items (still draft):

  1. P1 raceUpdateFleetManagedPolicyQueries now SELECT … FOR UPDATE, updates with AND fleet_managed_key = ?, and only cleans policies that still own the key after the update.
  2. P2 duplicate keys — batch validation rejects duplicate fleet_managed_key in the same fleet; DB collisions on idx_policies_fleet_managed_team_key return a clear conflict error.
  3. MySQL tests — coverage for managed-query update/unclaim and asset upsert/prune/first_seen_at.
  4. Docsfleet_managed_key documented in docs/Configuration/yaml-files.md; GitOps schema regenerated.

Still before ready-for-review: redeploy current HEAD and record manual QA (including the grace-period pass/fail scenario).

@cacaosteve
cacaosteve force-pushed the issue-3934-macos-up-to-date branch 2 times, most recently from 520c949 to 54e6865 Compare August 3, 2026 06:38
@cacaosteve

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
server/mdm/apple/gdmf/api_test.go (1)

302-309: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Loosen the wall-clock bound to avoid CI flakes.

retryBackoff is 1 second, so any bound below that proves cancellation interrupted the wait. A 250 ms bound on a loaded CI runner can fail even when the code is correct. Raise the bound while keeping it well under the backoff interval.

♻️ Proposed adjustment
 	select {
 	case err := <-result:
 		require.ErrorIs(t, err, context.Canceled)
-		require.Less(t, time.Since(canceledAt), 250*time.Millisecond)
+		require.Less(t, time.Since(canceledAt), 750*time.Millisecond)
 		require.Equal(t, 1, attempts)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/mdm/apple/gdmf/api_test.go` around lines 302 - 309, Increase the
time.Since(canceledAt) threshold in the cancellation test while keeping it below
the 1-second retryBackoff interval, so the test still verifies prompt
interruption without relying on an overly tight CI-sensitive bound.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@server/mdm/apple/gdmf/api_test.go`:
- Around line 302-309: Increase the time.Since(canceledAt) threshold in the
cancellation test while keeping it below the 1-second retryBackoff interval, so
the test still verifies prompt interruption without relying on an overly tight
CI-sensitive bound.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: dff5fc06-533c-4f6c-8949-8e5e9cc0042a

📥 Commits

Reviewing files that changed from the base of the PR and between 37ed986 and 8ce869d.

⛔ Files ignored due to path filters (1)
  • docs/Configuration/yaml-files.md is excluded by !**/*.md
📒 Files selected for processing (32)
  • changes/3934-macos-up-to-date-gdmf-policies
  • cmd/fleet/cron.go
  • cmd/fleet/cron_registration.go
  • cmd/fleetctl/fleetctl/generate_gitops.go
  • cmd/fleetctl/fleetctl/generate_gitops_test.go
  • docs/01-Using-Fleet/standard-query-library/standard-query-library.yml
  • server/datastore/mysql/apple_software_update_assets.go
  • server/datastore/mysql/apple_software_update_assets_test.go
  • server/datastore/mysql/migrations/tables/20260801062925_AddFleetManagedKeyToPolicies.go
  • server/datastore/mysql/migrations/tables/20260801062925_AddFleetManagedKeyToPolicies_test.go
  • server/datastore/mysql/policies.go
  • server/datastore/mysql/policies_test.go
  • server/datastore/mysql/schema.sql
  • server/fleet/apple_software_update_assets.go
  • server/fleet/cron_schedules.go
  • server/fleet/datastore.go
  • server/fleet/policies.go
  • server/fleet/policies_test.go
  • server/mdm/apple/gdmf/api.go
  • server/mdm/apple/gdmf/api_test.go
  • server/mdm/apple/gdmf/macos_versions.go
  • server/mdm/apple/gdmf/macos_versions_test.go
  • server/mdm/apple/gdmf/sync.go
  • server/mdm/apple/gdmf/sync_test.go
  • server/mock/datastore_mock.go
  • server/service/client.go
  • server/service/client_test.go
  • server/service/global_policies.go
  • server/service/global_policies_test.go
  • server/service/team_policies.go
  • server/service/team_policies_test.go
  • tools/gitops-auto-complete/generated-schema.json
🚧 Files skipped from review as they are similar to previous changes (12)
  • server/fleet/cron_schedules.go
  • changes/3934-macos-up-to-date-gdmf-policies
  • server/fleet/apple_software_update_assets.go
  • cmd/fleet/cron_registration.go
  • cmd/fleet/cron.go
  • docs/01-Using-Fleet/standard-query-library/standard-query-library.yml
  • server/datastore/mysql/migrations/tables/20260801062925_AddFleetManagedKeyToPolicies.go
  • server/datastore/mysql/schema.sql
  • server/mdm/apple/gdmf/macos_versions_test.go
  • server/mock/datastore_mock.go
  • server/mdm/apple/gdmf/sync.go
  • server/mdm/apple/gdmf/macos_versions.go

@cacaosteve
cacaosteve force-pushed the issue-3934-macos-up-to-date branch from 35891b1 to 641b77f Compare August 3, 2026 07:17
@cacaosteve

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@cacaosteve
cacaosteve force-pushed the issue-3934-macos-up-to-date branch from 641b77f to 4e556f3 Compare August 3, 2026 07:26
@cacaosteve
cacaosteve force-pushed the issue-3934-macos-up-to-date branch 2 times, most recently from 2ed5ce3 to ea5e004 Compare August 4, 2026 18:08
@cacaosteve
cacaosteve marked this pull request as ready for review August 4, 2026 18:20
@cacaosteve
cacaosteve force-pushed the issue-3934-macos-up-to-date branch from ea5e004 to 4eae1f2 Compare August 4, 2026 18:35
@rachaelshaw rachaelshaw assigned melpike and georgekarrv and unassigned melpike Aug 4, 2026
@cacaosteve
cacaosteve force-pushed the issue-3934-macos-up-to-date branch from 4eae1f2 to ac82cf8 Compare August 5, 2026 01:10
@allenhouchins
allenhouchins removed their request for review August 5, 2026 03:44
Introduce fleet_managed_key policies synced from Apple GDMF, with GitOps ownership, unique team-key locking, and nilaway-safe apply paths.
@MagnusHJensen

Copy link
Copy Markdown
Member

Hi @cacaosteve, we've recently introduced a lot of changes around the GDMF and storing available versions plus software update device ID's in relation to keeping Apple hosts on latests for OS updates as part of #39085

Once that is fully in, I would expect this PR to change drastically. If you still want to keep the community PR, you can re-work it to fit the new approach and re-open it so we can get it prioritised and linked.

Thank you for the contribution though, this is great work and would make a lot of IT admins happy.

@MagnusHJensen
MagnusHJensen marked this pull request as draft August 5, 2026 12:56
@cacaosteve

Copy link
Copy Markdown
Contributor Author

@MagnusHJensen Thanks — that makes sense, and I'd like to keep the community PR.

#3934 is the compliance-reporting side (Fleet-managed policies that stay current without hardcoded version pins), not MDM enforcement. Happy to rework this as a consumer of the shared GDMF / apple_software_update_assets cache from #39085 once that lands, rather than shipping a parallel fetch/upsert/cron.

Plan after the AULD backend is in:

  • Drop this PR's GDMF sync + ReplaceAppleSoftwareUpdateAssets cron
  • Derive up-to-date / acceptable floors from the shared asset table (posting_date / first_seen_at + grace days)
  • Keep fleet_managed_key + query rewrite + standard-query-library entries

I'll leave this in draft and re-open when the rework is ready. Please link #3934 when you prioritize.

@allenhouchins

Copy link
Copy Markdown
Member

Fleet-managed policies that stay current without hardcoded version pins

Just a heads up, we're already providing customers with an example of that here (no product changes required): https://github.com/fleetdm/fleet/blob/main/.github/scripts/dogfood-policy-updater-latest-macos.sh

@cacaosteve

Copy link
Copy Markdown
Contributor Author

@allenhouchins Thanks — yes, that dogfood updater is a solid GitOps pattern (SOFA → rewrite the version pins → PR).

It's a different shape than #3934 though:

  • Customers still maintain hardcoded pins plus their own Actions / token / PR merge loop
  • The query is still version >= 'x' OR version >= 'y' (today 26.4.1 / 15.7.5)
  • No grace / “acceptable until N days after release” without more scripting
  • Uses SOFA, while AULD is standardizing on GDMF + apple_software_update_assets

Happy to treat the script as the supported workaround for now and keep this PR parked in draft until you all decide whether in-product fleet_managed_key policies are still wanted after AULD lands. If you'd rather close #3934 / this PR as solved by the example, just say so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ability to create a policy to track that macOS is "up to date" rather than specifying a version

5 participants