Fix duplicate software inventory entries from v4.76.0 checksum change - #50427
Open
juan-fdz-hawa wants to merge 6 commits into
Open
Fix duplicate software inventory entries from v4.76.0 checksum change#50427juan-fdz-hawa wants to merge 6 commits into
juan-fdz-hawa wants to merge 6 commits into
Conversation
juan-fdz-hawa
temporarily deployed
to
Docker Hub
August 3, 2026 12:34 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
Author
|
@coderabbitai full review |
Contributor
✅ Action performedFull review finished. |
Contributor
WalkthroughThe change adds 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #50427 +/- ##
========================================
Coverage 68.20% 68.21%
========================================
Files 3943 3944 +1
Lines 251273 251625 +352
Branches 13406 13406
========================================
+ Hits 171390 171645 +255
- Misses 64532 64597 +65
- Partials 15351 15383 +32
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
server/datastore/mysql/software_test.go (1)
14111-14121: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueAdd
t.Helper()so failures point at the calling test line.
legacyNameFirstChecksumcallsrequire.NoErrorinternally. Withoutt.Helper(), a failure reports this helper's line instead of the call site.♻️ Proposed change
func legacyNameFirstChecksum(t *testing.T, s fleet.Software) []byte { + t.Helper() h := md5.New() //nolint:gosec // matches the (non-security) software checksum hash🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@server/datastore/mysql/software_test.go` around lines 14111 - 14121, Add t.Helper() at the start of legacyNameFirstChecksum, before the require.NoError call, so test failures report the calling test location.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cmd/fleet/cron.go`:
- Around line 2334-2339: The run-once software checksum reconciliation must
remain eligible after a failed execution. Update the startup cron flow around
the job registered with schedule.WithRunOnce and ds.ReconcileSoftwareChecksums
so failed reconciliation is not recorded as completed, allowing Start to retry
it on the next restart; preserve one-time completion after success and add a
test covering failure followed by restart and successful retry.
- Around line 2325-2339: Update the schedule created for
CronSoftwareChecksumMigration around schedule.WithRunOnce and the job callback
so the scheduler stops processing future ticker events after the first
successful scheduled ReconcileSoftwareChecksums run, while preserving
Trigger-based manual reruns. Add coverage that advances two intervals and
verifies ReconcileSoftwareChecksums is called only once by the schedule.
In `@server/datastore/mysql/software.go`:
- Around line 3397-3413: Update the duplicate-group query in the reconciliation
loop to use ds.writer(ctx) instead of ds.reader(ctx) when calling
sqlx.SelectContext, while preserving the existing batch processing,
reconciliation, and termination logic.
---
Nitpick comments:
In `@server/datastore/mysql/software_test.go`:
- Around line 14111-14121: Add t.Helper() at the start of
legacyNameFirstChecksum, before the require.NoError call, so test failures
report the calling test location.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: b8ca0cb4-5337-4b3f-b595-4a408f7c49f5
📒 Files selected for processing (10)
changes/36365-duplicate-software-checksumcmd/fleet/cron.gocmd/fleet/cron_registration.goserver/datastore/mysql/migrations/tables/20231212094238_AddUniqueHashToSoftware.goserver/datastore/mysql/software.goserver/datastore/mysql/software_test.goserver/fleet/cron_schedules.goserver/fleet/datastore.goserver/fleet/software.goserver/mock/datastore_mock.go
…#36365) Resolves #36365 The v4.76.0 checksum change (#34097) reordered the fields hashed into `Software.ComputeRawChecksum` for non-`apps` sources, so software rows created before the upgrade no longer matched re-ingested rows and got duplicated (same name/version/source, split host counts). - Make `ComputeRawChecksum` the sole source of truth and delete the drifted parallel SQL checksum formula that caused the mismatch. - Add `ReconcileSoftwareChecksums`, a one-shot migration that merges existing duplicates onto the canonical row (batched host_software repointing) and logs each merge. Runs once after startup; re-run with `fleetctl trigger --name software_checksum_migration`
juan-fdz-hawa
force-pushed
the
36365-duplicate-software-checksum-dedup
branch
from
August 3, 2026 12:58
6b01776 to
b77a0c1
Compare
juan-fdz-hawa
temporarily deployed
to
Docker Hub
August 3, 2026 12:58 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
juan-fdz-hawa
temporarily deployed
to
Docker Hub
August 3, 2026 13:36 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
juan-fdz-hawa
temporarily deployed
to
Docker Hub
August 3, 2026 14:50 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
juan-fdz-hawa
temporarily deployed
to
Docker Hub
August 3, 2026 15:32 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
juan-fdz-hawa
marked this pull request as ready for review
August 3, 2026 15:33
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
server/datastore/mysql/software_test.go (1)
14212-14227: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAssert the repointed host links for group 0.
The test forces the repoint loop with 8 host links on the stale row, but it only asserts software row counts. Add an assertion that the surviving
pkg0row carries all 8 host links. This confirms the batched repoint moved every link instead of only the first batch.♻️ Proposed additional assertion
require.Equal(t, groupCount, countDeb()) + // group 0's 8 host links (> reconcileRepointBatch) all moved onto the survivor. + var pkg0Hosts int + require.NoError(t, sqlx.GetContext(ctx, ds.reader(ctx), &pkg0Hosts, + `SELECT COUNT(*) FROM host_software hs + JOIN software s ON s.id = hs.software_id + WHERE s.name = 'pkg0' AND s.source = 'deb_packages'`)) + require.Equal(t, len(hosts), pkg0Hosts) var maxPerName int🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@server/datastore/mysql/software_test.go` around lines 14212 - 14227, Extend the ReconcileSoftwareChecksums test after the existing count assertions to query the surviving software row for group 0 (pkg0) and assert it has all 8 associated host links. Use the test’s existing datastore/query helpers and verify the repoint result specifically for pkg0, confirming every link from the stale row was moved.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@server/datastore/mysql/software_test.go`:
- Around line 14212-14227: Extend the ReconcileSoftwareChecksums test after the
existing count assertions to query the surviving software row for group 0 (pkg0)
and assert it has all 8 associated host links. Use the test’s existing
datastore/query helpers and verify the repoint result specifically for pkg0,
confirming every link from the stale row was moved.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 1a210e05-877a-4d9e-99d4-e35befd168c3
📒 Files selected for processing (11)
changes/36365-duplicate-software-checksumcmd/fleet/cron.gocmd/fleet/cron_registration.goserver/datastore/mysql/migrations/tables/20231212094238_AddUniqueHashToSoftware.goserver/datastore/mysql/software.goserver/datastore/mysql/software_test.goserver/fleet/cron_schedules.goserver/fleet/datastore.goserver/fleet/software.goserver/mock/datastore_mock.goserver/service/integration_core_test.go
🚧 Files skipped from review as they are similar to previous changes (8)
- server/fleet/datastore.go
- cmd/fleet/cron_registration.go
- changes/36365-duplicate-software-checksum
- server/fleet/cron_schedules.go
- server/mock/datastore_mock.go
- server/datastore/mysql/migrations/tables/20231212094238_AddUniqueHashToSoftware.go
- server/fleet/software.go
- server/datastore/mysql/software.go
jkatz01
approved these changes
Aug 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #36365
The v4.76.0 checksum change (#34097) reordered the fields hashed into
Software.ComputeRawChecksumfor non-appssources, so software rows created before the upgrade no longer matched re-ingested rows and got duplicated (same name/version/source, split host counts).ComputeRawChecksumthe sole source of truth and delete the drifted parallel SQL checksum formula that caused the mismatch.ReconcileSoftwareChecksums, a one-shot migration that merges existing duplicates onto the canonical row (batched host_software repointing) and logs each merge. Runs once after startup; re-run withfleetctl trigger --name software_checksum_migrationChecklist for submitter
If some of the following don't apply, delete the relevant line.
Changes file added for user-visible changes in
changes/,orbit/changes/oree/fleetd-chrome/changes.See Changes files for more information.
Input data is properly validated,
SELECT *is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters.Testing
Added/updated automated tests
Where appropriate, automated tests simulate multiple hosts and test for host isolation (updates to one hosts's records do not affect another)
QA'd all new/changed functionality manually
Summary by CodeRabbit
Bug Fixes
Documentation
fleetctl trigger --name software_checksum_migration.