Skip to content

Fix S3 carve cleanup never running and panic on empty carves (#43045) - #46462

Merged
juan-fdz-hawa merged 2 commits into
mainfrom
43045-s3-carve-cleanup-never-runs
Jun 1, 2026
Merged

Fix S3 carve cleanup never running and panic on empty carves (#43045)#46462
juan-fdz-hawa merged 2 commits into
mainfrom
43045-s3-carve-cleanup-never-runs

Conversation

@juan-fdz-hawa

@juan-fdz-hawa juan-fdz-hawa commented May 29, 2026

Copy link
Copy Markdown
Contributor

Related issue: Resolves #43045

Fixed a bug where the carve cleanup cron job called the MySQL implementation instead of the S3-aware implementation on S3-configured deployments, meaning expired carves were never marked as expired in S3. Also fixed a panic in S3 carve cleanup that occurred when there were no non-expired carves.

Checklist for submitter

If some of the following don't apply, delete the relevant line.

  • Changes file added for user-visible changes in changes/, orbit/changes/ or ee/fleetd-chrome/changes.
    See Changes files for more information.

Testing

  • Added/updated automated tests
  • QA'd all new/changed functionality manually

Summary by CodeRabbit

  • Bug Fixes

    • Cleanup now uses the S3-aware path on S3-configured deployments (no longer the MySQL path)
    • Prevented a panic when no non-expired carves are present during cleanup
    • Improved error aggregation so all cleanup failures are reported
    • Cleanup counters now increment only for successfully updated carves
  • Tests

    • Added tests covering S3 carve cleanup behavior, including empty and S3-absent cases

@codecov

codecov Bot commented May 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 44.44444% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.85%. Comparing base (4b04063) to head (fb5e4ba).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
server/datastore/s3/carves.go 57.14% 2 Missing and 1 partial ⚠️
cmd/fleet/cron.go 0.00% 1 Missing ⚠️
cmd/fleet/serve.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #46462      +/-   ##
==========================================
+ Coverage   66.82%   66.85%   +0.02%     
==========================================
  Files        2808     2808              
  Lines      223582   223603      +21     
  Branches    11347    11347              
==========================================
+ Hits       149413   149487      +74     
+ Misses      60615    60544      -71     
- Partials    13554    13572      +18     
Flag Coverage Δ
backend 68.58% <44.44%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@juan-fdz-hawa
juan-fdz-hawa force-pushed the 43045-s3-carve-cleanup-never-runs branch from 5c26d4a to af7dd98 Compare June 1, 2026 10:43
@juan-fdz-hawa
juan-fdz-hawa force-pushed the 43045-s3-carve-cleanup-never-runs branch from af7dd98 to 026ec8a Compare June 1, 2026 10:54
@juan-fdz-hawa

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor
✅ Actions performed

Full review triggered.

@coderabbitai

coderabbitai Bot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 1428cbc6-9d08-4201-a249-2e70fe859867

📥 Commits

Reviewing files that changed from the base of the PR and between b4309d3 and fb5e4ba.

📒 Files selected for processing (1)
  • server/datastore/s3/carves.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • server/datastore/s3/carves.go

Walkthrough

This PR fixes two bugs in S3 carve cleanup. It routes the cleanup cron job to invoke the S3-aware CarveStore.CleanupCarves instead of the MySQL datastore method, and fixes a panic that occurs when there are no non-expired carves in the database. The S3 implementation now short-circuits early on empty carve lists, uses errors.Join to aggregate update failures while continuing processing, and increments the cleanup counter only after successful updates. New test cases cover both the empty carves scenario and the full S3 cleanup path.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and concisely summarizes the main changes: fixing S3 carve cleanup not running and a panic on empty carves, directly matching the primary objectives.
Description check ✅ Passed The description includes a related issue reference, marks the changes file added, and indicates automated tests were added and manual QA performed. Required testing and changes file checkboxes are completed.
Linked Issues check ✅ Passed The PR implements both bug fixes from #43045: wiring carveStore to newCleanupsAndAggregationSchedule for S3-aware cleanup and adding an empty-slice guard in S3CarveStore.CleanupCarves to prevent panic.
Out of Scope Changes check ✅ Passed All changes directly address the two bugs specified in #43045: changes file for the fix, wiring changes to use S3-aware cleanup, guard against empty slice panic, and tests validating both fixes.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 43045-s3-carve-cleanup-never-runs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Fixed a bug where the carve cleanup cron job called the MySQL
implementation instead of the S3-aware implementation on S3-configured
deployments, meaning expired carves were never marked as expired in S3.
Also fixed a panic in S3 carve cleanup that occurred when there
were no non-expired carves.
@juan-fdz-hawa
juan-fdz-hawa force-pushed the 43045-s3-carve-cleanup-never-runs branch from 026ec8a to b4309d3 Compare June 1, 2026 11:12
@juan-fdz-hawa
juan-fdz-hawa marked this pull request as ready for review June 1, 2026 11:21
@juan-fdz-hawa
juan-fdz-hawa requested a review from a team as a code owner June 1, 2026 11:21

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

nulmete
nulmete previously approved these changes Jun 1, 2026
Comment thread server/datastore/s3/carves.go Outdated
carve.Expired = true
err = c.UpdateCarve(ctx, carve)
if uerr := c.UpdateCarve(ctx, carve); uerr != nil {
retErr = errors.Join(retErr, ctxerr.Wrap(ctx, uerr, "marking carve expired"))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: is it worth adding extra information to the error string? maybe the carve's ID?

@juan-fdz-hawa
juan-fdz-hawa merged commit 66667c3 into main Jun 1, 2026
41 checks passed
@juan-fdz-hawa
juan-fdz-hawa deleted the 43045-s3-carve-cleanup-never-runs branch June 1, 2026 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

S3 carve cleanup never runs and contains a panic on empty carves

2 participants