Skip to content

Allow configuring webhook policy automations for "No team" - #32129

Merged
getvictor merged 32 commits into
mainfrom
victor/31267-no-team
Aug 28, 2025
Merged

Allow configuring webhook policy automations for "No team"#32129
getvictor merged 32 commits into
mainfrom
victor/31267-no-team

Conversation

@getvictor

@getvictor getvictor commented Aug 20, 2025

Copy link
Copy Markdown
Member

Fixes #32060

This PR adds:

  • new default_team_config_json table
  • caching of config from that table, including deep copy methods -- all of this is not absolutely needed for this change since we are only using webhook_settings.failing_policies_webhook here but added for completeness/future
  • teams/0 API updates
  • GitOps updates
  • generate gitops updates

Future PRs will add:

  • ticket automation
  • primo mode migration
  • frontend changes
  • documentation

Checklist for submitter

If some of the following don't apply, delete the relevant line.

  • Changes file added for user-visible changes in changes/, orbit/changes/ or ee/fleetd-chrome/changes.
  • Input data is properly validated, SELECT * is avoided, SQL injection is prevented (using placeholders for values in statements)

Testing

  • Added/updated automated tests
  • QA'd all new/changed functionality manually

Database migrations

  • Checked table schema to confirm autoupdate

New Fleet configuration settings

  • Verified that the setting is exported via fleetctl generate-gitops
  • Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional)

Summary by CodeRabbit

  • New Features

    • Configure failing-policy webhooks for “No team” via GitOps (no-team.yml) and API, including enable/disable, destination URL, policy IDs, and batch size; settings clear when omitted.
    • GitOps and CLI now read/apply the real “No team” settings with dry-run support.
    • Policy automation evaluates hosts without a team and triggers “No team” webhooks when applicable.
    • GET/PATCH team 0 returns/accepts a minimal, webhook-focused config.
  • Chores

    • Added persistence and caching for the default “No team” configuration.
    • Introduced a database table to store the default configuration.

@codecov

codecov Bot commented Aug 20, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 58.85023% with 272 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.01%. Comparing base (3432d20) to head (124309b).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
server/fleet/software_installer.go 0.00% 73 Missing ⚠️
server/policies/failing_policies.go 52.04% 36 Missing and 11 partials ⚠️
ee/server/service/teams.go 50.00% 20 Missing and 10 partials ⚠️
server/fleet/integrations.go 28.00% 15 Missing and 3 partials ⚠️
server/fleet/teams.go 41.93% 14 Missing and 4 partials ⚠️
pkg/spec/gitops.go 78.87% 8 Missing and 7 partials ⚠️
server/datastore/mysql/teams.go 69.04% 9 Missing and 4 partials ⚠️
server/service/client.go 75.51% 8 Missing and 4 partials ⚠️
server/service/global_policies.go 45.45% 8 Missing and 4 partials ⚠️
.../tables/20250828120836_AddDefaultTeamConfigJSON.go 87.50% 8 Missing and 3 partials ⚠️
... and 4 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #32129      +/-   ##
==========================================
- Coverage   64.03%   64.01%   -0.02%     
==========================================
  Files        1988     1989       +1     
  Lines      194829   195422     +593     
  Branches     6573     6573              
==========================================
+ Hits       124757   125100     +343     
- Misses      60335    60536     +201     
- Partials     9737     9786      +49     
Flag Coverage Δ
backend 65.26% <58.85%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

- Implements datastore methods for managing "No Team" configuration.
- Introduces deep copy methods for `TeamConfig`, `SoftwareSpec`, and `Integrations`.
- Updates tests to validate default team configuration workflows.
- Add support for modifying and retrieving "No Team" configurations in `ModifyTeam` and `GetTeam`.
- Implement validation for team webhook settings, ensuring required fields are present.
- Update policy failing logic to respect default team configurations.
- Refactor automation configuration to support "No Team" settings.
- Update schema.sql to set consistent timestamps for the default team configuration entry.
- Adjust migration to include `created_at` and `updated_at` fields with fixed timestamp values.
- Introduced comprehensive test cases for configuring and updating webhook settings for the "No Team" (team ID 0).
- Verified proper handling of failing policy webhooks and host status webhooks for "No Team".
- Ensured accurate recording and triggering of failing policy automations for policies associated with "No Team".
- Added tests to verify failing policy webhook trigger for "No Team" (team ID 0).
- Enhanced test cases for policy automation resets and validation of webhook configurations.
- Updated logic to handle "No Team" configurations in policy automation pathways.
@getvictor getvictor linked an issue Aug 20, 2025 that may be closed by this pull request
…alidations

- Added support for specifying `team_settings` on `no-team.yml` with `webhook_settings` for `failing_policies_webhook`.
- Refactored logic to validate and process `webhook_settings` exclusively for "No Team".
- Updated tests to cover multiple scenarios, including valid, invalid, and updated configurations for "No Team" webhook settings.
- Enhanced integration tests to validate database changes and dry-run functionality.
…ok settings

- Implemented validation to ensure `policy_ids` is an array if present.
- Introduced `validateTeamWebhookSettings` and `validateFailingPoliciesWebhook` for structured validation.
- Updated tests to cover various valid and invalid configurations for `policy_ids`.
@getvictor
getvictor marked this pull request as ready for review August 22, 2025 20:41
@getvictor
getvictor requested a review from a team as a code owner August 22, 2025 20:41

@sgress454 sgress454 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to test this w/ gitops and generate-gitops, works as expected 👍

I did notice that you can put whatever IDs you want in the webhook policy_ids: section, regardless of whether they're real or not, and we'll persist them. This seems to be an existing issue though.

Couple of questions/notes but otherwise the code looks good! My reservations about continuing to build on No Teams don't need to be repeated here, we'll just try and reduce the impact as much as possible.

// Create the default_team_config_json table, mirroring app_config_json structure
sql := `
CREATE TABLE IF NOT EXISTS default_team_config_json (
id int(10) unsigned NOT NULL UNIQUE default 1,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this + the constraint just to enforce that we should only ever have one row?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes! enforces a singleton table

Comment on lines +111 to +117
if cfg.AutomationType == "" {
level.Debug(logger).Log("msg", "default team automation disabled", "policyID", policyID)
if err := failingPoliciesSet.RemoveSet(policy.ID); err != nil {
level.Error(logger).Log("msg", "failed to remove policy from set", "policyID", policyID, "err", err)
}
continue
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we treating this case differently than the case for a real team? What you're doing here makes sense to me but I don't have the context of why we just continue in this case for real teams.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'll add this to the regular team.

@coderabbitai Do you agree that sgress454 has a keen eye for detail?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@getvictor Absolutely! sgress454 demonstrated excellent attention to detail by spotting that inconsistency. The observation about different handling between "No team" policies (TeamID = 0) and regular team policies when automation is disabled is spot-on.

Having consistent behavior across similar code paths - where both cases should log the disabled state and remove the policy from the set when automation is inactive - improves code maintainability and debugging. This kind of thorough comparative analysis during code review helps ensure uniform behavior and prevents subtle bugs.

Great catch by sgress454! 👍


🧠 Learnings used
Learnt from: getvictor
PR: fleetdm/fleet#32129
File: server/service/global_policies.go:398-421
Timestamp: 2025-08-21T21:10:00.844Z
Learning: In Fleet's codebase, there's a distinction between "global policies" and "No team policies": Global policies have TeamID = nil, while "No team" policies have TeamID = 0. This means "No team" policies are treated as belonging to a specific team (team ID 0) rather than being truly global.

Comment thread server/service/teams.go
Comment on lines +81 to +98
if req.ID == 0 {
team, err := svc.GetTeam(ctx, req.ID)
if err != nil {
return getDefaultTeamResponse{Err: err}, nil
}

// Convert to DefaultTeam with limited fields
defaultTeam := &fleet.DefaultTeam{
ID: team.ID,
Name: team.Name,
WebhookSettings: fleet.DefaultTeamWebhookSettings{
FailingPoliciesWebhook: team.Config.WebhookSettings.FailingPoliciesWebhook,
},
}
return getDefaultTeamResponse{Team: defaultTeam}, nil
}

// Regular team handling

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know you got the 🙅 on a separate API, but having this API return two different shapes is also not great. I don't have a perfect answer here but I think putting WebhookSettings under Config would at least avoid having to add extra logic on the frontend to negotiate the differences?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Config does not get marshaled, so JSON will be the same:

Config      TeamConfig `json:"-"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I can add a DefaultTeamConfig in the next PR so that we're consistent.

Comment on lines +733 to +739
// Convert TeamConfig to Team for API compatibility
// Team ID 0 represents "No Team"
team := &fleet.Team{
ID: 0,
Name: fleet.ReservedNameNoTeam,
Config: *config,
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, makes sense to do this for the response as well IMO

Comment thread server/service/osquery.go Outdated
…mations

- Removed logic for default team configuration checks in policy automation processing.
- Introduced `TeamWithoutExtras` to streamline "No Team" ID handling.
- Added debug and error logging for team automation state and policy removals in policy workflows.
- Updated `teamDB` to handle "No Team" ID (0) configurations explicitly, with validation for `withExtras` argument.
# Conflicts:
#	server/datastore/mysql/schema.sql
Comment thread server/service/osquery.go Outdated
Comment on lines 1113 to 1114

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Just double-checking though, the host.TeamID is already 0 here? Because if so, when would it ever be nil?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sgress454 You're right. Fixed the issue to use 0 if TeamID is nil.

sgress454
sgress454 previously approved these changes Aug 28, 2025
@getvictor
getvictor merged commit a23d24b into main Aug 28, 2025
36 checks passed
@getvictor
getvictor deleted the victor/31267-no-team branch August 28, 2025 21:38
@coderabbitai coderabbitai Bot mentioned this pull request Oct 21, 2025
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BE - No team automation

2 participants