Skip to content

feat(flags): enforce filters validation one rule at a time - #95226

Merged
trunk-io[bot] merged 2 commits into
masterfrom
patricio/per-rule-filters-enforcement
Sep 4, 2026
Merged

trunk-io[bot] merged 2 commits into
masterfrom
patricio/per-rule-filters-enforcement

Conversation

@patricio-posthog

Copy link
Copy Markdown
Contributor

Problem

Filters validation is finished but cannot be turned on. The switch is one boolean for all of it, so enabling the rules that are ready would also start rejecting the rules that live traffic still violates every day.

Two cleanup migrations emptied the backlog for some rules, and those rules now see zero violations. They are ready to reject. Others are not.

Part of #50084.

Changes

  • The setting takes rule ids instead of true or false. A rule rejects only when it is listed.
  • A rule that is not listed keeps logging, even when another rule in the same request rejects. Turning one rule on cannot leak a second rule's error to a customer.
  • Unset behaves exactly like the old off. "*" behaves exactly like the old on.
- FEATURE_FLAG_FILTERS_ENFORCEMENT=true
+ FEATURE_FLAG_FILTERS_ENFORCED_RULES=cross_field.variant_rollout_sum_not_100

Rule ids are the ones already in the violation metrics and in the code of each error, so the setting reads the same as the dashboard.

Note

No behavior changes on deploy. The variable is unset in production, which rejects nothing new. The checks that predate this work still reject, as they do today.

Mechanical: four existing tests point at the new setting name, and comments that named the old switch now name the rollout.

How did you test this code?

New tests in TestFeatureFlagFiltersEnforcement cover the four states: a listed rule rejects, an unlisted rule is accepted, unset accepts, and "*" rejects. One more covers a request that violates both rules with one listed, and asserts the unlisted rule's message is absent from the response.

They catch a regression nothing else did: dropping the per-rule filter at either decision point would make every violating write fail as soon as one rule goes live. Reverting that filter locally fails both accept cases.

Not checked: any behavior in a real environment. The rollout starts with the variable unset, so the first real signal is a rule going in and its violation counter falling.

Automatic notifications

  • Publish to changelog?

Docs update

None.

Replaces the all-or-nothing enforcement switch with a set of rule ids to reject on.
A rule that is not listed keeps logging, including when another rule in the same
request rejects, so a rule can go live once its own violations reach zero.

Unset matches the old off and "*" matches the old on.
@patricio-posthog patricio-posthog self-assigned this Sep 4, 2026
@trunk-io

trunk-io Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

😎 Merged successfully - details.

@github-actions

github-actions Bot commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

✅ Playwright — all passed

All tests passed.

View test results →

@patricio-posthog
patricio-posthog marked this pull request as ready for review September 4, 2026 15:42
@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team September 4, 2026 15:43
@posthog-project-board-bot posthog-project-board-bot Bot moved this to In Review in Feature Flags Sep 4, 2026
@greptile-apps

greptile-apps Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor
Prompt To Fix All With AI
### Issue 1
posthog/settings/feature_flags.py:135-137
**Setting documentation missing**

This adds the `FEATURE_FLAG_FILTERS_ENFORCED_RULES` configuration option without updating the matching documentation under `docs/`. The repository requires configuration changes and their documentation to ship in the same PR, so this must be addressed before merging.

### Issue 2
posthog/settings/feature_flags.py:135-137
**Deployment configuration missing**

This introduces the `FEATURE_FLAG_FILTERS_ENFORCED_RULES` environment variable. The repository requires new environment variables to also be added to `posthog/charts` and supplied through `posthog/secrets`; without that deployment setup, this staged enforcement cannot be activated in production. This requirement must be satisfied before merging.

### Issue 3
products/feature_flags/backend/api/feature_flag.py:1681
**Structural filtering lacks coverage**

This branch independently filters structural errors before rejecting, but the new partial and mixed-enforcement tests intentionally use structurally valid cross-field violations. Add a case with two structural violations and only one configured rule, then assert that the response contains only that rule. Otherwise, a regression here could expose an unlisted rule without causing the current tests to fail.

### Issue 4
products/feature_flags/backend/api/test/test_feature_flag.py:8880
**Test signature lacks annotations**

The new `test_only_the_rules_in_the_setting_reject` method leaves its parameters and return value unannotated. This violates the repository directive to write Python function signatures as if mypy strict mode were enabled, so the annotations must be added before merging.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "feat(feature-flags): enforce filters val..." | Re-trigger Greptile

Comment thread posthog/settings/feature_flags.py Resolved
Comment thread posthog/settings/feature_flags.py Resolved
The structural tier decides separately from the cross-field one, and only the
latter was covered. Uses two variant-key violations, which the serde guard type
checks but does not reject, so they reach the structural tier.
@trunk-io

trunk-io Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

Static Badge   Static Badge   Static Badge

View Full Report ↗︎ ⋅ Docs

@github-project-automation github-project-automation Bot moved this from In Review to Approved in Feature Flags Sep 4, 2026
@trunk-io
trunk-io Bot merged commit ef52a7c into master Sep 4, 2026
278 checks passed
@github-project-automation github-project-automation Bot moved this from Approved to Done in Feature Flags Sep 4, 2026
@trunk-io
trunk-io Bot deleted the patricio/per-rule-filters-enforcement branch September 4, 2026 19:44
@deployment-status-posthog

deployment-status-posthog Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-09-04 20:14 UTC Run
prod-us ✅ Deployed 2026-09-04 20:28 UTC Run
prod-eu ✅ Deployed 2026-09-04 20:29 UTC Run

@haacked haacked changed the title feat(feature-flags): enforce filters validation one rule at a time feat(flags): enforce filters validation one rule at a time Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

2 participants