Skip to content

feat(cymbal): evaluate error tracking severity rules - #86268

Merged
trunk-io[bot] merged 3 commits into
masterfrom
hp/error-tracking-severity-rules-02-evaluator
Aug 24, 2026
Merged

trunk-io[bot] merged 3 commits into
masterfrom
hp/error-tracking-severity-rules-02-evaluator

Conversation

@hpouillot

@hpouillot hpouillot commented Aug 20, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Cymbal needs a bounded, failure-safe evaluator for the severity-rule contract before issue linking can use configured rules.

Changes

  • Loads enabled severity rules for the current team and evaluates them in configured order.
  • Uses the same HogVM issue and processed-property globals as assignment rules.
  • Returns the first matching typed severity and preserves no-match fallback for callers.
  • Disables invalid programs while allowing later rules to run; step exhaustion skips only the current event.
  • Adds dedicated bounded caching and static-label processing metrics.
  • Does not yet call the evaluator from issue linking.

PR stack

Merge in this order:

  1. feat(error-tracking): add severity rules API #86267
  2. feat(cymbal): evaluate error tracking severity rules #86268 — this PR
  3. feat(cymbal): apply severity rules on issue creation #86270
  4. feat(error-tracking): add severity rules settings #86271

Depends on: #86267
Followed by: #86270

This PR is based on hp/error-tracking-severity-rules-01-api; review only changes relative to that branch. After PR 1 is squash-merged, replay only this PR's unique commit onto updated master, force-push with lease, then retarget. Do not retarget blindly.

How did you test this code?

  • cargo fmt --check
  • cargo test -p cymbal --test severity_rules
  • cargo check -p cymbal

The tests cover team isolation, enabled-only loading, every supported severity, first-match ordering, fallback, invalid-program disabling, and step-budget skipping.

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Automatic notifications

  • Publish to changelog?

Docs update

None for this internal evaluator layer.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Implemented with pi coding agent. Repo guidance and the OpenSpec apply, Rust testing, and stacked PR skills shaped the implementation and split. A dedicated typed rule severity keeps the evaluator testable without widening Cymbal's internal inferred-severity type.

@github-actions

Copy link
Copy Markdown
Contributor

🤖 CI report

✅ Trunk lane — non-backend lane

This PR is assigned to the non-backend lane. It does not run backend Python tests and may merge in parallel with PRs in other lanes.

@hpouillot hpouillot added the skip-inkeep-docs Use this label to skip an Inkeep docs PR in posthog.com label Aug 20, 2026
@hpouillot
hpouillot force-pushed the hp/error-tracking-severity-rules-01-api branch from 3672947 to fd29f0a Compare August 20, 2026 10:22
@hpouillot
hpouillot force-pushed the hp/error-tracking-severity-rules-02-evaluator branch from 06cfaae to 6ac019e Compare August 20, 2026 10:22
@trunk-io

trunk-io Bot commented Aug 20, 2026 •

Copy link
Copy Markdown

Static Badge   Static Badge   Static Badge

View Full Report ↗︎ ⋅ Docs

@hpouillot
hpouillot force-pushed the hp/error-tracking-severity-rules-01-api branch from fd29f0a to b163bf9 Compare August 20, 2026 10:30
@hpouillot
hpouillot force-pushed the hp/error-tracking-severity-rules-02-evaluator branch from 6ac019e to 18c7e3a Compare August 20, 2026 10:30
@hpouillot
hpouillot force-pushed the hp/error-tracking-severity-rules-01-api branch from b163bf9 to 78263a9 Compare August 20, 2026 10:57
@hpouillot
hpouillot force-pushed the hp/error-tracking-severity-rules-02-evaluator branch from 18c7e3a to c381c72 Compare August 20, 2026 10:57
@hpouillot
hpouillot requested review from a team, ablaszkiewicz and cat-ph August 20, 2026 11:13
@hpouillot
hpouillot marked this pull request as ready for review August 20, 2026 11:13
@greptile-apps

greptile-apps Bot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor
Prompt To Fix All With AI
### Issue 1
rust/cymbal/src/modes/processing/rules/severity.rs:220-228
**Disabled rules remain cached**

After an invalid rule is disabled in Postgres, `get_severity_rules` continues returning the cached vector containing it. Each event for that team re-executes the invalid program and issues another database update until the default five-minute TTL expires, adding repeated writes and processing overhead.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "feat(cymbal): evaluate severity rules" | Re-trigger Greptile

@ablaszkiewicz ablaszkiewicz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approved to not block but I think greptiles comment should be addressed. Removed auto merge

@hpouillot
hpouillot force-pushed the hp/error-tracking-severity-rules-02-evaluator branch from c381c72 to 8b099dd Compare August 20, 2026 14:06
Base automatically changed from hp/error-tracking-severity-rules-01-api to master August 20, 2026 15:51
@trunk-io

trunk-io Bot commented Aug 20, 2026 •

Copy link
Copy Markdown

😎 Merged successfully - details.

@trunk-io
trunk-io Bot merged commit 2a19052 into master Aug 24, 2026
196 checks passed
@trunk-io
trunk-io Bot deleted the hp/error-tracking-severity-rules-02-evaluator branch August 24, 2026 09:30
@deployment-status-posthog

deployment-status-posthog Bot commented Aug 24, 2026 •

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-08-24 10:13 UTC Run
prod-us ✅ Deployed 2026-08-24 10:36 UTC Run
prod-eu ✅ Deployed 2026-08-24 10:33 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-inkeep-docs Use this label to skip an Inkeep docs PR in posthog.com

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants