Audit Sentry error volume and filter noise to stop paying overages

[melvin-bot]

We're paying Sentry overage charges. We need to go through our error volume and cut what we don't act on.

Problem

Sentry event volume has grown past our plan's quota and we're being billed for overages. A large share of that volume is likely noise — errors nobody triages, network failures outside our control, third-party/extension noise, and a small number of high-frequency issues that dominate the event count.

Scope

1. Pull the current usage breakdown from Sentry: events by project, platform, and issue over the last 60–90 days. Identify which issues account for the bulk of the quota.
2. For the top issues by volume, decide one of: fix it, filter it, or sample it.
3. Add inbound filters and client-side filtering for events we never act on. Candidates worth checking:
   • Network/offline errors (Network request failed, aborted fetches, timeouts)
   • Browser extension and third-party script errors
   • Unsupported/legacy browser noise
   • Known noisy libraries and expected user-facing errors already handled in product
4. Tune beforeSend / ignoreErrors / denyUrls in the Sentry client config, and review sample rates (tracesSampleRate, profilesSampleRate, replaysSessionSampleRate) — performance and replay events count against billing too.
5. Set spend caps and per-key rate limits so a single regression can't blow the quota again.
6. Document what we filtered and why, so we don't silently drop something we later need.

Success criteria

• Monthly Sentry event volume drops back under quota with no overage charges.
• We can name every category of filtered event and justify it.
• Remaining events are ones we actually triage.

Related: #91302 (Top Sentry crashes monthly check-in), #76128 (Sentry INP event analysis)

Reported by @rlinoz via DM with MelvinBot.

[rlinoz]

@MelvinBot inspect errors in Sentry and list the top 5 ones that should be logs instead of errors or could be dropped

[MelvinBot]

⚠️ The Sentry MCP server failed to connect on this run, so I could not pull live event counts. Everything below is ranked from code evidence in Expensify/App plus the noise categories your own audit in #91302 already declares non-actionable. Treat the ranking as a shortlist to verify against Discover, not a measured top 5.

Top 5 candidates to downgrade or drop

1. Known network/offline failures — drop at ingest.
ignoreErrors contains exactly one entry, UPDATE_REQUIRED (setupSentry.ts:61). But CONST.ERROR already enumerates ~12 benign network strings (CONST/index.ts:2576-2595) that the network middleware deliberately files as Log.hmmm/Log.info (Middleware/Logging.ts:110-150). The app already says these aren't errors; Sentry's global handlers still bill them as error events. Adding that list to ignoreErrors is the single highest-leverage change.

2. [BootSplash] splash screen is still visible — make it a log.
Fires Sentry.captureMessage on a 10-second interval with no cap for as long as the splash is up (bootsplashTelemetry.ts:46-51). One stuck boot emits unbounded events. It's already level: 'warning' — nobody triages it as a crash. Use Sentry.logger.warn, or capture once and attach a duration.

3. Expense API Error: <command> returned <jsonCode> — classify before sending.
Every non-200 jsonCode on RequestMoney/TrackExpense/OpenReport goes out at level: 'error' with no allow/deny list on the code (trackExpenseCreationError.ts:100-137), so expected validation and permission responses land in the error stream. trackMFAFlowOutcome already does this correctly — routine and alternative outcomes go to Log.info, only anomalous ones become events (trackMFAFlowOutcome.ts:61). Copy that shape here.

4. Expected-by-design exceptions in auth and Pusher — make them logs.
Two captureException calls whose own comments say the condition is normal:

• errorType: 'network_retry' in reauthentication — the comment states this is a spotty connection and the user is deliberately not logged out (Reauthentication.ts:263-274).
• [Pusher] instance not found — the comment says this happens when disconnect() runs during Upgrade-Required teardown (Pusher/index.ts:284).

5. Third-party and environmental noise — drop, don't tag.
denyUrls only covers extension schemes (setupSentry.ts:26), and https-served third-party scripts are merely tagged third_party_code, not dropped (integrations/index.web.ts:42-47). Lazy chunk-load failures are still captured as warning-level exceptions even though the comment calls them "silent and largely environmental" (LazyModalSlot.tsx:17-22). #91302 already excludes FullStory fs.js, LinkedIn ads, MetaMask, gib.js, cross-origin Script error., and transient IDB/CacheStorage errors from triage — if they're excluded from triage, they should be excluded from billing.

Two structural findings worth acting on alongside these

There is no beforeSend for error events. Sentry.init wires up beforeSendTransaction and beforeSendLog but nothing for errors (setupSentry.ts:63-65). Items 1 and 5 need that hook (or ignoreErrors/denyUrls additions) to land anywhere. The existing classCallCheckNoiseFilterIntegration shows the working pattern — a processEvent returning null (classCallCheckNoiseFilter.ts:66-69).

tracesSampleRate: 1.0 (setupSentry.ts:54) sends 100% of transactions. Profiling is already sampled at 0.1. On most Sentry bills transactions outweigh errors, so if the goal is stopping overages this is likely a bigger lever than any single error signature — worth checking before spending effort on the five above.

Whatever you drop, drop it loudly. Every item above except #2 and #3 removes events with no trace. Per the success criteria in the issue, pair each filter with a Discover query that proves it's still matching, the way #93837 tracked third_party_code:True volume per release.

---

Next Steps: Reply with @MelvinBot <your feedback> to refine this, or re-run once the Sentry MCP connection is fixed and I'll rank these against real event counts.

---

view run

[rlinoz]

@MelvinBot implement 2 and 4

[MelvinBot]

🤖 Draft PR created: #99789
Please mark it as "Ready for review" when it is ready for review.

Items 2 and 4 are implemented. All three call sites now send their signal to Sentry logs instead of the error stream — nothing is dropped, it just stops billing against the error quota (enableLogs: true is already set in setupSentry).

#	Change	Before	After
2	bootsplashTelemetry.ts	captureMessage every 10s, uncapped	Sentry.logger.warn carrying stuckForMs, capped at 6 reports
4a	trackAuthenticationError.ts	every error type → captureException	network_retry → log with its tags preserved; all other types unchanged
4b	Pusher/index.ts + index.native.ts	captureException on teardown race	Sentry.logger.warn; __DEV__ still rejects loudly

The 6-report cap on the bootsplash monitor is load-bearing, not decoration: the interval only clears when the splash state moves off VISIBLE, so a boot that never completes would otherwise report for the entire session. Six covers the first minute.

Design notes and test results

Why network_retry and not the whole tracker. missing_params, auth_failure, and unexpected_error still capture exceptions. Only network_retry moved, because that is the UNABLE_TO_RETRY path whose comment states the user is deliberately not logged out and the retry mechanism handles it. Its tags survive as log attributes, so it stays searchable by authentication_function and authentication_json_code. This is the same shape trackMFAFlowOutcome already uses.

Tests run

Check	Result
npm run typecheck	passed
npm run lint-changed	passed
npx eslint <changed files>	passed — only pre-existing seatbelt warnings on untouched Pusher lines
npm run spell-changed	passed, 0 issues
New unit tests	8/8 passed
Existing suites over touched code (3 Pusher, ReauthenticationMiddlewareTest, forwardLogsToSentryTest, setupSentryIntegrationOrderTest)	39/39 passed

Two new test files: tests/unit/bootsplashTelemetryTest.ts and tests/unit/trackAuthenticationErrorTest.ts.

Not run, and why: the full npm test suite (only the suites covering touched code were run); npm run prettier (App has no prettier script or config — ESLint is the formatting gate here); storybook smoke test and React Compiler check (no .tsx or component changes).

Not covered by a test: the Pusher change. PusherSubscribeWebTest's harness never reaches the production !socket branch, because __DEV__ is true under Jest so that path rejects instead. It is a one-line swap of the telemetry call with no surrounding logic change, and the existing Pusher suites pass.

Items 1, 3 and 5 from the shortlist are untouched. Item 1 (adding the known network strings to ignoreErrors) is still the biggest single lever, and it needs a beforeSend hook that does not exist yet — say the word and I will take that one next.

---

view run

[mountiny]

Concrete filter candidates from the 14 day production crash scan on 2026-08-30. These are all error.unhandled:true and unresolved, so they sit at the top of the crash list and crowd out real crashes, but none of them are App bugs.

Sentry ID	Users (14d)	Events (14d)	What it actually is
APP-AV2	642	1433	gib.js (Group-IB) cross origin frame access. Whole stack is gib.js, tagged third_party_code: True
APP-2A9	69	843	Failed to load the gib.js script — Group-IB script blocked or offline
APP-AWM	38	86	Same gib.js cross origin throw, / route
APP-FF	19	46	Same gib.js cross origin throw, search route
APP-DKT	32	665	Failed to fetch (www.google.com)
APP-JTF	25	899	Failed to fetch (pagead2.googlesyndication.com) — ad blockers
APP-BMR	26	77	Failed to connect to MetaMask — browser extension

APP-AV2 alone is the single highest user count in the whole 14 day window, ahead of every real crash. The four gib.js rows together are roughly 768 users and 2400 events.

Worth deciding whether Group-IB throws should be dropped at the SDK level via beforeSend on the third_party_code tag, or filtered as inbound rules. Either way they should not appear in unhandled crash triage.

Separately, APP-J1W and APP-GXP (the minified reading 'se' pair) contributed roughly 2 million events in 14 days between them from about 383 users. That event to user ratio is the largest single volume contributor found in this scan and is likely worth a dedicated look for the overage question.

[melvin-bot]

@rlinoz

The solution for this issue has been 🚀 deployed to production 🚀 in version 9.4.68-1 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• Send expected bootsplash, auth retry and Pusher teardown signals to Sentry logs instead of the error stream #99789

[rlinoz]

missing mobile-expensify PR