Skip to content

fix(replay): keep session replay masks aligned with the captured frame - #786

Merged
ioannisj merged 6 commits into
mainfrom
fix/replay-scroll-mask-drift
Aug 28, 2026
Merged

ioannisj merged 6 commits into
mainfrom
fix/replay-scroll-mask-drift

Conversation

@ioannisj

@ioannisj ioannisj commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Session replay masks drift off the content they cover while a screen scrolls or animates. The mask rects get measured at one instant and the pixels get rendered at another, so on a fast scroll the redaction lands behind the text it's meant to hide.

Two things cause it:

  • Mask geometry came from the model layer (toAbsoluteRect). During a Core Animation animation the model layer sits at its destination while the screenshot renders the presentation layer's in-flight position.
  • drawHierarchy lags the render server, so even correct rects can sit ahead of the pixels that actually ship.

The base fix is toPresentationRect: mask geometry now reads the presentation tree, the same source the screenshot renders from. Wireframe geometry stays on toAbsoluteRect, it has no pixels to agree with.

On top of that there are two mechanisms, both driven by the same measurement. Before each capture scheduleSettledCapture samples mask geometry, waits 25ms (roughly one display-pipeline depth), samples again, and bands the result by the largest per-mask displacement. All of it lives in PostHog/Replay/PostHogMaskSettleAnalysis.swift.

1. Drop to the presentation-tree renderer on fast scrolling

The band picks the renderer:

  • Still, at or under 80pt/s (2pt across the window): rects used as collected, full drawHierarchy.
  • Drift, at or under 2000pt/s (50pt): still drawHierarchy, with the masks inflated as described below.
  • Motion, anything faster: render(in:) against the presentation tree, whose masks can't disagree with its own pixels.

Unreadable geometry fails closed into motion, so an owner that appeared or disappeared between the samples, a duplicate owner, a count mismatch, or a nil sample all take the aligned renderer rather than guessing. Pairing is by owner identity (MaskedRegion), not by index, because index pairing would compare unrelated rects and report a false settle whenever the set changed shape.

The cost of the motion band is real and it's in the changeset: render(in:) draws blur, video and Metal flat, and it ignores layer.mask, maskView and Core Animation filters. Anything that must never appear in a recording needs postHogMask().

2. Inflate masks across the two samples

In the drift band the frame keeps full fidelity and the masks absorb the movement instead. Each mask becomes the union of four rects: where it was, where it is, a lead of 1/16 of the measured displacement past the newer sample, and the older sample pushed back by the full displacement. That last one covers a position that was never sampled, since the display pipeline is deeper than a single settle window and the pixels on screen trail the older sample.

screenshotModeBackgroundCapture uses the same idea without a threshold. performBracketedBackgroundCapture puts the off-main render between two samples and masks the per-owner union, so the mask provably covers wherever the content sat while the render ran. An owner present in only one of those samples emits its rect as collected rather than dropping the frame, because a scroll that recycles cells changes the owner set constantly and dropping those frames would freeze the recording for the whole interaction.

Defaults are deliberately conservative

The thresholds are set so that in practice only a fling changes anything. Measured on device across a 25ms window: slow drift moves about 1.5pt, a Core Animation animation about 4pt, a scroll fling 185pt or more. So ordinary movement stays on the renderer session replay has always used, and the flattened frames are limited to the part of a fling nobody is reading anyway. Every constant is tunable in one place with the reasoning next to it, so these are easy to move once feedback lands.

💚 How did you test it?

33 tests in PostHogTests/PostHogMaskScenarioTest.swift, in two groups:

  • Scenario coverage: per-row and whole-row masks tracking new positions after scrolling, leaf masks hugging their labels, fail-closed nesting of postHogMask() and postHogNoMask().
  • Settle bands: both boundaries exactly and just past, a nil or unpairable sample falling to motion, a duplicate owner not trapping, permuted sample order still settling, and the inflated rects covering the swept region.
  • Manual tests on simulator and physical device on local mock server inspecting the snapshots as they come through
mask-drift-demo.mp4

If you have a repro or a sample project that showed masks sliding off during a scroll, please run this branch against it and check the playback, particularly whether the flattened frames during a fling are an acceptable trade for aligned masks. That judgement is the part I'd most like a second opinion on.

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

DRI: @ioannisj
Autonomy: Human-driven (agent-assisted)

@ioannisj ioannisj self-assigned this Aug 28, 2026
@github-actions

github-actions Bot commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

posthog-ios Compliance Report

Date: 2026-08-28 15:15:02 UTC
Duration: 225818ms

✅ All Tests Passed!

45/45 tests passed


Capture Tests

✅ 29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 2813ms
Format Validation.Event Has Uuid ✅ 2800ms
Format Validation.Event Has Lib Properties ✅ 2652ms
Format Validation.Distinct Id Is String ✅ 2692ms
Format Validation.Token Is Present ✅ 2786ms
Format Validation.Custom Properties Preserved ✅ 2766ms
Format Validation.Event Has Timestamp ✅ 2784ms
Retry Behavior.Retries On 503 ✅ 11639ms
Retry Behavior.Does Not Retry On 400 ✅ 4779ms
Retry Behavior.Does Not Retry On 401 ✅ 4801ms
Retry Behavior.Respects Retry After Header ✅ 7672ms
Retry Behavior.Implements Backoff ✅ 21792ms
Retry Behavior.Retries On 500 ✅ 8817ms
Retry Behavior.Retries On 502 ✅ 9111ms
Retry Behavior.Retries On 504 ✅ 9187ms
Retry Behavior.Max Retries Respected ✅ 21834ms
Deduplication.Generates Unique Uuids ✅ 3015ms
Deduplication.Preserves Uuid On Retry ✅ 5423ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 16861ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 7830ms
Deduplication.No Duplicate Events In Batch ✅ 2811ms
Deduplication.Different Events Have Different Uuids ✅ 2943ms
Compression.Sends Gzip When Enabled ✅ 291ms
Batch Format.Uses Proper Batch Structure ✅ 2834ms
Batch Format.Flush With No Events Sends Nothing ✅ 327ms
Batch Format.Multiple Events Batched Together ✅ 2934ms
Error Handling.Does Not Retry On 403 ✅ 4915ms
Error Handling.Does Not Retry On 413 ✅ 4838ms
Error Handling.Retries On 408 ✅ 5557ms

Feature_Flags Tests

✅ 16/16 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 2799ms
Request Payload.Flags Request Uses V2 Query Param ✅ 2819ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 2782ms
Request Payload.Flags Request Omits Authorization Header ✅ 3000ms
Request Payload.Token In Flags Body Matches Init ✅ 2813ms
Request Payload.Groups Round Trip ✅ 2831ms
Request Payload.Groups Default To Empty Object ✅ 2756ms
Request Payload.Person Properties Distinct Id Auto Populated When Caller Omits It ✅ 2871ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 2860ms
Request Payload.Disable Geoip Omitted Defaults To False ✅ 2854ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 2954ms
Request Lifecycle.No Flags Request On Init Alone ✅ 52ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 2806ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 5600ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 2777ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 3097ms

@turnipdabeets turnipdabeets left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ioannisj
ioannisj marked this pull request as ready for review August 28, 2026 15:02
@ioannisj
ioannisj requested a review from a team as a code owner August 28, 2026 15:02
Comment thread PostHog/Replay/UIView+Util.swift Resolved
@veria-ai

veria-ai Bot commented Aug 28, 2026 •

Copy link
Copy Markdown

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "test(replay): derive mask settle boundar..." | Re-trigger Greptile

@ioannisj
ioannisj merged commit ac3dc31 into main Aug 28, 2026
49 checks passed
@ioannisj
ioannisj deleted the fix/replay-scroll-mask-drift branch August 28, 2026 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants