Skip to content

fix(flags): keep shadow mismatch state in redis - #90135

Merged
trunk-io[bot] merged 10 commits into
masterfrom
fix/flags-shadow-tracker-redis
Aug 28, 2026
Merged

trunk-io[bot] merged 10 commits into
masterfrom
fix/flags-shadow-tracker-redis

Conversation

@JamesPatrickGill

@JamesPatrickGill JamesPatrickGill commented Aug 27, 2026 •

Copy link
Copy Markdown
Member

Problem

The Rust flags cache builder ramp gates on a parity metric that could read zero whether or not the two caches agreed.

  • flags_cache_shadow_mismatch_total counts drift the shadow compare saw twice in a row for one team. That confirmed count is the gate.
  • Promoting a first sighting to confirmed needs the team's previous shadow build. The tracker held that pending state in a process-local HashMap.
  • Builder pods roll, and Kafka partitions move between them, more often than the 24 hour FLAGS_CACHE_SHADOW_MISMATCH_TTL elapses. The real confirmation window was the pod lifetime, not the TTL.
  • A team that rebuilt less often than a pod lived could never confirm, however far its cache entry had drifted.
  • A gate that cannot fire is worse than a weak one, because the zero reads as parity.
  • No drift went uncounted. flags_cache_shadow_mismatch_first_sight_total saw every occurrence.

Tracking issue: #88339. Shadow mode landed in #87891 and the shadow publisher in #88585.

Changes

Nothing a user sees changes. A shadow build never writes a served cache entry, and this PR does not change what the compare compares.

  • A mismatch now confirms whenever the team's next shadow build lands inside the TTL, whichever pod handles it.
  • The pending fingerprints live in the flags Redis tier, one key per mismatching team, at posthog:1:feature_flags/shadow_mismatch/{team_id}.
  • The key's TTL is the expiry, driven by FLAGS_CACHE_SHADOW_MISMATCH_TTL. The in-process prune path is gone.
  • The tracker shares the Redis client build_cache_clients already builds for the hypercache writer and reader. No second connection and no new setting.
  • Each tracker call carries a 500ms timeout, the same bound common_hypercache puts on its own Redis access. Shadow teams run in front of the next batch of real invalidations, so an unbounded wait there becomes head-of-line delay for real builds.
  • Store failures fall to first sight, never to confirmed. An unreachable Redis loses confirmations instead of inventing them, and no store failure fails a build.
  • A failed write clears the key rather than leaving the previous observation in place. Without that, the next build compares against an observation from two builds ago and can confirm a disagreement the build in between did not see.
  • Store failures increment flags_cache_shadow_tracker_store_errors_total{op}, where op is read, write or clear. Like the rest of the shadow telemetry, it must not feed the real-build failure alert.
  • The stored fingerprint set is capped at 256 per team. A systematic builder bug makes every flag on every team disagree at once, and the whole set would otherwise be written, held for the TTL, and read back, against the Redis tier the serve path reads from.
  • The fingerprint hashes with SHA-256 instead of DefaultHasher. The standard library does not promise DefaultHasher output across Rust versions, so a toolchain bump would change every fingerprint and demote each team's pending mismatch to a first sighting for one build after that deploy.

What confirmed still cannot catch

Warning

confirmed is a lower bound on drift, not a measure of it. A low count is not evidence of parity.

A team's next shadow build only happens because Python rebuilt the team, and the fingerprint hashes the built value and the cached value. So an edit between two shadow builds can move the fingerprint and start the count over.

Case Why it cannot confirm
Drift on the flag the intervening edit touched Both sides change, so the fingerprint changes. Every occurrence reports as a first sighting.
evaluation_metadata drift on an actively edited team The subject is the whole team, so almost any edit churns the fingerprint.
A team that rebuilds less often than the TTL There is no second build inside the window to confirm against.

First-sight logging is the planned way to cover these cases. It is not in this PR, and at the low ramp steps a first sighting still logs nothing, so its issue_type count is all an operator gets.

Costs

  • One Redis round trip per clean shadow build (DEL), two on a mismatch (GET then SETEX), and one more DEL when a write fails. These were in-process HashMap operations before.
  • A failed clear leaves the previous fingerprints standing until the TTL, so an identical disagreement returning later can confirm one build early. It needs the same content hash to recur. The counter shows it: op="clear" alone, or op="write" with op="clear".
  • A GET and DEL pipeline would make the read destructive, and was rejected: a failed pipeline leaves the same residual and costs an extra write on every mismatch.
  • A fingerprint collision would need a flag key holding a marker byte, because the hashed parts are separated by markers rather than escaped. It would cost one wrongly paired confirmation for one team.
  • Redis holds one key per mismatching team, bounded by the TTL and by the 256-fingerprint cap.
  • No migration. The prior state was in-process, so the first deploy starts from an empty store.

Follow-up in the grafana-dashboards repo: the feature-flags-cache-builder dashboard needs a panel for flags_cache_shadow_tracker_store_errors_total, and one for first sightings by issue_type. The store-error series does not exist until it first increments, so an alert on it needs absent_over_time() or or vector(0) rather than a comparison against zero.

How did you test this code?

Unit tests over MockRedisClient, extending the existing tracker tests in cache_shadow.rs. No test needs a live Redis.

Every branch the tracker takes is covered, including each way the store can fail:

  • Confirm across a process restart. The mock records writes but serves no reads, so a helper replays the recorded write into a fresh client. No existing test covered state crossing a process boundary.
  • A clean build clears the pending key, asserted against the recorded DEL.
  • A failed write clears the stale observation, asserted against the recorded DEL on a store that already held a previous observation.
  • A failed write whose clear also fails reports both ops. That pair is the documented residual.
  • A stored value that does not parse reports as a read failure rather than as an absent key.
  • The 256-fingerprint cap, with 276 diffs: 256 confirm and 20 do not.
  • The fingerprint encoding is pinned by value, so a change to the hashed parts, the marker bytes, or the digest cannot pass silently.
  • The write carries the configured TTL, and the key is asserted by string. A tracker write that landed on posthog:1:cache/teams/... would corrupt a served entry.

Each assertion above was checked by mutation rather than by passing: breaking the branch it covers fails that test and, in every case but one, nothing else. Two of these tests exist because the first version of the clear test passed with the DEL deleted.

Two branches stay uncovered on purpose. The cap test cannot pin which diffs are stored, because the next build compares all of its diffs against the stored set, so any 256 of them produce 256 confirmations. The 500ms timeout firing needs a delay hook on get and setex in the shared common_redis mock, and what it would prove is four lines whose error path the read-failure test already covers.

Not run: the builder against a live Redis or Kafka.

Automatic notifications

  • Publish to changelog?

Docs update

No doc change was needed. docs/internal/feature-flags/hypercache-system.md describes the Python verifier's parity checks, and never the Rust shadow compare or the first-sight and confirmed semantics.

🤖 Agent context

Claude Code (Opus 5) wrote the commits from a written brief that named the defect, the deliverables, and the out-of-scope areas. Tools used: Bash, Read, Edit, Write, Agent.

Skills invoked: /writing-tests, /writing-code-comments, /writing-user-facing-copy, /ste-writing, /writing-pr-descriptions, /write-plan, /worktree-brief.

Across the session: the branch did not contain the shadow compare from #87891 at all, so it was fast-forwarded to master before any work started. The fingerprint change came out as its own commit once it turned out to be a prerequisite for comparing the value across processes rather than a cleanup.

A read-only review agent then ran the repo's specialized correctness, testing and performance reviewers over the two original commits. It found two defects that the original commits' own comments denied: a failed write left the previous observation in place, and the clear test passed with the DEL deleted. The last four commits are that review applied. Two of the reviewers' own claims did not survive verification, including one that read the wrong docstring and repeated it back as correct.

No local Greptile pass ran. The Greptile CLI install fails with EPERM on this machine.

Public artifact: everything in the diff and in this description comes from this repository. The brief that drove the work stays untracked and out of the diff.

JamesPatrickGill and others added 2 commits August 27, 2026 13:27
The fingerprint decides whether a shadow disagreement is the same one the
team's previous shadow build saw. `DefaultHasher` gives the same value for
the same input inside one process, but the standard library does not
promise that value across Rust versions.

The next change writes the fingerprint to Redis and compares it in a later
process, so the value must be the same in every build. SHA-256 truncated to
64 bits is. The marker bytes between the hashed parts keep an absent part
distinct from an empty one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A shadow mismatch counts as confirmed when the team's next shadow build
sees the same disagreement. The tracker kept that pending state in a
process-local `HashMap`, so the state died with the consumer pod. Builder
pods roll far more often than the 24h `FLAGS_CACHE_SHADOW_MISMATCH_TTL`
elapses, which made the real confirmation window the pod lifetime: a team
that rebuilds less often than a pod lives could never confirm, however
wrong its cache entry was. `flags_cache_shadow_mismatch_total` therefore
measured pod churn, not parity, and the serve ramp gates on it.

The pending fingerprints now live in the flags Redis tier, under
`posthog:1:feature_flags/shadow_mismatch/{team_id}`, written with the
configured TTL. Expiry is that TTL, so the in-process prune path is gone.
The tracker uses the Redis client `build_cache_clients` already builds for
the hypercache writer and reader, so there is no new connection and no new
setting.

Every store failure resolves towards first sight, never towards confirmed:
a mismatch we cannot prove is a repeat is reported as a first sighting, so
an unreachable Redis loses confirmations instead of inventing them. No
store failure fails a shadow build. Failures increment
`flags_cache_shadow_tracker_store_errors_total{op}`, which like the rest of
the shadow telemetry must not feed the real-build failure alert.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@JamesPatrickGill JamesPatrickGill self-assigned this Aug 27, 2026
@trunk-io

trunk-io Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

😎 Merged successfully - details.

@github-actions

Copy link
Copy Markdown
Contributor

🤖 CI report

✅ Trunk lane — non-backend lane

This PR is assigned to the non-backend lane. It does not run backend Python tests and may merge in parallel with PRs in other lanes.

@trunk-io

trunk-io Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

Static Badge   Static Badge   Static Badge

View Full Report ↗︎ ⋅ Docs

JamesPatrickGill and others added 5 commits August 27, 2026 15:10
…ails

`observe` reads the team's previous fingerprints, then writes the current
ones. A failed write left the previous build's fingerprints in place with
their own TTL, so the team's next build compared against an observation
from two builds ago and could report `confirmed` for a disagreement the
build in between did not see. That inflates the metric the serve ramp
gates on.

Redis rejects writes with `OOM command not allowed` at `maxmemory` while
still serving reads, so every team's key freezes at whatever it held. The
condition is reachable, not theoretical.

A failed write now clears the key instead, which keeps the fallback at
first sight. Redis serves DEL when it refuses a write for `maxmemory`, so
the compensation works in the case that produces the problem. When the
clear fails too, the counter shows `op="write"` with `op="clear"`.

Three comments claimed the old behavior held the invariant. They now
describe the residual instead of denying it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The tracker's three Redis calls were the only ones in the builder with no
outer timeout. Every `common_hypercache` access wraps its call in a 500ms
`tokio::time::timeout`; the tracker relied on the client's response
timeout, which defaults to 1000ms and which
`FLAGS_REDIS_RESPONSE_TIMEOUT_MS=0` disables entirely.

Shadow teams run sequentially after a batch's real builds, so a slow Redis
adds its wait per team to the head-of-line delay in front of the next
batch of real invalidations. At the default batch size that reached the
consumer's own poll interval, which drops the consumer from the group and
redelivers the batch into the same slow Redis.

Each call now carries the same 500ms bound as the hypercache reads beside
it. An elapsed timer reports as the client's timeout error, so it lands on
the existing store-error path: the build falls back to first sight and the
failure counts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`tracker_clears_pending_state_on_match` passed with the DEL removed. The
test built each step's store from the previous step's recorded calls, and
an unseeded mock reads as an absent key, so the third build reported a
first sighting whether or not the clean build cleared anything. On master
the same test held one `HashMap` across all three calls, where the removal
was load-bearing, so the coverage was lost in the move to Redis.

It now asserts against the store: exactly one DEL, on the team's key.

`tracker_write_failure_leaves_nothing_to_confirm_against` asserted the
same tautology, and its name described behavior the code did not have. It
now injects the write failure on a store that holds a previous
observation, and asserts the compensating clear. Renamed to match.

Both fixes were checked by mutation: dropping the DEL fails the first,
dropping only the compensating clear fails the second and nothing else.

Also models TTL expiry as an explicitly absent key rather than an
unconfigured mock, extracts the second disagreement into
`other_mismatch_diffs`, and names the team key once.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`write_pending` stored one fingerprint per diff with no bound. The state
shadow mode exists to find is a systematic builder bug, where every flag
on every team disagrees at once, so a large team would write hundreds of
kilobytes, hold it for the TTL, and read it back on its next build,
against the Redis tier the serve path reads from. The confirmed log line
already caps the same data at 20 entries and 4096 bytes; the Redis payload
had no equivalent, and the in-memory prune threshold that bounded the old
map went away with it.

Capped at 256 fingerprints per team. `diff_live_entry` returns a stable
order (metadata, then flags by id, then cohorts by id), so a disagreement
keeps its place and a truncated set still confirms.

Also corrects two comments this change's earlier commits got wrong. The
marker bytes in the fingerprint are not an escape, because `subject` is a
formatted string rather than JSON and can carry a marker byte; the fixed
part count is what keeps distinct disagreements apart. And the serve-path
cache key is `posthog:1:cache/teams/{team}/feature_flags/flags.json`, not
the path the key comment named.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@JamesPatrickGill
JamesPatrickGill marked this pull request as ready for review August 27, 2026 14:42
@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team August 27, 2026 14:43
@posthog-project-board-bot posthog-project-board-bot Bot moved this to In Review in Feature Flags Aug 27, 2026
@greptile-apps

greptile-apps Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "Merge remote-tracking branch 'origin/mas..." | Re-trigger Greptile

@haacked haacked changed the title fix(feature-flags): keep shadow mismatch state in redis fix(flags): keep shadow mismatch state in redis Aug 27, 2026

@haacked haacked left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice fix! No blocking issues. A few non-blocking suggestions inline: one optional perf idea and three test-coverage gaps on the store-failure paths.

},
);
let mut store_errors = Vec::new();
let prior = match self.read_pending(key.clone()).await {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: On the mismatch path, observe reads Redis then writes it as two sequential round-trips (GET at line 536, then SETEX at line 545), each under its own 500ms timeout. The write payload doesn't depend on the read result, so the two are independent and could collapse into one pipelined round-trip via execute_pipeline.

This is invisible normally because mismatches are rare. It matters in the exact scenario shadow mode exists to catch: a systematic builder bug that makes every team mismatch at once. Then every shadow build takes the two-round-trip path serially across the batch, and during a Redis brownout each team can burn up to 500ms on the GET timeout plus another 500ms on the SETEX before falling back. Pipelining GET+SETEX halves both the round-trips and the timeout budget on that path. Unlike the GET+DEL pipeline the PR description rejected as destructive, GET+SETEX is safe to combine. Your call whether the rarity of this path justifies the added branching.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right that the two are independent. One round trip would halve both the trips and the timeout budget on the mismatch path.

One thing pushes me the other way. execute_pipeline wraps the commands in MULTI/EXEC, so a connection-level failure means neither ran. Today a failed read still lets the write land, so a bad read costs this build's confirmation and the chain continues. Pipelined, that same failure costs the write too, so the next build has nothing to compare against either. Under the intermittent slowness your scenario describes, the current shape degrades more gracefully.

That is recoverable. Per-command results survive MULTI/EXEC, so an inner SETEX error could still map to op="write", and only a connection error would collapse both. But it rewrites the failure semantics on the path this PR just spent several commits getting right, for a case that needs a Redis brownout and a systematic builder bug at the same time.

If that worst case ever bites, lowering STORE_TIMEOUT or narrowing the producer-side gate gets the same relief with no new branching. So I am leaving it for now, and will revisit if the ramp shows shadow duration pushing real-build latency. Happy to do it if you would rather close this with code.

assert_eq!(second.store_errors, vec![TrackerStoreOp::Read]);
}

#[tokio::test]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: The write-then-clear compound failure has no test, so the store_errors == [Write, Clear] path in observe ships unverified. tracker_write_failure_clears_the_stale_observation injects only a setex failure and lets the compensating del succeed, so it never reaches the case where the write fails and the clear that follows also fails. That's the documented residual the PR calls out, and it's the one that leaves stale fingerprints standing until the TTL.

Mirror the write-failure test but also seed del_ret(PENDING_KEY_TEAM_7, Err(CustomRedisError::Timeout)), then assert store_errors == vec![TrackerStoreOp::Write, TrackerStoreOp::Clear].

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks. Added tracker_write_and_clear_failure_leaves_the_stale_observation. It seeds both set_ret and del_ret with errors and asserts store_errors == [Write, Clear].

Checked that it bites: making clear_pending swallow its error fails it. Good catch that this was the one residual with no test behind it.

/// the TTL expires the window, so it reads as "no previous build" and not as
/// an error. A value that does not parse is an error, because a stored shape
/// this build cannot read is worth seeing on the failure counter.
async fn read_pending(&self, key: String) -> Result<HashSet<u64>, CustomRedisError> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Nothing tests the parse-error branch in read_pending, where a stored value that isn't valid JSON returns Err(ParseError) and the build falls back to first sight with a Read store error. That branch is the whole reason read_pending returns an error rather than an empty set there, so a corrupt stored shape shows up on the failure counter instead of silently looking like an absent key.

Seed the read with a non-JSON value via second_pod.get_ret(PENDING_KEY_TEAM_7, Ok("not json".to_string())), then assert the diffs come back as first sight and store_errors == vec![TrackerStoreOp::Read]. tracker_read_failure_falls_back_to_first_sight is the test to mirror.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. tracker_unreadable_stored_value_is_reported_not_ignored seeds the read with "not json" and asserts first sight plus store_errors == [Read].

Verified by making the parse error read as an absent key, which fails it.

key: String,
diffs: &[ShadowDiff],
) -> Result<(), CustomRedisError> {
let fingerprints: Vec<u64> = diffs

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: The 256-fingerprint cap in write_pending has no test, and its comment makes a correctness claim that matters: because diff_live_entry returns diffs in a stable order, a diff that fits under the cap keeps the same slot across builds and still confirms, whereas a cap over an unordered set would silently never confirm. Nothing exercises that.

A test that runs two builds with more than 256 diffs and asserts a within-cap diff confirms on the second build would catch a regression that reordered or unstably truncated the set. No existing test builds more than one diff, so this would be a new test rather than an extension of an existing one.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added, and it turned up that the justification in that comment was overstated, including mine.

tracker_confirms_the_diffs_that_fit_the_stored_cap runs 276 diffs and asserts that 256 confirm and 20 do not. Removing the cap fails it. But storing the last 256 instead of the first leaves it green.

That is not a gap in the test. It is the mechanism: the next build compares every diff it finds against the stored set, so any 256 of them produce 256 confirmations. The aggregate count does not depend on the order at all.

What the stable order actually buys is narrower than the comment claimed. It makes one specific flag's disagreement confirm on every build, instead of on roughly nine builds in ten. Both comments now say that. The test comment says what it does not pin, so nobody reads it as an ordering guard.

/// It replays *attempted* writes. The mock records a call whether or not it
/// returned an error, so a test that injects a write failure must build the
/// next store itself rather than call this.
fn next_build(prior: &MockRedisClient) -> MockRedisClient {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: The "replay recorded writes into a fresh mock" logic exists twice now: next_build here (which replays both setex and del) and the inline loop in shadow_observation in flags_cache_builder.rs (which replays only setex). They model the same cross-process behavior, so a change to the mock's recording shape would have to move both. Since they're in separate test modules, folding them into one shared helper is optional; a short comment on each pointing at the other would be enough.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment on each pointing at the other, rather than folding them into a shared helper, because they sit in different test modules.

Both now say that a change to what the mock records has to reach both places.

@github-project-automation github-project-automation Bot moved this from In Review to Approved in Feature Flags Aug 27, 2026
JamesPatrickGill and others added 3 commits August 28, 2026 08:48
Three paths shipped unverified. Each is a branch the tracker takes when
Redis misbehaves, which is when the confirmed count matters most.

- A write that fails and a clear that then also fails. That pair is the
  residual `observe` documents, and the counter is the only signal that a
  team's next confirmation may be one build early.
- A stored value that does not parse. It reports as a read failure rather
  than as an absent key, so a value shape that changed across deploys shows
  on the counter instead of looking like an expired window.
- The 256-fingerprint cap, with more diffs than it holds.

Also corrects the cap's own comment. It claimed a cap over an unordered
set would never confirm. Mutation shows otherwise: storing the last 256
instead of the first leaves the test green, because the next build compares
every diff it finds against whatever was stored, so any 256 of them produce
256 confirmations. The stable order makes one flag's disagreement confirm on
every build rather than on some. The aggregate count does not depend on it.

The test comment now says what the test pins and what it does not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The fingerprint is written to Redis and compared by a later process, so
its encoding is a cross-process contract. Nothing guarded it. A change to
the hashed parts, the marker bytes, or the digest would stop every team's
in-flight pending state from matching for one TTL window after the deploy,
and confirmations would drop to zero with nothing saying why.

Two values are pinned: one with every part present, one with none, so both
marker paths are covered. Called with literal parts rather than through
`diff_live_entry`, so a new field on `FeatureFlag` does not fail it.

Mutation-checked: swapping a marker byte fails it, and dropping a hashed
part fails it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…eached

`clear_pending` treated `Err(NotFound)` from DEL as success. The real
client cannot produce it: `RedisClient::del` returns `Ok(())` whatever the
key's state, because Redis answers a DEL of a missing key with 0 rather
than an error. The arm existed because `MockRedisClient::del` returns
`NotFound` for a key no test configured, so every clean-build test reached
it.

A branch only a test double can reach is a branch the tests shape rather
than verify, and the comment above it explained Redis DEL semantics as
though that were the reason it was there.

The two tests that clear a key now say what DEL returns. Production
behavior is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@JamesPatrickGill

Copy link
Copy Markdown
Member Author

/trunk merge

@trunk-io
trunk-io Bot merged commit 5682f62 into master Aug 28, 2026
198 checks passed
@trunk-io
trunk-io Bot deleted the fix/flags-shadow-tracker-redis branch August 28, 2026 09:48
@github-project-automation github-project-automation Bot moved this from Approved to Done in Feature Flags Aug 28, 2026
@deployment-status-posthog

deployment-status-posthog Bot commented Aug 28, 2026 •

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-08-28 10:11 UTC Run
prod-us ✅ Deployed 2026-08-28 10:59 UTC Run
prod-eu ✅ Deployed 2026-08-28 10:59 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

2 participants