[haacked]

Problem

Anyone measuring per-flag usage still has to scan the events table, because the table built to serve those queries has never had a producer.

flag_evaluations, its schema mirror, its deletion sweep, and its column kinds all landed first (#72990, #80994, #82056, #83382). This PR adds the missing producer.

#83382 was the last blocker. It recreated the table to make the typed columns assignable, which was only free while the table had no producer.

Changes

• The ingestion pipeline can now write $feature_flag_called events into flag_evaluations.
• Nothing is different for any user yet, except one timestamp detail: events.created_at, which HogQL exposes, now records event-assembly time instead of serialize time. The gap is milliseconds, and both tables now agree on the value for the same event.
• The fork ships dark by construction: the topic, mode, and teams settings each default off.
• Enabling is a sequence: create the topic, set INGESTION_OUTPUT_FLAG_EVALUATIONS_TOPIC, set INGESTION_FLAG_EVALUATIONS_MODE to dual_write, and name teams in INGESTION_FLAG_EVALUATIONS_TEAMS. Mode without topic or teams is a logged no-op.
• An empty topic also skips startup topic verification, so deploying this cannot crash a fleet whose broker lacks the topic.
• INGESTION_FLAG_EVALUATIONS_EXCLUDED_TEAMS holds back named teams. Setting it to * disables the fork.
• Once enabled, an event lands in flag_evaluations and still reaches the events table unchanged.
• Every failure path continues toward the events table, so a broken shadow write cannot drop a real event.

The decisions below carry the correctness of this PR. The rest of the diff is mechanical wiring of config, outputs, and topics.

• The fork sits between create-event and emit-event, where the assembled events row exists.
• It matches on the event name, so the $experiment_exposure duplicate that create-event appends for allowlisted teams stays out of the shadow table.
• The row goes through serializeEvent, so shared fields are byte-identical to what the events table receives for the same event.
• That identity needs one stamp: created_at is now set once at create-event and reused, where each serialization previously stamped it fresh.
• The producer omits flag_key, response, session_id, request_id, and $group_0..4, which the shard computes from properties.
• Those columns are DEFAULT, which only fills when the insert omits it, so a producer that named one would store a value contradicting properties.
• A lane gate restricts the fork to the real-time lanes. The backfill owns history for this table, so a delayed lane that also forked would double-count the per-flag usage the table exists to answer. The dedup service next to it gates the same way.
• The produce ack rides in the batch's side effects, so an unwritable topic leaves it pending and the consumer stops committing offsets. Neither ack arm runs while it is merely pending, so ingestion_flag_evaluations_pending_acks carries that signal.
• The ack never rejects, because only consumer v2 gets the redelivery a rejection asks for. A produce that fails for good counts produce_failed and drops the shadow row.
• Consumer v1 stores the offset from a .finally regardless, then stops the process on the unhandled rejection.
• ingestion-api-server turns one rejection into a pod shutdown for every team it serves, and the gRPC driver fails every open stream.
• PipelineContext.sideEffects now documents that contract at the declaration site.

Note

Shedding the topic dependency during an incident takes both env vars, not just the mode. The mode stops a running consumer from forking, but startup topic verification follows the output registration, which the mode does not touch. A pod that restarts with INGESTION_OUTPUT_FLAG_EVALUATIONS_TOPIC still set will not start while the topic is unreachable.

Before

flowchart LR
    CE[create-event] --> EE[emit-event]
    EE --> EV[(events)]
    classDef phBlue fill:#1d4aff,stroke:#1d4aff,color:#fff;
    classDef phGray fill:#e5e7eb,stroke:#c7ccd1,color:#000;
    class CE,EE phBlue;
    class EV phGray;

Loading

After

flowchart LR
    CE[create-event] --> FK[fork-flag-evaluations]
    FK --> EE[emit-event]
    EE --> EV[(events)]
    FK --> TP[clickhouse_flag_evaluations]
    TP --> KT[kafka_flag_evaluations]
    KT --> MV[flag_evaluations_mv]
    MV --> FE[(flag_evaluations)]
    classDef phBlue fill:#1d4aff,stroke:#1d4aff,color:#fff;
    classDef phYellow fill:#f9bd2b,stroke:#f9bd2b,color:#000;
    classDef phGray fill:#e5e7eb,stroke:#c7ccd1,color:#000;
    class CE,EE,MV phBlue;
    class FK,TP,KT phYellow;
    class EV,FE phGray;

Loading

How did you test this code?

An end-to-end test covers the whole path, which no unit test can. A topic-key typo, a row column ClickHouse cannot parse, or a broken projection all produce "row queued but never lands".

• dual_write mode: flag_key and response read back computed from properties, though the producer sends neither.
• dual_write mode: person_id matches what the events table stamped for the same event, which per-flag uniques depend on.
• Default mode: the event reaches the events table and flag_evaluations stays empty.

Unit tests cover the row mapping, the service, and the fork step. New groups catch three regressions no existing test did: a delayed lane forking when the env vars say dual_write, a produce_failed outcome going uncounted, and a pending ack moving no counter at all, which is the stall.

The outage behavior was measured against a second broker carrying only flag_evaluations, so that output's cluster could be stopped while the events output kept running. With it stopped, the event still reached the events table and the batch's side-effect drain stayed pending past 60 seconds with no error raised. It stalls rather than dropping.

The created_at change touches every event, so its consumers were surveyed rather than reasoned about. It sits in no ORDER BY, PARTITION BY, TTL, or version-column position; the events ReplacingMergeTree version is _timestamp. No query computes lag from it. The one behavioral consumer is distinct_id_usage_mv, which buckets by minute.

Not checked: any behavior on a real multi-shard cluster, any load or throughput characteristic of the fork, and customer-authored HogQL that filters on events.created_at, which is stored per team rather than in this repo.

Manual test plan

Unit tests already cover the row mapping and column set, the service's config parsing (invalid mode, empty topic, wildcard exclusion), the team gate, and the fork step's outcomes and failure isolation; one e2e covers dual_write and default mode through the ingestion consumer. This manual plan targets what none of them reach: the startup/deploy safety claim, the four-step enable sequence and its partial states, the ingestion-api-server entrypoint, lane and dedup interactions, the created_at change's blast radius beyond flag events, and rollback under a topic outage.

Deploy safety and the enable sequence

• Dark deploy against a broker lacking the topic: Do not create clickhouse_flag_evaluations on the broker. Leave INGESTION_OUTPUT_FLAG_EVALUATIONS_TOPIC empty. Start the ingestion consumer and send a $pageview. Confirm startup reaches ready with no Output topic verification failed error and the event lands in events.
  • Verified: IngestionOutputsComponent.start() resolves with an empty topic (checkTopicExists returns early), and events still reach events.
• Topic named but absent is fatal at startup: Set INGESTION_OUTPUT_FLAG_EVALUATIONS_TOPIC=clickhouse_flag_evaluations with the topic absent from the broker. Start the consumer. Confirm it fails fast with Output topic verification failed for: flag_evaluations rather than starting and failing per event.
  • Verified: start() threw Output topic verification failed for: flag_evaluations. Caveat: the local Redpanda has auto_create_topics_enabled=true and the metadata probe created the topic as a side effect, but the check failed first.
• Mode without topic is a logged no-op: Set INGESTION_FLAG_EVALUATIONS_MODE=dual_write with INGESTION_OUTPUT_FLAG_EVALUATIONS_TOPIC empty. Start the consumer and send a $feature_flag_called. Confirm the warning INGESTION_FLAG_EVALUATIONS_MODE is set but INGESTION_OUTPUT_FLAG_EVALUATIONS_TOPIC is empty, not forking and that the event still reaches events.
  • Verified: warning logged, zero rows in flag_evaluations, event present in events.
• Mode and topic without teams forks nobody: Set mode dual_write and a real topic, leaving INGESTION_FLAG_EVALUATIONS_TEAMS empty. Send a $feature_flag_called. Confirm zero messages on clickhouse_flag_evaluations and the event present in events.
  • Verified: zero rows, event present in events.
• Invalid mode falls back to disabled: Set INGESTION_FLAG_EVALUATIONS_MODE=garbage. Start the consumer and send a $feature_flag_called. Confirm the warning Invalid INGESTION_FLAG_EVALUATIONS_MODE, falling back to disabled and no row in flag_evaluations.
  • Verified: Invalid INGESTION_FLAG_EVALUATIONS_MODE warning logged, zero rows.

Rollout gating on a live stack

• Per-team ramp: Set INGESTION_FLAG_EVALUATIONS_TEAMS to one team ID. Send $feature_flag_called from that team and from a second team. Confirm only the named team's row appears in flag_evaluations and both events appear in events.
  • Partially verified: TEAMS=* forks and TEAMS=<other id> does not. Two teams were not run through one consumer simultaneously.
• Excluded-team escape hatch: Set INGESTION_FLAG_EVALUATIONS_TEAMS=* and INGESTION_FLAG_EVALUATIONS_EXCLUDED_TEAMS=<team_id>. Send a $feature_flag_called from that team. Confirm no row in flag_evaluations, ingestion_flag_evaluations_events_total{outcome="continued_team_not_enabled"} increments, and the event reaches events.
  • Verified: zero rows with TEAMS=* plus the team excluded; continued_team_not_enabled increments (see metrics item).
• Wildcard exclusion as kill switch: With mode dual_write and teams *, set INGESTION_FLAG_EVALUATIONS_EXCLUDED_TEAMS=* and restart. Confirm the warning INGESTION_FLAG_EVALUATIONS_EXCLUDED_TEAMS is "*", disabling the flag evaluations fork and that no fork occurs for any team.
  • Verified: disabling the flag evaluations fork warning logged, no fork.
• ingestion-api-server entrypoint: Apply the same dual_write config to ingestion-api-server rather than the ingestion consumer. POST a batch containing a $feature_flag_called. Confirm the row lands in flag_evaluations, since the e2e only exercises IngestionConsumer.
  • Verified by inspection only: the server calls the same createFlagEvaluationsService(this.config) and passes it into the same createJoinedIngestionPipeline. No HTTP round trip was run, because booting the server stands up its own Postgres, Redis, Kafka, and HTTP stack.

Lane and cross-feature interactions

• Overflow lane forks: Enable dual_write on the overflow lane and force a $feature_flag_called to overflow. Confirm the row still lands in flag_evaluations with the correct team_id.
  • Verified: with INGESTION_LANE=overflow and dual_write, the row landed in flag_evaluations with the right team_id.
• Historical lane stays dark: Leave the historical lane at disabled while realtime runs dual_write. Replay a historical $feature_flag_called. Confirm no message on clickhouse_flag_evaluations from that lane and the event still lands in events.
  • Verified, and the first run found a gap: the fork had no lane gate, so INGESTION_LANE=historical with dual_write forked. A lane gate now restricts the fork to the real-time lanes, matching the dedup service, with unit cases covering historical, async, overflow, and the unset lane.
• Dedup runs before the fork: Set INGESTION_FEATURE_FLAG_CALLED_DEDUP_MODE=drop alongside dual_write. Send two identical $feature_flag_called events inside the dedup TTL. Confirm exactly one row in events and exactly one in flag_evaluations, not two in either.
  • Verified: two identical events with dedup drop produced exactly one row in events and one in flag_evaluations.
• Group columns materialize from properties: Send a $feature_flag_called carrying $groups. Query flag_evaluations. Confirm $group_0 reads back populated even though the producer never sends it.
  • Verified: with a group-type mapping seeded, $group_0 read back as org-42 from flag_evaluations though the producer never sends it.
• Personless events agree across tables: Send a $feature_flag_called with process_person_profile=false. Confirm flag_evaluations.person_id matches events.person_id for that UUID, whatever value person processing assigned.
  • Verified: flag_evaluations.person_id == events.person_id with $process_person_profile=false (a real UUID, not the zero sentinel).

The created_at change (affects all events, not just flag calls)

• Non-flag events keep a sane created_at: With the fork disabled, send a $pageview. Run SELECT created_at FROM events for that UUID. Confirm it is populated, within seconds of now, and not a 1970 epoch value.
  • Verified: $pageview created_at was a current timestamp, not 1970.
• Exposure duplicate shares one stamp: For a team listed in EXPERIMENT_EXPOSURE_DUPLICATION_TEAMS, send a $feature_flag_called. Confirm the $feature_flag_called and $experiment_exposure rows in events carry identical created_at, and only the former appears in flag_evaluations.
  • Verified: both events rows carried an identical created_at; only $feature_flag_called reached flag_evaluations.
• HogQL still exposes created_at: Query created_at via HogQL for a recent event. Confirm it returns event-assembly time and no query error.
  • Partially verified: the HogQL events schema still maps created_at to the ClickHouse column (posthog/hogql/database/schema/events.py:116), and the column value was confirmed current. No HogQL query was executed, because that needs the Django app running.

Observability and rollback

• Metric names and labels reach the scrape endpoint: Drive one event per outcome (enabled team, excluded team, missing $feature_flag, and one carrying $set). Scrape /_metrics. Confirm ingestion_flag_evaluations_events_total with labels dual_written, continued_team_not_enabled, and continued_invalid_flag_key, plus ingestion_flag_evaluations_set_props_total.
  • Verified via the prom-client registry: dual_written, continued_team_not_enabled, continued_invalid_flag_key, and ingestion_flag_evaluations_set_props_total all present. The fork also exports ingestion_flag_evaluations_pending_acks and a produce_failed outcome, added after the outage run showed a stall moved no counter. Registry contents, not an HTTP GET.
• Topic outage stalls rather than drops: With the fork live, make clickhouse_flag_evaluations unwritable and send events (requires stopping the broker or toxiproxy). Confirm the consumer stalls without committing offsets, then restore the topic and confirm every event still reaches events with no loss.
  • Verified with a disposable second broker (the shared one cannot be stopped in isolation). With the consumer already running and that broker stopped: the event still reached events, and the batch's side-effect drain stayed pending past 60 seconds with no error raised. It stalls rather than dropping. No log line named the fork, so ingestion_flag_evaluations_pending_acks is what attributes a stall to it. Both consumers do surface the wait itself at CONSUMER_BACKGROUND_TASK_TIMEOUT_MS.
• Rollback sheds the dependency: With the fork live and the topic still broken, set INGESTION_FLAG_EVALUATIONS_MODE=disabled and restart. Confirm the consumer resumes, events flow to events, and no new rows appear in flag_evaluations.
  • Verified, and it qualifies the docstring. Disabling the mode stops a running consumer from forking, but startup topic verification is driven by the OUTPUT registration, which the mode does not touch. With the broker down, a restart with the topic still set fails startup (Output topic verification failed for: flag_evaluations); clearing INGESTION_OUTPUT_FLAG_EVALUATIONS_TOPIC lets it start. During an incident, where pods restart, the mode alone is not enough.

---

events.created_at consumer survey

The one change in this branch that touches every event, not just flag calls, so its blast radius was surveyed directly rather than reasoned about.

No structural role. created_at appears in no ORDER BY, PRIMARY KEY, PARTITION BY, SAMPLE BY, TTL, or version-column position on any events table. The ReplacingMergeTree version column is _timestamp (Kafka message time), not created_at (posthog/models/event/sql.py:209,213,680). Dedup and sort semantics are unchanged.

No lag or latency queries. Sweeps for dateDiff(... created_at ...), created_at - timestamp, and min/max/argMin(created_at) returned zero events-table hits.

One behavioral consumer, negligible. distinct_id_usage_mv buckets events into minutes by created_at (posthog/models/distinct_id_usage/sql.py:98-105). A millisecond shift can move a single event across a minute boundary in a usage counter with a 7-day TTL.

Everything else is value passthrough: batch exports (whose interval bounds use inserted_at or timestamp, never created_at), the events MVs, HogQL exposure, and frontend display. The value moves earlier by single-digit milliseconds, and nothing branches on it.

Not enumerable from the repo: customer-authored HogQL and saved insights are stored per-team in Postgres. The column's documented meaning ("server-side ingestion time") is preserved and the value moves earlier, so boundary predicates are unaffected in any practical sense.

Automatic notifications

• Publish to changelog?

Docs update

None. No public docs describe this table yet.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Written in Claude Code. Skills invoked: /writing-code-comments, /writing-tests, /writing-pr-descriptions, /test-plan, /squash, /create-pr.

The branch was written on 2026-08-11 and sat unopened while #83382 landed. A later session rebased it and re-verified it against the recreated table, which produced one correctness fix: the row type documented the typed columns as MATERIALIZED, which #83382 made false. Under DEFAULT the distinction matters, because a sent value would win over the computed one.

A /review-code --fix pass found the blocking issue that the output was registered unconditionally with a real topic default, so startup topic verification failed any broker lacking the topic. The fix added the empty-topic default and the service's topic guard.

This session ran the manual test plan above against a local stack. Three findings came out of it and are in the diff. The e2e polled only the flag_evaluations read before asserting on events, which passed one run in four; it now polls both. createFlagEvaluationsService(this.config) never typechecked, because the consumer's config type did not surface INGESTION_OUTPUT_FLAG_EVALUATIONS_TOPIC. The fork had no lane gate while the dedup service beside it did, and the user chose to add one rather than document the difference.

A later session addressed review comments from @eli-r-ph and Graphite. That pass changed the rejection contract: the ack no longer rejects, because only consumer v2 honors a rejected side effect, while v1 stops the process and ingestion-api-server stops the pod. It also corrected the "silent stall" claim here and in the docstrings, fixed a double count in the fork-error outcome, and recorded which Helm release carries the env vars. Skills invoked in that pass: /address-pr-reviews, /writing-code-comments, /writing-tests, /writing-pr-descriptions.

Nothing in this PR derives from session-private material. The measurements come from the local dev stack.

[trunk-io]

😎 Merged successfully - details.

[github-actions]

🤖 CI report

✅ Trunk lane — non-backend lane

This PR is assigned to the non-backend lane. It does not run backend Python tests and may merge in parallel with PRs in other lanes.

[trunk-io]

   

View Full Report ↗︎ ⋅ Docs

[posthog]

🦔 ReviewHog reviewed this pull request

Found 1 must fix, 0 should fix, 1 consider.

Published 2 findings (view the review).

Resolved comments: 1 fixed, 1 declined

[posthog]

ReviewHog Alpha 🦔 If you find any issues helpful - please reply "valid", "invalid", etc., for evaluation purposes 🙏

[posthog]

ReviewHog Report

Found 1 must fix, 1 consider.

[greptile-apps]

Prompt To Fix All With AI

### Issue 1
nodejs/src/ingestion/common/steps/event-processing/fork-flag-evaluations-step.ts:102-111
**Promise-chain acknowledgement handling**

The new Kafka acknowledgement lifecycle uses `ack.then(...)` instead of the repository's preferred async/await style, making its pending, success, and failure behavior harder to follow and maintain consistently.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (1): Last reviewed commit: "feat(flags): shadow-route $feature_flag_..." | Re-trigger Greptile}

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[Copilot]

Pull request overview

Adds a dark-by-default ingestion fork that writes eligible $feature_flag_called events to flag_evaluations while preserving the existing events path.

Changes:

• Adds topic, output, rollout, team, and lane configuration.
• Maps events to flag-evaluation rows and reuses created_at.
• Adds unit and end-to-end coverage.

Review findings:

• Critical (2 votes): Partial queue failures can replay rows and double-count usage without idempotent deduplication.
• Critical (1 vote): MessageSizeTooLarge can repeatedly halt the consumer instead of being treated as a continued fork failure.
• Critical (1 vote): Already-rejected acknowledgements may be cleaned up before batch draining, allowing offsets to commit.
• Moderate (2 votes): Empty team configuration should disable the service as a logged no-op.
• Nit (2 votes): Default-mode E2E coverage should send an event and verify both destinations.

Reviewed changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Summary
nodejs/tests/ingestion/flag-evaluations-e2e.test.ts	Covers end-to-end flag-evaluation ingestion.
nodejs/tests/helpers/kafka.ts	Supports test Kafka topic setup.
nodejs/tests/helpers/ingestion-outputs.ts	Adds test output wiring.
nodejs/tests/helpers/ingestion-e2e.ts	Provides ingestion query and polling helpers.
nodejs/src/types.ts	Updates the processed-event timestamp contract.
nodejs/src/servers/ingestion-api-server.ts	Wires the service into API ingestion.
nodejs/src/ingestion/pipelines/analytics/outputs/registry.ts	Registers the new output.
nodejs/src/ingestion/pipelines/analytics/joined-ingestion-pipeline.ts	Propagates the service through the pipeline.
nodejs/src/ingestion/pipelines/analytics/event-subpipeline.ts	Inserts the fork before normal emission.
nodejs/src/ingestion/pipelines/ai/steps/split-ai-events-step.test.ts	Covers AI event-splitting compatibility.
nodejs/src/ingestion/ingestion-consumer.ts	Initializes the service for Kafka ingestion.
nodejs/src/ingestion/config.ts	Defines rollout and output configuration.
nodejs/src/ingestion/common/team-gate.ts	Provides shared team gating.
nodejs/src/ingestion/common/steps/event-processing/fork-flag-evaluations-step.ts	Queues shadow writes and tracks acknowledgements.
nodejs/src/ingestion/common/steps/event-processing/fork-flag-evaluations-step.test.ts	Tests fork routing, outcomes, and failures.
nodejs/src/ingestion/common/steps/event-processing/emit-event-step.ts	Reuses the assembled event timestamp.
nodejs/src/ingestion/common/steps/event-processing/emit-event-step.test.ts	Tests event emission behavior.
nodejs/src/ingestion/common/steps/event-processing/create-event.ts	Assigns created_at during event assembly.
nodejs/src/ingestion/common/steps/event-processing/create-event-step.test.ts	Tests event creation and timestamp handling.
nodejs/src/ingestion/common/flag-evaluations/flag-evaluations-service.ts	Implements rollout and lane gating.
nodejs/src/ingestion/common/flag-evaluations/flag-evaluations-service.test.ts	Tests configuration and gating behavior.
nodejs/src/ingestion/common/flag-evaluations/flag-evaluation-row.ts	Maps events to the destination row shape.
nodejs/src/ingestion/common/flag-evaluations/flag-evaluation-row.test.ts	Tests row mapping and columns.
nodejs/src/ingestion/common/feature-flag-called-dedup/feature-flag-called-dedup-service.ts	Aligns deduplication with realtime lanes.
nodejs/src/common/outputs/index.ts	Defines the new output identifier.
nodejs/src/common/config/kafka-topics.ts	Defines the flag-evaluations topic.

Suppressed comments (3)

nodejs/src/ingestion/common/flag-evaluations/flag-evaluation-row.ts:39

• This mapping calls serializeEvent, while emit-event-step immediately serializes the same ProcessedEvent again. Every enabled flag event therefore JSON-stringifies both properties blobs and repeats the sanitization/allocation work, which can materially raise CPU and GC cost for a high-volume or wildcard rollout. Reuse one serialized RawKafkaEvent for both outputs instead of serializing the event twice.

    const serialized = serializeEvent(event)

nodejs/src/ingestion/common/steps/event-processing/create-event.ts:102

• The existing create-event test only checks that created_at is populated; no automated path asserts that the stamp generated here is exactly the one emitted to both outputs. Since this change affects every event and claims timestamp parity, add an assertion in the dual-write E2E that flagEvaluations[0].created_at equals events[0].created_at.

        created_at: DateTime.utc(),

nodejs/tests/ingestion/flag-evaluations-e2e.test.ts:107

• The key cross-output invariant for the new created_at stamp is not asserted here: the test compares person_id but never compares the two timestamps. A regression that restores per-serialization timestamping would pass the current unit tests and this E2E. Assert that flagEvaluations[0].created_at equals events[0].created_at.

            // person_id must match what the events table stamped for the same
            // event — the parity per-flag uniques depend on.
            expect(flagEvaluations[0].person_id).not.toBe('00000000-0000-0000-0000-000000000000')
            expect(flagEvaluations[0].person_id).toBe(events[0].person_id)

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[greptile-apps]

_{Reviews (2): Last reviewed commit: "fix(flags): stop oversized flag_evaluati..." | Re-trigger Greptile}

[greptile-apps]

_{Reviews (3): Last reviewed commit: "fix(flags): stop oversized flag_evaluati..." | Re-trigger Greptile}

[eli-r-ph]

thanks for the updates! 🚀

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-08-25 18:58 UTC	Run
prod-us	✅ Deployed	2026-08-25 19:22 UTC	Run
prod-eu	✅ Deployed	2026-08-25 19:27 UTC	Run