Skip to content

fix: omit failed remote feature flag values - #251

Merged
marandaneto merged 3 commits into
mainfrom
fix/spec211-failed-flags
Nov 13, 2026
Merged

marandaneto merged 3 commits into
mainfrom
fix/spec211-failed-flags

Conversation

@marandaneto

@marandaneto marandaneto commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

💡 Motivation and Context

A detailed /flags record marked failed: true means evaluation failed, not that the flag is disabled. Implements the direct failed-record behavior in PostHog/sdk-specs#211:

  • Return the existing no-result outcome for failed direct reads and omit failed values/payloads from snapshots and enrichment.
  • Preserve successful same-context local true/false/variant values and payloads when remote fallback fails.
  • Track evaluation_failed with a nil response, preserve valid diagnostic metadata, suppression/dedupe and independent response errors; failed reads do not seed capture context.
  • Malformed optional nested metadata is omitted when building failed diagnostics.
  • Review follow-up: only traverse the flags envelope for diagnostics when it is a map. Non-map envelopes retain rich getter {:ok, nil} and check's existing UnexpectedResponseError, without a tracking crash or event. Global parser behavior and missing outer-key errors are unchanged.

No new public API/dependency, blanket rescue, global parser rejection, manual version bump or duplicate release intent; existing patch changeset remains.

Snapshot scope: existing snapshot missing/unresolved classification is deliberately preserved, including its treatment of a returned failed key. Broader snapshot unresolved/telemetry policy is not changed by this envelope-guard fix, and no full snapshot-contract conformance claim is made.

💚 How did you test it?

Current local result at 37bee902d2f609710a6e2932f21b6a32dfdc10d6, Elixir 1.20.4 / OTP29, existing locked dependencies:

  • Existing nested malformed-metadata baseline: 1 passed.
  • New Mox malformed-envelope regression first reproduced FunctionClauseError in Access.get/3 at the new diagnostic lookup for "flags": "bad"; suppressed getter control passed before that crash. The same mix test test/posthog/feature_flags_test.exs:769 passes after the map guard.
  • Regression covers string, nil, boolean, number, empty/nonempty list envelopes; default rich getter/check results and exact error body/message are preserved; no events captured.
  • Feature flag, real local fallback and snapshot neighbors: 182 passed, including existing nested metadata/global-error-bit cases, diagnostics, suppression/dedupe and local value/payload pairing.
  • Full mix test: 974 passed, 18 live integration tests excluded.
  • Full format, Credo (75 files, no issues), compile-with-warnings-as-errors, public API check and mix hex.build passed; package output was written outside the worktree. git diff --check passed.
  • Exact committed branch reviewed by mandatory autoreview against origin/main (401a4a02742058d0c221986502924b75926ac5e2): clean, no accepted/actionable findings. Exact-target Authoring audit/deslop completed.

No dependency installs or lockfile changes. Supported Elixir/OTP matrix, hosted CI, live-service integration, benchmarks or shared cross-SDK harness were not rerun locally. Previous CI/compliance reports are historical and not a current green-CI/full-conformance claim.

📝 Checklist

  • New follow-up commit signed and locally verified.
  • Full branch diff reviewed; mandatory local autoreview clean.
  • Regression reproduced before production changes; existing meaningful controls retained.
  • Public API unchanged; no documentation change needed for an internal traversal guard.
  • Existing patch changeset retained; no release/manual version changes.

🤖 Agent context

Autonomy: Human-driven (agent-assisted). Human DRI: @marandaneto. Pi implemented the approved fixes and ran the tests, source/test audit, and isolated autoreview listed above. Human review is required before merge. No shareable session link is available.

@marandaneto marandaneto self-assigned this Nov 12, 2026
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

posthog-elixir Compliance Report

Date: 2026-11-12T13:44:11.951004+00:00
Duration: 108645ms

⚠️ Some Tests Failed

40/47 tests passed, 7 failed


Capture Tests

⚠️ 29/30 tests passed, 1 failed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 609ms
Format Validation.Event Has Uuid ✅ 610ms
Format Validation.Event Has Lib Properties ✅ 610ms
Format Validation.Distinct Id Is String ✅ 609ms
Format Validation.Token Is Present ✅ 610ms
Format Validation.Custom Properties Preserved ✅ 609ms
Format Validation.Event Has Timestamp ✅ 610ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc ❌ 610ms
Retry Behavior.Retries On 503 ✅ 5615ms
Retry Behavior.Does Not Retry On 400 ✅ 2613ms
Retry Behavior.Does Not Retry On 401 ✅ 2612ms
Retry Behavior.Respects Retry After Header ✅ 5611ms
Retry Behavior.Implements Backoff ✅ 15624ms
Retry Behavior.Retries On 500 ✅ 5616ms
Retry Behavior.Retries On 502 ✅ 5617ms
Retry Behavior.Retries On 504 ✅ 5616ms
Retry Behavior.Max Retries Respected ✅ 15612ms
Deduplication.Generates Unique Uuids ✅ 621ms
Deduplication.Preserves Uuid On Retry ✅ 5616ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 10622ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 5617ms
Deduplication.No Duplicate Events In Batch ✅ 615ms
Deduplication.Different Events Have Different Uuids ✅ 613ms
Compression.Sends Gzip When Enabled ✅ 610ms
Batch Format.Uses Proper Batch Structure ✅ 611ms
Batch Format.Flush With No Events Sends Nothing ✅ 606ms
Batch Format.Multiple Events Batched Together ✅ 615ms
Error Handling.Does Not Retry On 403 ✅ 2613ms
Error Handling.Does Not Retry On 413 ✅ 2612ms
Error Handling.Retries On 408 ✅ 5614ms

Failures

format_validation.non_utc_event_timestamp_is_converted_to_utc

Event 0 field 'timestamp' instant '2026-11-12T13:42:27.890222Z' != expected '2025-01-02T03:04:05Z'

Feature_Flags Tests

⚠️ 11/17 tests passed, 6 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 8ms
Request Payload.Flags Request Uses V2 Query Param ✅ 7ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 7ms
Request Payload.Flags Request Omits Authorization Header ✅ 7ms
Request Payload.Token In Flags Body Matches Init ✅ 7ms
Request Payload.Groups Round Trip ✅ 7ms
Request Payload.Groups Default To Empty Object ❌ 7ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 6ms
Request Payload.Disable Geoip Omitted Defaults To False ❌ 8ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 6ms
Request Lifecycle.No Flags Request On Init Alone ✅ 4ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 608ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 12ms
Request Lifecycle.Mock Response Value Is Returned To Caller ❌ 7ms
Retry Behavior.Retries Flags On 502 ❌ 310ms
Retry Behavior.Retries Flags On 504 ❌ 311ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ❌ 610ms

Failures

request_payload.groups_default_to_empty_object

Field 'groups' not found in /flags request body at path 'groups'. Available keys: ['distinct_id', 'api_key', 'flag_keys_to_evaluate']

request_payload.disable_geoip_omitted_defaults_to_false

Field 'geoip_disable' not found in /flags request body at path 'geoip_disable'. Available keys: ['distinct_id', 'api_key', 'flag_keys_to_evaluate']

request_lifecycle.mock_response_value_is_returned_to_caller

Last action result missing field 'value'. Keys: ['error', 'success']

retry_behavior.retries_flags_on_502

Last action result missing field 'value'. Keys: ['error', 'success']

retry_behavior.retries_flags_on_504

Last action result missing field 'value'. Keys: ['error', 'success']

side_effect_events.get_feature_flag_captures_feature_flag_called_event

Expected 1 events with name '$feature_flag_called', got 0

@marandaneto
marandaneto marked this pull request as ready for review November 12, 2026 12:13
@marandaneto
marandaneto requested a review from a team as a code owner November 12, 2026 12:13
@veria-ai

veria-ai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

PR overview

No security issues were found in this pull request.

Reviews

@posthog posthog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed 663d0a0: 1 Should fix.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

Comment thread lib/posthog/feature_flags.ex Resolved

@ioannisj ioannisj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 An AI agent wrote this on behalf of @ioannisj.

No findings from me. I went through the failed-record handling on direct reads, snapshots and the $feature_flag_called event, plus the non-map flags guard, and it lines up with the failed-flag spec.

@marandaneto
marandaneto merged commit e0b9bb6 into main Nov 13, 2026
30 of 31 checks passed
@marandaneto
marandaneto deleted the fix/spec211-failed-flags branch November 13, 2026 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants