Repository navigation
feat(node): report unresolved local flags on evaluateFlags snapshots - #5234
Merged
Merged
Conversation
Implements PostHog/sdk-specs#85: the snapshot exposes `unresolvedFlags` (key -> reason), reads of those keys report `local_evaluation_inconclusive` instead of `flag_missing`, and the experience continuity warning logs once per flag definition. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: 6abf34b0-7fbe-430e-bd75-fbf8684ef581
Contributor
Contributor
|
[Medium risk] Adds unresolved flag tracking to local evaluation snapshots. The PR appears safe to merge, with a non-blocking fix needed for warnings after flags are disabled and re-enabled. Reviews (1) · Last reviewed commit: "feat(node): report unresolved local flag..." · Reviewed by Greptile |
4 of 5 tasks
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Contributor
|
Size Change: +24.2 kB (+0.1%) Total Size: 24.9 MB 📦 View Changed
ℹ️ View Unchanged
|
Contributor
Node SDK compliance v2Commit: b6547ab Capture v0 — acceptance✅ 5 passed / ❌ 26 non-passing / Drill-down: run artifacts → ❌ acceptance/public/capture-exception.feature::Server exception capture delivers a native handled exception with caller properties — unsupported_bindingRequired public operation unavailable: /capture_exception Code: Failed step: acceptance/public/capture-exception.feature:9 capture exception is called with JSON arguments: ❌ acceptance/public/capture-exception.feature::Server exception capture delivers a native handled exception without caller properties — unsupported_bindingRequired public operation unavailable: /capture_exception Code: Failed step: acceptance/public/capture-exception.feature:30 capture exception is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot reads share one evaluation and deduplicate exposure — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:763 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot projections distinguish disabled variant and missing flags — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:788 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot evaluation without reads delivers no exposure — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:813 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot payload reads are silent and retain JSON semantics — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:838 evaluate flags and read is called with JSON arguments: ❌ server_snapshot_payload_false — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:864 evaluate flags and read is called with JSON arguments: ❌ server_snapshot_payload_zero — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:864 evaluate flags and read is called with JSON arguments: ❌ server_snapshot_payload_null — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:864 evaluate flags and read is called with JSON arguments: ❌ server_snapshot_payload_string — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:864 evaluate flags and read is called with JSON arguments: ❌ server_snapshot_payload_json_looking_string — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:864 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot independent requests retain identity and group context — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:897 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot forwards explicit evaluation context — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:930 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot request-time key scope and enumeration stay silent — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:955 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot empty request-time keys make no evaluation request — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:980 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot explicit-key filters preserve the original and stay silent — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:1005 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot filtered values retain decisions and share exposure dedupe — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:1030 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot accessed-key filtering before value use is empty and silent — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:1055 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot accessed-key filtering selects used flags without parent leakage — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:1080 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot supported caller defaults affect only missing enablement — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:1105 evaluate flags and read is called with JSON arguments: Harness exit: 1; report validation exit: 1. Capture v1 — acceptance✅ 5 passed / ❌ 26 non-passing / Drill-down: run artifacts → ❌ acceptance/public/capture-exception.feature::Server exception capture delivers a native handled exception with caller properties — unsupported_bindingRequired public operation unavailable: /capture_exception Code: Failed step: acceptance/public/capture-exception.feature:9 capture exception is called with JSON arguments: ❌ acceptance/public/capture-exception.feature::Server exception capture delivers a native handled exception without caller properties — unsupported_bindingRequired public operation unavailable: /capture_exception Code: Failed step: acceptance/public/capture-exception.feature:30 capture exception is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot reads share one evaluation and deduplicate exposure — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:763 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot projections distinguish disabled variant and missing flags — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:788 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot evaluation without reads delivers no exposure — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:813 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot payload reads are silent and retain JSON semantics — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:838 evaluate flags and read is called with JSON arguments: ❌ server_snapshot_payload_false — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:864 evaluate flags and read is called with JSON arguments: ❌ server_snapshot_payload_zero — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:864 evaluate flags and read is called with JSON arguments: ❌ server_snapshot_payload_null — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:864 evaluate flags and read is called with JSON arguments: ❌ server_snapshot_payload_string — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:864 evaluate flags and read is called with JSON arguments: ❌ server_snapshot_payload_json_looking_string — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:864 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot independent requests retain identity and group context — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:897 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot forwards explicit evaluation context — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:930 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot request-time key scope and enumeration stay silent — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:955 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot empty request-time keys make no evaluation request — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:980 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot explicit-key filters preserve the original and stay silent — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:1005 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot filtered values retain decisions and share exposure dedupe — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:1030 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot accessed-key filtering before value use is empty and silent — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:1055 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot accessed-key filtering selects used flags without parent leakage — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:1080 evaluate flags and read is called with JSON arguments: ❌ acceptance/public/evaluate-flags.feature::Server snapshot supported caller defaults affect only missing enablement — unsupported_bindingRequired public operation unavailable: /evaluate_flags/read Code: Failed step: acceptance/public/evaluate-flags.feature:1105 evaluate flags and read is called with JSON arguments: Harness exit: 1; report validation exit: 1. Capture v0 — migration✅ 55 passed / ❌ 1 non-passing / Drill-down: run artifacts → ❌ migration/yaml-parity-v1/remote-flags-v1.feature::Omitted GeoIP settings default to enabled in flag requests — failed_assertionFlags field differs Code: Failed step: migration/yaml-parity-v1/remote-flags-v1.feature:134 the first flags request field "geoip_disable" should equal JSON false Operation: Field: Expected: false Actual value: true Harness exit: 1; report validation exit: 1. Capture v1 — migration✅ 120 passed / ❌ 1 non-passing / Drill-down: run artifacts → ❌ migration/yaml-parity-v1/remote-flags-v1.feature::Omitted GeoIP settings default to enabled in flag requests — failed_assertionFlags field differs Code: Failed step: migration/yaml-parity-v1/remote-flags-v1.feature:134 the first flags request field "geoip_disable" should equal JSON false Operation: Field: Expected: false Actual value: true Harness exit: 1; report validation exit: 1. |
Contributor
posthog-js Compliance ReportDate: 2026-10-22 17:21:30 UTC ✅ All Tests Passed!26/26 tests passed Capture Tests✅ 26/26 tests passed View Details
|
Contributor
|
This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![posthog[bot]](https://avatars.githubusercontent.com/in/1203049?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
PostHog Review · Standard · reviewed 72e053c: 5 Should fix.
This is the only automatic review while this pull request is a draft. Comment @posthog review to review new changes sooner. Otherwise, a review of the new changes runs when you mark it ready for review.
Contributor
Coherence4 spec problems · 0 failing chokepoints · 2 spec gaps · 0 guard failures 44 components, 7 bullets (5 invariants, 2 requirements), 4 problems Spec problems (spec --check)
Chokepoints: none declared. Spec gaps: 2 entrances with outside or unknown trust and no traced control, and 10 more held by the adoption baseline
Close one with |
marandaneto
reviewed
Oct 22, 2026
marandaneto
left a comment
marandaneto
left a comment
Member
There was a problem hiding this comment.
Automated advisory code review. Posting the remaining exact-array precedence case; the omitted-property case is already covered by the existing unresolved review thread.
marandaneto
approved these changes
Oct 22, 2026
marandaneto
left a comment
marandaneto
left a comment
Member
There was a problem hiding this comment.
still a bunch of unresolved comments from posthog bot, if they are addressed, all good
…rn about them at load
This was referenced Oct 27, 2026
Merged
Merged
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When local evaluation cannot resolve a flag (most often a flag with experience continuity) and no remote value fills the gap, the flag is absent from the
evaluateFlags()snapshot. A flag that exists but was inconclusive looks the same as a flag that does not exist, and$feature_flag_calledreportsflag_missingfor both. In local-only mode this is permanent.Why: implement the spec change in PostHog/sdk-specs#85 so callers can detect these flags and the event data separates them from missing flags.
Changes
FeatureFlagEvaluations.unresolvedFlags: read-only map of flag key to reason (experience_continuity,unsupported_definition,missing_context,unresolved_dependency). Empty ononly()/onlyAccessed()snapshots. Reading it does not count as an access.local_evaluation_inconclusiveinstead offlag_missing. Return values,keysand capture enrichment are unchanged.InconclusiveMatchError(core) andRequiresServerEvaluationcarry a reason, set at each raise site. When several causes apply, the spec's fixed precedence picks one:experience_continuity,unsupported_definition,unresolved_dependency,missing_context.unsupported_definitiontoo: static cohorts, missing or circular flag dependencies, and unknown group types.Spec notes: unexpected (non-inconclusive) local errors still read as
flag_missing; a group flag with no group key still resolves tofalse, as before.Release info Sub-libraries affected
Libraries affected
Checklist
Created with PostHog Desktop
🤖 Generated with Claude Code