diff --git a/acceptance/private/local-feature-flag-evaluator.feature b/acceptance/private/local-feature-flag-evaluator.feature index aa6390b..8d585ab 100644 --- a/acceptance/private/local-feature-flag-evaluator.feature +++ b/acceptance/private/local-feature-flag-evaluator.feature @@ -963,14 +963,28 @@ Feature: Local Feature Flag Evaluator Then the group result should be Examples: - | group_type | children | result | - | "OR" | true and inconclusive | true | - | "OR" | false and inconclusive | inconclusive | - | "AND" | false and inconclusive | false | - | "AND" | true and inconclusive | inconclusive | - | "property" | true and true | true | - | "OR" | no children | false | - | "AND" | no children | true | + | group_type | children | result | + | "OR" | true and missing-property inconclusive | true | + | "OR" | false and missing-property inconclusive | inconclusive | + | "AND" | false and missing-property inconclusive | false | + | "AND" | true and missing-property inconclusive | inconclusive | + | "property" | true and true | true | + | "OR" | no children | false | + | "AND" | no children | true | + + @server + Scenario Outline: Reached unsupported cohort groups stop evaluation before later siblings + Given the SDK is initialized with token "test-token" and local evaluation enabled + And a dynamic cohort group combines child results with group type + When the group is evaluated locally + Then the group result should be + + Examples: + | group_type | children | result | + | "OR" | unsupported empty XOR group then true | requires server evaluation | + | "AND" | unsupported empty XOR group then false | requires server evaluation | + | "OR" | true then unsupported empty XOR group | true | + | "AND" | false then unsupported empty XOR group | false | @server Scenario: Negation does not turn an inconclusive cohort leaf into a match diff --git a/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/.openspec.yaml b/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/.openspec.yaml new file mode 100644 index 0000000..2b638f8 --- /dev/null +++ b/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-11-10 diff --git a/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/design.md b/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/design.md new file mode 100644 index 0000000..fbdd344 --- /dev/null +++ b/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/design.md @@ -0,0 +1,31 @@ +## Context + +The canonical SDK contract currently treats all unsupported cohort children as composable uncertainty. The Rust feature-flags backend instead propagates a parsing error when a nested group with an unsupported type is reached. The operator's evaluation loop uses `?`, so later siblings are not evaluated after that error. + +Evidence: [group parsing](https://github.com/PostHog/posthog/blob/22a27e765f3e6a126a2cae814f0aba5b288424bd/rust/feature-flags/src/cohorts/cohort_models.rs#L265-L309) and [group evaluation](https://github.com/PostHog/posthog/blob/22a27e765f3e6a126a2cae814f0aba5b288424bd/rust/feature-flags/src/cohorts/cohort_operations.rs#L310-L450). + +## Goals / Non-Goals + +**Goals:** Define backend-compatible traversal for a reached group with an unsupported type, distinguish hard fallback from missing-context uncertainty, and retain both child orders as contract examples. + +**Non-Goals:** Change the Rust backend, unknown leaf-type semantics, missing-context composition, release-condition order, public APIs, cohort cycle handling, or harness suite selection. + +## Decisions + +- Treat a reached unsupported group as a hard local-evaluation boundary. A local SDK cannot claim a match from a later sibling when the backend would stop before that sibling. +- Retain short-circuiting before the invalid group is reached. Do not pre-scan all descendants for unsupported types. +- Keep missing property context composable. Local callers can provide partial context even when the backend has complete stored properties. +- Translate the backend error into the SDK's existing server-fallback signal, not an exception escaping into host application code. +- Apply and archive the delta with OpenSpec. Update the descriptive acceptance matrix without introducing new harness step phrases or executable opt-in tags. + +The alternative was to change the Rust backend to continue after an unsupported group. That was rejected for this change because backend parity is the agreed contract. + +## Risks / Trade-offs + +- Unsupported definitions can require an extra remote request when the invalid group precedes a decisive sibling. The spec now makes that ordering visible instead of claiming order independence for errors. +- The local harness checkout at `15011d6868ee73e11333860b6392d35298b92535` has neither the v2 Gherkin catalogue nor its runner or step registry. Acceptance edits remain descriptive and unselected. Syntax validation is separate from executable harness coverage. +- PostHog/posthog-go#440's sibling-composition commit differs from this correction and must be reverted in a new signed commit. PostHog/posthog-js#5649 already preserves the hard fallback boundary. + +## Migration Plan + +Publish this spec correction and link it from PostHog/posthog-go#440. Revert Go's sibling-composition follow-up without rewriting history. No normal supported definitions, dependencies, CI selectors, or public signatures change. SDKs must change code if they suppress a reached unsupported-group failure with later siblings or continue visiting siblings after a result is already decisive. Python at `1f185fd20a2b57d16486170f50cf436e01a54506` has the latter behavior and needs a separate SDK follow-up; no Python code is changed here. diff --git a/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/proposal.md b/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/proposal.md new file mode 100644 index 0000000..a761b62 --- /dev/null +++ b/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/proposal.md @@ -0,0 +1,27 @@ +## Why + +The cross-SDK review of [posthog-go#431](https://github.com/PostHog/posthog-go/pull/431) and the follow-up on [posthog-go#440](https://github.com/PostHog/posthog-go/pull/440#discussion_r4526467057) exposed a mismatch between the SDK contract and the Rust feature-flags backend. The current contract allows a later decisive sibling to override an unsupported cohort group, while the backend stops when that group is reached. + +## What Changes + +- Make backend parity the contract for a reached cohort group with an unsupported type, such as `XOR`: stop local evaluation and require server evaluation without visiting later siblings. +- Preserve definition-order short-circuiting. A decisive earlier sibling can finish the group before the unsupported child is reached. +- Keep three-valued composition for missing property context, valid AND/OR/property groups, empty groups, and negation safety unchanged. +- Clarify the distinction in the canonical spec and the existing descriptive acceptance matrix. +- Archive and sync this correction in the same PR. + +## Capabilities + +### New Capabilities + +None. + +### Modified Capabilities + +- `local-feature-flag-evaluator`: distinguish a reached unsupported group from a composable, context-inconclusive child. + +## Impact + +This is a canonical specification correction, not a change to the Rust SDK. The authority is the [Rust feature-flags backend](https://github.com/PostHog/posthog/blob/22a27e765f3e6a126a2cae814f0aba5b288424bd/rust/feature-flags/src/cohorts/cohort_operations.rs#L310-L450). Normal supported definitions and public APIs are unchanged. Malformed definitions may require a remote request where the previous specification allowed a local answer. + +PostHog/posthog-go#440 will revert its sibling-composition follow-up to restore immediate server fallback. PostHog/posthog-js#5649 already uses the hard server-evaluation boundary. Python's [group evaluator at `1f185fd`](https://github.com/PostHog/posthog-python/blob/1f185fd20a2b57d16486170f50cf436e01a54506/posthog/feature_flags.py#L1442-L1513) still evaluates later siblings after recording a decisive result, so it needs a separate short-circuit follow-up for the decisive-first cases. This is a source comparison, not a Python runtime conformance test. No dependency updates, harness changes, or new acceptance-suite opt-in tags are included. diff --git a/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/specs/local-feature-flag-evaluator/spec.md b/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/specs/local-feature-flag-evaluator/spec.md new file mode 100644 index 0000000..c890c2a --- /dev/null +++ b/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/specs/local-feature-flag-evaluator/spec.md @@ -0,0 +1,84 @@ +## MODIFIED Requirements + +### Requirement: Specialized cohort matching preserves scope and inconclusive state + +A criterion with property type `cohort` SHALL use `in` by default when its operator is omitted. `in` SHALL match a definitive referenced membership result; `not_in` SHALL negate only a definitive result. A static cohort with cached local membership is therefore conclusive. A static cohort definition without membership data, a missing dynamic definition, a dependency cycle, or unavailable required context SHALL be inconclusive and MUST NOT be represented as membership false merely to evaluate `not_in`. + +Dynamic cohort groups SHALL recursively apply three-valued composition, except for the hard fallback required when an unsupported group type is reached. `OR` returns true when any child is true, false when every child is false, and inconclusive otherwise. `AND` and its legacy wire alias `property` return false when any child is false, true when every child is true, and inconclusive otherwise. An empty `OR` is false; an empty `AND` or `property` group is true. + +A leaf's `negation` field SHALL invert only a definitive match/no-match result. It MUST NOT convert an inconclusive or invalid leaf into a match. Unsupported or malformed cohort criteria SHALL remain unresolved locally and MUST NOT throw into application code. An unsupported group type SHALL use the hard fallback described below rather than a composable inconclusive child result. The cohort-only `between` and `not_between` operators SHALL use the numeric-range semantics defined above. + +Children SHALL be evaluated in definition order. A supported group SHALL short-circuit when a child decides its result. If evaluation reaches a child group with an unsupported type, such as `XOR`, the evaluator SHALL stop local evaluation immediately and require server evaluation. The failure SHALL propagate through enclosing groups without evaluating later siblings, even if one would otherwise decide the result. A decisive earlier sibling can finish the group before the unsupported group is reached. This hard fallback does not change three-valued composition for missing property context or permit negation to convert the failure into a match. + +Implementations SHALL detect cohort cycles and bound recursion. A definition beyond an implementation's safe recursion bound MAY become inconclusive, but MUST NOT be silently truncated or crash application code. + +#### Scenario Outline: Cohort membership operators compare definitive membership (@server) +- **GIVEN** a local condition references cohort `42` with operator +- **AND** cohort `42` has a definitive local membership result +- **WHEN** the condition is evaluated locally +- **THEN** the criterion match result should be + + | operator | membership | matches | + | omitted | true | true | + | `in` | false | false | + | `not_in` | true | false | + | `not_in` | false | true | + +#### Scenario Outline: Unavailable membership remains inconclusive for both polarities (@server) +- **GIVEN** a local condition references cohort `42` with operator +- **AND** cohort membership is unavailable because +- **WHEN** the condition is evaluated locally +- **THEN** the criterion should be inconclusive +- **AND** remote evaluation should remain eligible + + | operator | reason | + | `in` | the static cohort has no cached membership | + | `not_in` | the static cohort has no cached membership | + | `not_in` | the dynamic cohort definition is missing | + | `not_in` | the cohort dependency graph contains a cycle | + +#### Scenario Outline: Dynamic cohort groups use three-valued composition (@server) +- **GIVEN** a dynamic cohort group combines child results with group type +- **AND** any inconclusive child is missing a required person property locally +- **WHEN** the group is evaluated locally +- **THEN** the group result should be + + | group-type | children | result | + | `OR` | true and inconclusive | true | + | `OR` | false and inconclusive | inconclusive | + | `AND` | false and inconclusive | false | + | `AND` | true and inconclusive | inconclusive | + | `property` | true and true | true | + | `OR` | no children | false | + | `AND` | no children | true | + +#### Scenario Outline: Reached unsupported cohort groups stop evaluation before later siblings (@server) +- **GIVEN** a supported dynamic cohort group of type has children in definition order +- **AND** the unsupported child is a nested group with type `XOR` and an empty `values` list +- **WHEN** the cohort is evaluated locally +- **THEN** the local result should be +- **AND** requiring server evaluation should leave remote fallback eligible without throwing into application code + + | group-type | children | result | + | `OR` | unsupported then true | requires server evaluation | + | `AND` | unsupported then false | requires server evaluation | + | `OR` | true then unsupported | true | + | `AND` | false then unsupported | false | + +#### Scenario: Negation does not turn an inconclusive cohort leaf into a match (@server) +- **GIVEN** a dynamic cohort contains a negated leaf whose required person property is unavailable locally +- **WHEN** a feature flag depending on that cohort is evaluated locally +- **THEN** the cohort criterion should be inconclusive +- **AND** the negation should not turn it into a match +- **AND** remote evaluation should remain eligible + +#### Scenario Outline: Malformed cohort structures are inconclusive (@server) +- **GIVEN** a referenced dynamic cohort contains +- **AND** the outer membership operator is `not_in` +- **WHEN** the cohort condition is evaluated locally +- **THEN** the cohort condition should be inconclusive +- **AND** `not_in` should not turn the failure into a match + + | malformation | + | a known leaf type missing its required key | + | unknown group type `XOR` | diff --git a/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/tasks.md b/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/tasks.md new file mode 100644 index 0000000..4d50f72 --- /dev/null +++ b/openspec/changes/archive/2026-11-10-align-unsupported-cohort-groups-with-backend/tasks.md @@ -0,0 +1,10 @@ +## 1. Establish the contract + +- [x] 1.1 Compare Go #440 with the Rust feature-flags backend and confirm backend parity as the chosen contract. +- [x] 1.2 Write the complete modified cohort requirement and record the missing-context exception and both child orders. + +## 2. Apply and validate + +- [x] 2.1 Update the existing descriptive acceptance matrix without adding new step phrases or executable opt-in tags. +- [x] 2.2 Validate the complete delta and parse the acceptance feature and four ordering examples. +- [x] 2.3 Review the targeted contract change against the Rust backend's parsing and evaluation paths. diff --git a/openspec/specs/local-feature-flag-evaluator/spec.md b/openspec/specs/local-feature-flag-evaluator/spec.md index 4c52a1f..4d3c420 100644 --- a/openspec/specs/local-feature-flag-evaluator/spec.md +++ b/openspec/specs/local-feature-flag-evaluator/spec.md @@ -916,9 +916,11 @@ A prerelease property SHALL satisfy a range only when the range contains a prere A criterion with property type `cohort` SHALL use `in` by default when its operator is omitted. `in` SHALL match a definitive referenced membership result; `not_in` SHALL negate only a definitive result. A static cohort with cached local membership is therefore conclusive. A static cohort definition without membership data, a missing dynamic definition, a dependency cycle, or unavailable required context SHALL be inconclusive and MUST NOT be represented as membership false merely to evaluate `not_in`. -Dynamic cohort groups SHALL recursively apply three-valued composition. `OR` returns true when any child is true, false when every child is false, and inconclusive otherwise. `AND` and its legacy wire alias `property` return false when any child is false, true when every child is true, and inconclusive otherwise. An empty `OR` is false; an empty `AND` or `property` group is true. +Dynamic cohort groups SHALL recursively apply three-valued composition, except for the hard fallback required when an unsupported group type is reached. `OR` returns true when any child is true, false when every child is false, and inconclusive otherwise. `AND` and its legacy wire alias `property` return false when any child is false, true when every child is true, and inconclusive otherwise. An empty `OR` is false; an empty `AND` or `property` group is true. -A leaf's `negation` field SHALL invert only a definitive match/no-match result. It MUST NOT convert an inconclusive or invalid leaf into a match. Unsupported or malformed cohort criteria SHALL be inconclusive and MUST NOT throw into application code. The cohort-only `between` and `not_between` operators SHALL use the numeric-range semantics defined above. +A leaf's `negation` field SHALL invert only a definitive match/no-match result. It MUST NOT convert an inconclusive or invalid leaf into a match. Unsupported or malformed cohort criteria SHALL remain unresolved locally and MUST NOT throw into application code. An unsupported group type SHALL use the hard fallback described below rather than a composable inconclusive child result. The cohort-only `between` and `not_between` operators SHALL use the numeric-range semantics defined above. + +Children SHALL be evaluated in definition order. A supported group SHALL short-circuit when a child decides its result. If evaluation reaches a child group with an unsupported type, such as `XOR`, the evaluator SHALL stop local evaluation immediately and require server evaluation. The failure SHALL propagate through enclosing groups without evaluating later siblings, even if one would otherwise decide the result. A decisive earlier sibling can finish the group before the unsupported group is reached. This hard fallback does not change three-valued composition for missing property context or permit negation to convert the failure into a match. Implementations SHALL detect cohort cycles and bound recursion. A definition beyond an implementation's safe recursion bound MAY become inconclusive, but MUST NOT be silently truncated or crash application code. @@ -949,6 +951,7 @@ Implementations SHALL detect cohort cycles and bound recursion. A definition bey #### Scenario Outline: Dynamic cohort groups use three-valued composition (@server) - **GIVEN** a dynamic cohort group combines child results with group type +- **AND** any inconclusive child is missing a required person property locally - **WHEN** the group is evaluated locally - **THEN** the group result should be @@ -961,6 +964,19 @@ Implementations SHALL detect cohort cycles and bound recursion. A definition bey | `OR` | no children | false | | `AND` | no children | true | +#### Scenario Outline: Reached unsupported cohort groups stop evaluation before later siblings (@server) +- **GIVEN** a supported dynamic cohort group of type has children in definition order +- **AND** the unsupported child is a nested group with type `XOR` and an empty `values` list +- **WHEN** the cohort is evaluated locally +- **THEN** the local result should be +- **AND** requiring server evaluation should leave remote fallback eligible without throwing into application code + + | group-type | children | result | + | `OR` | unsupported then true | requires server evaluation | + | `AND` | unsupported then false | requires server evaluation | + | `OR` | true then unsupported | true | + | `AND` | false then unsupported | false | + #### Scenario: Negation does not turn an inconclusive cohort leaf into a match (@server) - **GIVEN** a dynamic cohort contains a negated leaf whose required person property is unavailable locally - **WHEN** a feature flag depending on that cohort is evaluated locally