Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 22 additions & 8 deletions acceptance/private/local-feature-flag-evaluator.feature
Original file line number Diff line number Diff line change
Expand Up @@ -963,14 +963,28 @@ Feature: Local Feature Flag Evaluator
Then the group result should be <result>

Examples:
| group_type | children | result |
| "OR" | true and inconclusive | true |
| "OR" | false and inconclusive | inconclusive |
| "AND" | false and inconclusive | false |
| "AND" | true and inconclusive | inconclusive |
| "property" | true and true | true |
| "OR" | no children | false |
| "AND" | no children | true |
| group_type | children | result |
| "OR" | true and missing-property inconclusive | true |
| "OR" | false and missing-property inconclusive | inconclusive |
| "AND" | false and missing-property inconclusive | false |
| "AND" | true and missing-property inconclusive | inconclusive |
| "property" | true and true | true |
| "OR" | no children | false |
| "AND" | no children | true |

@server
Scenario Outline: Reached unsupported cohort groups stop evaluation before later siblings
Given the SDK is initialized with token "test-token" and local evaluation enabled
And a dynamic cohort group combines child results <children> with group type <group_type>
When the group is evaluated locally
Then the group result should be <result>

Examples:
| group_type | children | result |
| "OR" | unsupported empty XOR group then true | requires server evaluation |
| "AND" | unsupported empty XOR group then false | requires server evaluation |
| "OR" | true then unsupported empty XOR group | true |
| "AND" | false then unsupported empty XOR group | false |

@server
Scenario: Negation does not turn an inconclusive cohort leaf into a match
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-11-10
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
## Context

The canonical SDK contract currently treats all unsupported cohort children as composable uncertainty. The Rust feature-flags backend instead propagates a parsing error when a nested group with an unsupported type is reached. The operator's evaluation loop uses `?`, so later siblings are not evaluated after that error.

Evidence: [group parsing](https://github.com/PostHog/posthog/blob/22a27e765f3e6a126a2cae814f0aba5b288424bd/rust/feature-flags/src/cohorts/cohort_models.rs#L265-L309) and [group evaluation](https://github.com/PostHog/posthog/blob/22a27e765f3e6a126a2cae814f0aba5b288424bd/rust/feature-flags/src/cohorts/cohort_operations.rs#L310-L450).

## Goals / Non-Goals

**Goals:** Define backend-compatible traversal for a reached group with an unsupported type, distinguish hard fallback from missing-context uncertainty, and retain both child orders as contract examples.

**Non-Goals:** Change the Rust backend, unknown leaf-type semantics, missing-context composition, release-condition order, public APIs, cohort cycle handling, or harness suite selection.

## Decisions

- Treat a reached unsupported group as a hard local-evaluation boundary. A local SDK cannot claim a match from a later sibling when the backend would stop before that sibling.
- Retain short-circuiting before the invalid group is reached. Do not pre-scan all descendants for unsupported types.
- Keep missing property context composable. Local callers can provide partial context even when the backend has complete stored properties.
- Translate the backend error into the SDK's existing server-fallback signal, not an exception escaping into host application code.
- Apply and archive the delta with OpenSpec. Update the descriptive acceptance matrix without introducing new harness step phrases or executable opt-in tags.

The alternative was to change the Rust backend to continue after an unsupported group. That was rejected for this change because backend parity is the agreed contract.

## Risks / Trade-offs

- Unsupported definitions can require an extra remote request when the invalid group precedes a decisive sibling. The spec now makes that ordering visible instead of claiming order independence for errors.
- The local harness checkout at `15011d6868ee73e11333860b6392d35298b92535` has neither the v2 Gherkin catalogue nor its runner or step registry. Acceptance edits remain descriptive and unselected. Syntax validation is separate from executable harness coverage.
- PostHog/posthog-go#440's sibling-composition commit differs from this correction and must be reverted in a new signed commit. PostHog/posthog-js#5649 already preserves the hard fallback boundary.

## Migration Plan

Publish this spec correction and link it from PostHog/posthog-go#440. Revert Go's sibling-composition follow-up without rewriting history. No normal supported definitions, dependencies, CI selectors, or public signatures change. SDKs must change code if they suppress a reached unsupported-group failure with later siblings or continue visiting siblings after a result is already decisive. Python at `1f185fd20a2b57d16486170f50cf436e01a54506` has the latter behavior and needs a separate SDK follow-up; no Python code is changed here.
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
## Why

The cross-SDK review of [posthog-go#431](https://github.com/PostHog/posthog-go/pull/431) and the follow-up on [posthog-go#440](https://github.com/PostHog/posthog-go/pull/440#discussion_r4526467057) exposed a mismatch between the SDK contract and the Rust feature-flags backend. The current contract allows a later decisive sibling to override an unsupported cohort group, while the backend stops when that group is reached.

## What Changes

- Make backend parity the contract for a reached cohort group with an unsupported type, such as `XOR`: stop local evaluation and require server evaluation without visiting later siblings.
- Preserve definition-order short-circuiting. A decisive earlier sibling can finish the group before the unsupported child is reached.
- Keep three-valued composition for missing property context, valid AND/OR/property groups, empty groups, and negation safety unchanged.
- Clarify the distinction in the canonical spec and the existing descriptive acceptance matrix.
- Archive and sync this correction in the same PR.

## Capabilities

### New Capabilities

None.

### Modified Capabilities

- `local-feature-flag-evaluator`: distinguish a reached unsupported group from a composable, context-inconclusive child.

## Impact

This is a canonical specification correction, not a change to the Rust SDK. The authority is the [Rust feature-flags backend](https://github.com/PostHog/posthog/blob/22a27e765f3e6a126a2cae814f0aba5b288424bd/rust/feature-flags/src/cohorts/cohort_operations.rs#L310-L450). Normal supported definitions and public APIs are unchanged. Malformed definitions may require a remote request where the previous specification allowed a local answer.

PostHog/posthog-go#440 will revert its sibling-composition follow-up to restore immediate server fallback. PostHog/posthog-js#5649 already uses the hard server-evaluation boundary. Python's [group evaluator at `1f185fd`](https://github.com/PostHog/posthog-python/blob/1f185fd20a2b57d16486170f50cf436e01a54506/posthog/feature_flags.py#L1442-L1513) still evaluates later siblings after recording a decisive result, so it needs a separate short-circuit follow-up for the decisive-first cases. This is a source comparison, not a Python runtime conformance test. No dependency updates, harness changes, or new acceptance-suite opt-in tags are included.
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
## MODIFIED Requirements

### Requirement: Specialized cohort matching preserves scope and inconclusive state

A criterion with property type `cohort` SHALL use `in` by default when its operator is omitted. `in` SHALL match a definitive referenced membership result; `not_in` SHALL negate only a definitive result. A static cohort with cached local membership is therefore conclusive. A static cohort definition without membership data, a missing dynamic definition, a dependency cycle, or unavailable required context SHALL be inconclusive and MUST NOT be represented as membership false merely to evaluate `not_in`.

Dynamic cohort groups SHALL recursively apply three-valued composition, except for the hard fallback required when an unsupported group type is reached. `OR` returns true when any child is true, false when every child is false, and inconclusive otherwise. `AND` and its legacy wire alias `property` return false when any child is false, true when every child is true, and inconclusive otherwise. An empty `OR` is false; an empty `AND` or `property` group is true.

A leaf's `negation` field SHALL invert only a definitive match/no-match result. It MUST NOT convert an inconclusive or invalid leaf into a match. Unsupported or malformed cohort criteria SHALL remain unresolved locally and MUST NOT throw into application code. An unsupported group type SHALL use the hard fallback described below rather than a composable inconclusive child result. The cohort-only `between` and `not_between` operators SHALL use the numeric-range semantics defined above.

Children SHALL be evaluated in definition order. A supported group SHALL short-circuit when a child decides its result. If evaluation reaches a child group with an unsupported type, such as `XOR`, the evaluator SHALL stop local evaluation immediately and require server evaluation. The failure SHALL propagate through enclosing groups without evaluating later siblings, even if one would otherwise decide the result. A decisive earlier sibling can finish the group before the unsupported group is reached. This hard fallback does not change three-valued composition for missing property context or permit negation to convert the failure into a match.

Implementations SHALL detect cohort cycles and bound recursion. A definition beyond an implementation's safe recursion bound MAY become inconclusive, but MUST NOT be silently truncated or crash application code.

#### Scenario Outline: Cohort membership operators compare definitive membership (@server)
- **GIVEN** a local condition references cohort `42` with operator <operator>
- **AND** cohort `42` has a definitive local membership result <membership>
- **WHEN** the condition is evaluated locally
- **THEN** the criterion match result should be <matches>

| operator | membership | matches |
| omitted | true | true |
| `in` | false | false |
| `not_in` | true | false |
| `not_in` | false | true |

#### Scenario Outline: Unavailable membership remains inconclusive for both polarities (@server)
- **GIVEN** a local condition references cohort `42` with operator <operator>
- **AND** cohort membership is unavailable because <reason>
- **WHEN** the condition is evaluated locally
- **THEN** the criterion should be inconclusive
- **AND** remote evaluation should remain eligible

| operator | reason |
| `in` | the static cohort has no cached membership |
| `not_in` | the static cohort has no cached membership |
| `not_in` | the dynamic cohort definition is missing |
| `not_in` | the cohort dependency graph contains a cycle |

#### Scenario Outline: Dynamic cohort groups use three-valued composition (@server)
- **GIVEN** a dynamic cohort group combines child results <children> with group type <group-type>
- **AND** any inconclusive child is missing a required person property locally
- **WHEN** the group is evaluated locally
- **THEN** the group result should be <result>

| group-type | children | result |
| `OR` | true and inconclusive | true |
| `OR` | false and inconclusive | inconclusive |
| `AND` | false and inconclusive | false |
| `AND` | true and inconclusive | inconclusive |
| `property` | true and true | true |
| `OR` | no children | false |
| `AND` | no children | true |

#### Scenario Outline: Reached unsupported cohort groups stop evaluation before later siblings (@server)
- **GIVEN** a supported dynamic cohort group of type <group-type> has children <children> in definition order
- **AND** the unsupported child is a nested group with type `XOR` and an empty `values` list
- **WHEN** the cohort is evaluated locally
- **THEN** the local result should be <result>
- **AND** requiring server evaluation should leave remote fallback eligible without throwing into application code

| group-type | children | result |
| `OR` | unsupported then true | requires server evaluation |
| `AND` | unsupported then false | requires server evaluation |
| `OR` | true then unsupported | true |
| `AND` | false then unsupported | false |

#### Scenario: Negation does not turn an inconclusive cohort leaf into a match (@server)
- **GIVEN** a dynamic cohort contains a negated leaf whose required person property is unavailable locally
- **WHEN** a feature flag depending on that cohort is evaluated locally
- **THEN** the cohort criterion should be inconclusive
- **AND** the negation should not turn it into a match
- **AND** remote evaluation should remain eligible

#### Scenario Outline: Malformed cohort structures are inconclusive (@server)
- **GIVEN** a referenced dynamic cohort contains <malformation>
- **AND** the outer membership operator is `not_in`
- **WHEN** the cohort condition is evaluated locally
- **THEN** the cohort condition should be inconclusive
- **AND** `not_in` should not turn the failure into a match

| malformation |
| a known leaf type missing its required key |
| unknown group type `XOR` |
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
## 1. Establish the contract

- [x] 1.1 Compare Go #440 with the Rust feature-flags backend and confirm backend parity as the chosen contract.
- [x] 1.2 Write the complete modified cohort requirement and record the missing-context exception and both child orders.

## 2. Apply and validate

- [x] 2.1 Update the existing descriptive acceptance matrix without adding new step phrases or executable opt-in tags.
- [x] 2.2 Validate the complete delta and parse the acceptance feature and four ordering examples.
- [x] 2.3 Review the targeted contract change against the Rust backend's parsing and evaluation paths.
20 changes: 18 additions & 2 deletions openspec/specs/local-feature-flag-evaluator/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -916,9 +916,11 @@ A prerelease property SHALL satisfy a range only when the range contains a prere

A criterion with property type `cohort` SHALL use `in` by default when its operator is omitted. `in` SHALL match a definitive referenced membership result; `not_in` SHALL negate only a definitive result. A static cohort with cached local membership is therefore conclusive. A static cohort definition without membership data, a missing dynamic definition, a dependency cycle, or unavailable required context SHALL be inconclusive and MUST NOT be represented as membership false merely to evaluate `not_in`.

Dynamic cohort groups SHALL recursively apply three-valued composition. `OR` returns true when any child is true, false when every child is false, and inconclusive otherwise. `AND` and its legacy wire alias `property` return false when any child is false, true when every child is true, and inconclusive otherwise. An empty `OR` is false; an empty `AND` or `property` group is true.
Dynamic cohort groups SHALL recursively apply three-valued composition, except for the hard fallback required when an unsupported group type is reached. `OR` returns true when any child is true, false when every child is false, and inconclusive otherwise. `AND` and its legacy wire alias `property` return false when any child is false, true when every child is true, and inconclusive otherwise. An empty `OR` is false; an empty `AND` or `property` group is true.

A leaf's `negation` field SHALL invert only a definitive match/no-match result. It MUST NOT convert an inconclusive or invalid leaf into a match. Unsupported or malformed cohort criteria SHALL be inconclusive and MUST NOT throw into application code. The cohort-only `between` and `not_between` operators SHALL use the numeric-range semantics defined above.
A leaf's `negation` field SHALL invert only a definitive match/no-match result. It MUST NOT convert an inconclusive or invalid leaf into a match. Unsupported or malformed cohort criteria SHALL remain unresolved locally and MUST NOT throw into application code. An unsupported group type SHALL use the hard fallback described below rather than a composable inconclusive child result. The cohort-only `between` and `not_between` operators SHALL use the numeric-range semantics defined above.

Children SHALL be evaluated in definition order. A supported group SHALL short-circuit when a child decides its result. If evaluation reaches a child group with an unsupported type, such as `XOR`, the evaluator SHALL stop local evaluation immediately and require server evaluation. The failure SHALL propagate through enclosing groups without evaluating later siblings, even if one would otherwise decide the result. A decisive earlier sibling can finish the group before the unsupported group is reached. This hard fallback does not change three-valued composition for missing property context or permit negation to convert the failure into a match.

Implementations SHALL detect cohort cycles and bound recursion. A definition beyond an implementation's safe recursion bound MAY become inconclusive, but MUST NOT be silently truncated or crash application code.

Expand Down Expand Up @@ -949,6 +951,7 @@ Implementations SHALL detect cohort cycles and bound recursion. A definition bey

#### Scenario Outline: Dynamic cohort groups use three-valued composition (@server)
- **GIVEN** a dynamic cohort group combines child results <children> with group type <group-type>
- **AND** any inconclusive child is missing a required person property locally
- **WHEN** the group is evaluated locally
- **THEN** the group result should be <result>

Expand All @@ -961,6 +964,19 @@ Implementations SHALL detect cohort cycles and bound recursion. A definition bey
| `OR` | no children | false |
| `AND` | no children | true |

#### Scenario Outline: Reached unsupported cohort groups stop evaluation before later siblings (@server)
- **GIVEN** a supported dynamic cohort group of type <group-type> has children <children> in definition order
- **AND** the unsupported child is a nested group with type `XOR` and an empty `values` list
- **WHEN** the cohort is evaluated locally
- **THEN** the local result should be <result>
- **AND** requiring server evaluation should leave remote fallback eligible without throwing into application code

| group-type | children | result |
| `OR` | unsupported then true | requires server evaluation |
| `AND` | unsupported then false | requires server evaluation |
| `OR` | true then unsupported | true |
| `AND` | false then unsupported | false |

#### Scenario: Negation does not turn an inconclusive cohort leaf into a match (@server)
- **GIVEN** a dynamic cohort contains a negated leaf whose required person property is unavailable locally
- **WHEN** a feature flag depending on that cohort is evaluated locally
Expand Down
Loading