Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-08-27
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
## Context

The Rust `/flags` service already has two deliberate normalization rules in `property_matching.rs`: `exact`/`is_not` use Unicode lowercase, while `icontains`, prefix, and suffix operators use ASCII lowercase. SDKs currently use a mix of ASCII-only comparison, Unicode lowercase, Unicode casefold/equivalence APIs, and type-strict equality. Those APIs agree for ordinary Latin ASCII but diverge for values such as `Ä`, `ß`, and Greek sigma.

The backend also compares canonical `serde_json::Value::to_string()` representations. JSON strings contribute their unquoted contents; other JSON values use compact JSON with sorted object keys and `serde_json` number formatting. Some host runtimes irreversibly collapse distinct JSON number kinds (notably `323` and `323.0`) before local evaluation receives them.

Before either stringification or ordinary array membership, the backend applies a boolean-like gate to the complete `exact`/`is_not` filter value. Its recursive `all()` predicates classify all-boolean-like arrays as aggregate booleans and classify an empty array as truthy by vacuous truth. This behavior is surprising, but this change documents the released `/flags` comparator rather than changing it. [PostHog/posthog#90694](https://github.com/PostHog/posthog/pull/90694) is a draft proposal to replace that behavior later.

## Goals / Non-Goals

**Goals:**

- Make the operator-specific backend behavior explicit and testable.
- Distinguish Unicode lowercase from both ASCII lowercase and Unicode casefold/equivalence.
- Define the backend boolean gate, conditional equality-array semantics, and canonical `serde_json` stringification model.
- State how implementations behave when their host value model has already discarded lexical information.

**Non-Goals:**

- Change `/flags` behavior or analytics/HogQL property-filter semantics.
- Require SDKs to retain raw JSON source text or replace their JSON parser.
- Define new property-filter operators or change missing-property fallback behavior.

## Decisions

### Specify normalization per operator family

After the boolean gate, `exact` and `is_not` normalize each stringified candidate with Rust-compatible full Unicode lowercase and compare the resulting strings. Full lowercase includes contextual final sigma and multi-code-point mappings such as dotted capital I. Search operators normalize only ASCII `A` through `Z`. Generic "ignore case", simple lowercase, casefold, locale-sensitive, accent-insensitive, and collation APIs are not interchangeable with these transforms and therefore do not satisfy the contract unless they demonstrably produce the same results.

Alternative considered: require ASCII folding everywhere. This was rejected because it would make SDK exact matching diverge from the existing backend. Requiring Unicode folding everywhere was rejected for the same reason on contains/prefix/suffix operators and would undo the backend's deliberate performance choice.

### Preserve the backend boolean gate before equality arrays

For `exact`, the backend first classifies the complete filter as boolean-like. Booleans and `true`/`false` strings are boolean-like; an array is boolean-like when every element is recursively boolean-like. Boolean-like filters compare aggregate truthiness, including the backend's vacuously truthy empty array. Only a filter array containing at least one non-boolean-like element reaches ANY membership. `is_not` negates the complete result. Other composite JSON values continue through canonical JSON stringification; this does not redefine the separate `icontains_multi` operators.

### Define stringification at the evaluator boundary

The canonical model follows the backend: strings remain unquoted, while non-string JSON values use `serde_json::Value::to_string()`. Arrays are compact, object keys are recursively sorted, and finite numbers use `serde_json`'s canonical exponent and plain-decimal cutovers. Consequently an integer `323` and a floating-point `323.0` stringify differently when the input representation preserves that distinction.

If a host runtime has already collapsed those values before the evaluator runs, the SDK must not guess the discarded numeric kind. A dedicated inconclusive result is preferred when it can safely preserve remote fallback; otherwise the SDK uses one deterministic representation and documents the limitation. Conformance tests use typed fixtures only where the runtime can represent the distinction.

## Risks / Trade-offs

- [Unicode versions can differ between language runtimes] → Use word-final sigma and dotted-I discriminator vectors that test Rust-compatible full lowercase, while treating unavoidable Unicode database version differences as runtime constraints.
- [A host number model may not preserve `323.0`] → Prefer inconclusive fallback where available; otherwise document the limitation and prohibit invented formatting rather than requiring raw-token parsing.
- [The released boolean gate has counterintuitive empty-array behavior] → Specify the measured behavior for current SDK parity and track the proposed cleanup separately in [PostHog/posthog#90694](https://github.com/PostHog/posthog/pull/90694).
- [Existing SDK behavior changes for rare non-ASCII comparisons] → Ship focused regression tests and compare against the `/flags` matcher vectors.
- [Analytics filters have different semantics] → Scope every requirement explicitly to local feature-flag evaluation and the `/flags` service.
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
## Why

Local feature-flag evaluators currently disagree with the `/flags` service on case-insensitive property matching, so the same flag and properties can produce different results locally and remotely. The existing contract specifies ASCII-only prefix/suffix matching but leaves `icontains`, `exact`/`is_not`, and value stringification incomplete, allowing implementations to choose incompatible Unicode comparison APIs.

## What Changes

- Define the backend-compatible, operator-specific case normalization contract: ASCII lowercase for the contains/prefix/suffix families and Unicode lowercase for `exact`/`is_not`.
- Define the released backend's boolean-like gate before stringification or filter-array membership, including aggregate boolean arrays and vacuously truthy empty arrays.
- Define conditional ANY/NONE semantics for non-boolean-like `exact`/`is_not` filter arrays.
- Define canonical `serde_json` stringification, including compact composites, sorted object keys, finite-number spellings, and runtimes that cannot preserve distinct numeric kinds.
- Add Unicode discriminator scenarios for Rust-compatible full lowercase, including contextual final sigma and dotted-I expansion.
- Preserve existing missing-property inconclusive behavior.

## Capabilities

### New Capabilities

None.

### Modified Capabilities

- `local-feature-flag-evaluator`: Specify complete string/equality property-filter matching semantics compatible with the Rust flags service.

## Impact

This changes the normative local-evaluation contract used by server-side SDK implementations and their conformance tests. It does not change the public feature-flag API or the released flags-service backend; SDK matchers must align with the existing backend behavior described in PostHog/posthog#78019. [PostHog/posthog#90694](https://github.com/PostHog/posthog/pull/90694) separately proposes replacing the surprising boolean and empty-array behavior.
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
## ADDED Requirements

### Requirement: Backend-compatible property-filter stringification

The local evaluator SHALL stringify operands before applying the `exact`, `is_not`, `icontains`, `not_icontains`, `starts_with`,
`not_starts_with`, `ends_with`, or `not_ends_with` operator, using the same canonical representation as the flags service. A JSON string SHALL contribute its unquoted contents. Every other JSON value SHALL use the canonical output of Rust `serde_json::Value::to_string()`.

The canonical representation SHALL use lowercase JSON booleans and null, compact arrays without optional whitespace, and compact objects whose keys are recursively sorted lexicographically. Numbers SHALL use `serde_json`'s canonical finite-number spelling. This includes preserving the integer-versus-floating-point distinction when the runtime retains it, preserving negative zero for a floating-point value, and using the same plain-decimal and exponent cutovers as the flags service. The boolean-filter and filter-array precedence specified below SHALL run before treating a filter array as one string value.

When a host runtime irreversibly collapses distinct JSON values before local evaluation receives them, such as integer `323` and floating-point `323.0`, the SDK MUST NOT invent the discarded numeric kind. An evaluator with a dedicated inconclusive result SHOULD use it for an ambiguous `exact` or `is_not` comparison so remote evaluation remains eligible. Otherwise the SDK SHALL use one deterministic, locale-independent representation and document the limitation.

#### Scenario: Integer and preserved floating-point values remain distinct

- **WHEN** an `exact` filter with string value `"323"` is evaluated against JSON integer property `323`
- **THEN** the local evaluation result should be true
- **WHEN** an `exact` filter with string value `"323.0"` is evaluated against a distinctly represented JSON floating-point property `323.0`
- **THEN** the local evaluation result should be true
- **AND** an `exact` filter with string value `"323"` should not match that floating-point property

#### Scenario: Composite values use compact sorted JSON

- **WHEN** an `exact` filter with string value `"[1,2]"` is evaluated against array property `[1, 2]`
- **THEN** the local evaluation result should be true
- **WHEN** an `exact` filter with string value `"{\"a\":2,\"b\":1}"` is evaluated against object property `{"b": 1, "a": 2}`
- **THEN** the local evaluation result should be true

#### Scenario Outline: Numbers use serde_json canonical spellings

- **WHEN** an `exact` filter with string value <string-value> is evaluated against numeric property <property>
- **THEN** the local evaluation result should be true

| string-value | property |
| `"-0.0"` | `-0.0` |
| `"1e-7"` | `0.0000001` |
| `"1e+16"` | `10000000000000000.0` |
| `"0.00001"` | `0.00001` |
| `"0.000099"` | `0.000099` |

#### Scenario: A collapsed host number is not reconstructed

- **GIVEN** the SDK runtime represents source JSON values `323` and `323.0` as the same host value
- **WHEN** that host value reaches local property matching
- **THEN** the evaluator should not guess whether the source value was an integer or floating-point number
- **AND** it should return inconclusive when its result model supports safe remote fallback
- **OR** it should use one documented deterministic representation when no inconclusive result exists

### Requirement: Backend boolean gate and Unicode-lowercase exact property filters

The local evaluator SHALL implement `exact` and `is_not` with the flags service's ordered algorithm. It SHALL first classify the complete filter value as boolean-like. JSON booleans and case-insensitive strings `"true"` and `"false"` are boolean-like. A JSON array is boolean-like when every element is recursively boolean-like. An empty array is therefore boolean-like because every element of an empty iterator satisfies the predicate.

When the filter is boolean-like, the evaluator SHALL compare its aggregate truthiness with the property's aggregate truthiness before applying ordinary filter-array membership or stringification. A boolean is truthy according to its value. A string is truthy only when it case-insensitively equals `"true"`. An array is truthy when every element is recursively truthy, including an empty array. Every other JSON value is falsey. The property need not itself be boolean-like, so a false-like filter currently matches arbitrary falsey values such as `"banana"`, `0`, null, `{}`, and `""`.

When the filter is not boolean-like and is an array, the evaluator SHALL independently stringify and full-Unicode-lowercase each array element. `exact` SHALL match when ANY element equals the stringified, full-Unicode-lowercased property value. When the filter is neither boolean-like nor an array, the evaluator SHALL stringify and full-Unicode-lowercase the filter and property before comparing them. `is_not` SHALL be the logical negation of the complete exact result.

Full Unicode lowercase SHALL reproduce Rust `str::to_lowercase`, including context-sensitive and multi-code-point mappings. Implementations MUST NOT substitute ASCII-only lowercase, Unicode casefold/equivalence, accent removal, locale-sensitive comparison, simple Unicode lowercase, or a generic case-insensitive collation whose results differ from Rust lowercase-then-compare.

When the required property key is absent from caller-supplied local evaluation context, both operators SHALL remain inconclusive and eligible for remote fallback.

#### Scenario: A false-like filter uses aggregate truthiness

- **WHEN** an `exact` filter with value `false`, `"false"`, or `["false"]` is evaluated against property `"banana"`, `0`, null, `{}`, or `""`
- **THEN** the local evaluation result should be true
- **AND** the corresponding `is_not` filter should return false

#### Scenario: A boolean-like array takes precedence over ANY membership

- **WHEN** an `exact` filter with value `["true", "false"]` is evaluated against property `"true"`
- **THEN** the local evaluation result should be false
- **WHEN** the same filter is evaluated against property `"pro"`
- **THEN** the local evaluation result should be true

#### Scenario: An empty filter array is vacuously truthy

- **WHEN** an `exact` filter with value `[]` is evaluated against property `true`, `"true"`, `[]`, or `[true]`
- **THEN** the local evaluation result should be true
- **WHEN** the same filter is evaluated against property `"us"`, null, false, or `1`
- **THEN** the local evaluation result should be false
- **AND** `is_not` should return the logical inverse in every case

#### Scenario: A non-boolean-like array uses case-insensitive ANY membership

- **WHEN** an `exact` filter with value `["FREE", "PRO"]` is evaluated against property `"pro"`
- **THEN** the local evaluation result should be true
- **AND** the corresponding `is_not` filter should return false

#### Scenario: Unicode lowercase matches non-ASCII case variants

- **WHEN** an `exact` filter with value `"ä"` is evaluated against property `"Ä"`
- **THEN** the local evaluation result should be true

#### Scenario: Unicode casefold expansion is not exact lowercase equality

- **WHEN** an `exact` filter with value `"ss"` is evaluated against property `"ß"`
- **THEN** the local evaluation result should be false

#### Scenario: A single sigma does not equal final sigma

- **WHEN** an `exact` filter with value `"ς"` is evaluated against property `"Σ"`
- **THEN** the local evaluation result should be false

#### Scenario: Full lowercase applies contextual final sigma

- **WHEN** an `exact` filter with value `"οδος"` is evaluated against property `"ΟΔΟΣ"`
- **THEN** the local evaluation result should be true
- **WHEN** an `exact` filter with value `"οδοσ"` is evaluated against property `"ΟΔΟΣ"`
- **THEN** the local evaluation result should be false

#### Scenario: Full lowercase expands dotted capital I

- **WHEN** an `exact` filter with value `"i̇"` is evaluated against property `"İ"`
- **THEN** the local evaluation result should be true
- **WHEN** an `exact` filter with value `"i"` is evaluated against property `"İ"`
- **THEN** the local evaluation result should be false

#### Scenario: Missing equality property is inconclusive

- **WHEN** an `exact` or `is_not` filter is evaluated without its required property in the supplied context
- **THEN** local evaluation should be inconclusive
- **AND** remote evaluation should remain eligible when enabled

### Requirement: ASCII-lowercase string search property filters

The local evaluator SHALL implement `icontains`, `not_icontains`, `starts_with`, `not_starts_with`,
`ends_with`, and `not_ends_with` by stringifying both operands and lowercasing
only ASCII characters `A` through `Z`. It SHALL then apply the corresponding substring, prefix, or
suffix check. Each `not_*` operator SHALL be the logical negation of its positive operator for a
present, supported property value. These operators MUST NOT use Unicode lowercase, Unicode
casefold/equivalence, locale-sensitive comparison, or accent removal.

When the required property key is absent from caller-supplied local evaluation context, these
operators SHALL remain inconclusive and eligible for remote fallback.

#### Scenario: ASCII case variants match all positive search operators

- **WHEN** filters use ASCII case variants between their values and the supplied property
- **THEN** `icontains`, `starts_with`, and `ends_with` should perform case-insensitive matches
- **AND** their corresponding `not_*` operators should return the logical inverse

#### Scenario: Non-ASCII case variants do not match the ASCII family

- **WHEN** an `icontains` or `starts_with` filter with value `"ä"` is evaluated against property `"Äbc"`
- **THEN** the local evaluation result should be false
- **WHEN** an `ends_with` filter with value `"ä"` is evaluated against property `"bcÄ"`
- **THEN** the local evaluation result should be false
- **AND** the corresponding `not_*` operators should return true

#### Scenario: Missing string-search property is inconclusive

- **WHEN** any string-search operator is evaluated without its required property in the supplied context
- **THEN** local evaluation should be inconclusive
- **AND** remote evaluation should remain eligible when enabled

## REMOVED Requirements

### Requirement: String prefix/suffix property filter operators

**Reason**: The narrower requirement is superseded by the complete backend-compatible stringification, equality, and ASCII string-search requirements above.

**Migration**: Implementations that already satisfy prefix/suffix behavior retain it under `ASCII-lowercase string search property filters` and add the specified `icontains`, equality, and stringification semantics.
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
## 1. Define the matcher contract

- [x] 1.1 Specify canonical `serde_json` value stringification and host numeric-representation limits
- [x] 1.2 Specify the released boolean gate and conditional `exact`/`is_not` array membership semantics
- [x] 1.3 Specify Rust-compatible full Unicode lowercase for equality and ASCII lowercase for string search
- [x] 1.4 Add boolean, empty-array, composite, numeric, Unicode, and missing-property discriminator scenarios

## 2. Publish the canonical specification

- [x] 2.1 Validate the OpenSpec change in strict mode
- [x] 2.2 Archive the completed change and align the Behavior pointer with the replacement requirements
- [x] 2.3 Validate all canonical specifications and archived artifacts in strict mode
Loading
Loading