Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .sampo/changesets/called-context-tolerant-rule-pair.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
pypi/posthog-sdk-test-harness: minor
---

Accept a config version 2 `$feature_flag_called` that drops a broken rule pair, following PostHog/sdk-specs#162 (contract package 9.0.0).

- `$feature_flag_rule_id` and `$feature_flag_rule_type` are sent together or not at all, and only with a rule type the reason code allows. An SDK that reads a record with a missing rule ID, a missing rule type or a disallowed rule type sends neither field, and keeps the config version and reason code.
- `schemas/feature_flag_called_context.schema.json` now accepts a config version 2 call with no rule, variant or experiment attribution for `targeting_match`, `rollout_miss`, `experiment_paused` and `holdout`, as it already did for a malformed `experiment_split`. A half pair and a disallowed rule type, such as `targeting_match` with `experiment`, are still rejected, and complete pairs are unchanged.
- A config version 2 call whose `$feature_flag_reason_code` is missing or is not a registered reason code is now accepted with no rule, variant or experiment attribution. It keeps config version 2 and the raw code, if any. Registered codes still go through their own branches, so their behaviour is unchanged.
- `bin/update-feature-flag-rules-v2-presence-schemas.py` generates these branches: one per reason that names a rule in `rules/response_presence.json`, and one for a missing or unregistered reason code. Every other branch now requires its reason code, so the branches do not overlap. The malformed-split branch is now generated the same way, with the same shape.
- The meta-tests check a call without attribution for each of those reasons, for the local `flag_disabled`, and for a missing or unregistered reason code, the complete call for every matrix row, each half pair and disallowed rule type, and each attribution field added to a call without attribution.
- A failed record reads as disabled, so the legacy payload accessor yields no payload for it, even when the record carries `metadata.payload`. Reader cases gain an optional `legacy_payload` expectation, the decoded JSON value the accessor returns or `null` for none. `readers.v1_at_v4` declares its decoded payload, and `readers.v1_failed`, a failed config version 1 record that still carries a payload, pins it as `null`. A meta-test checks every declared `legacy_payload` and requires one on any failed reader case that carries a payload.
- The set of accepted calls grows, so this is a major contract version. Consumers that vendor the called-context schema need to copy this release.
70 changes: 62 additions & 8 deletions bin/update-feature-flag-rules-v2-presence-schemas.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
#!/usr/bin/env python3
"""Regenerate the terminal-reason branches of the presence and called-context schemas from the presence matrix."""
"""Regenerate the terminal-reason branches of the presence and called-context schemas from the presence matrix.

The called context also gets one dropped-attribution branch per reason that names a rule, and one branch for a
reason code that is missing or has no branch of its own.
"""

import json
from pathlib import Path
Expand Down Expand Up @@ -47,22 +51,72 @@ def called_branch(row: dict[str, Any]) -> dict[str, Any]:
properties = {"$feature_flag_reason_code": {"const": row["reason"]}}
if row["rule_type"]:
properties["$feature_flag_rule_type"] = {"const": row["rule_type"]}
result: dict[str, Any] = {"properties": properties}
if row["required"]:
result["required"] = [event_property(field) for field in row["required"]]
# The reason code is required, so a call without one only matches the unregistered-reason branch.
result: dict[str, Any] = {
"properties": properties,
"required": ["$feature_flag_reason_code", *[event_property(field) for field in row["required"]]],
}
result["not"] = {"anyOf": [{"required": [event_property(field)]} for field in row["forbidden"]]}
return result


def no_attribution(rows: list[dict[str, Any]]) -> dict[str, Any]:
"""Forbid every field that any row requires or forbids: the rule, variant and experiment attribution."""
fields = dict.fromkeys(field for row in rows for field in [*row["required"], *row["forbidden"]])
return {"anyOf": [{"required": [event_property(field)]} for field in fields]}


def dropped_attribution_branches(rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""A call for a record that breaks its reason's rows keeps the reason and omits all attribution.

One branch per reason that names a rule. A half rule pair, a rule type the reason does not allow, or a
malformed split is dropped whole, so the call carries none of the fields that any row requires or forbids.
"""
reasons = dict.fromkeys(row["reason"] for row in rows if row["rule_type"])
return [
{
"properties": {"$feature_flag_reason_code": {"const": reason}},
"required": ["$feature_flag_reason_code"],
"not": no_attribution(rows),
}
for reason in reasons
]


def unregistered_reason_branch(rows: list[dict[str, Any]], reasons: list[str]) -> dict[str, Any]:
"""A call whose reason code is missing, or is not one of the reasons with a branch, carries no attribution.

The other branches require a reason code from ``reasons``, so this branch never overlaps them.
"""
return {
"properties": {"$feature_flag_reason_code": {"not": {"enum": reasons}}},
"not": no_attribution(rows),
}


def called_branches(rows: list[dict[str, Any]], reasons: list[str]) -> list[dict[str, Any]]:
"""Every generated called-context branch: one per matrix row, the dropped-attribution diagnostics, then the
branch for a missing or unregistered reason code. ``reasons`` lists every reason with a branch of its own."""
return (
[called_branch(row) for row in rows]
+ dropped_attribution_branches(rows)
+ [unregistered_reason_branch(rows, reasons)]
)


def regenerate(rows: list[dict[str, Any]], presence: dict[str, Any], called: dict[str, Any]) -> None:
"""Rewrite the generated branches in place; hand-written branches carry a description and stay behind them."""
presence["allOf"][1]["properties"]["flags"]["additionalProperties"]["then"]["oneOf"] = [
response_branch(row) for row in rows
]
branches = called["allOf"][0]["then"]["oneOf"]
called["allOf"][0]["then"]["oneOf"] = [called_branch(row) for row in rows] + [
branch for branch in branches if "description" in branch
]
hand_written = [branch for branch in called["allOf"][0]["then"]["oneOf"] if "description" in branch]
reasons = list(
dict.fromkeys(
[row["reason"] for row in rows]
+ [branch["properties"]["$feature_flag_reason_code"]["const"] for branch in hand_written]
)
)
called["allOf"][0]["then"]["oneOf"] = called_branches(rows, reasons) + hand_written


def _load(path: Path) -> Any:
Expand Down
6 changes: 3 additions & 3 deletions contracts/feature_flag_rules_v2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,12 +129,12 @@ Run `python -m pytest tests/test_feature_flag_rules_v2_*.py` from the repository
- `schemas/definitions_v2.schema.json` describes the mixed-version definitions feed, including its required cohort map and optional boolean `minimal_flag_called_events`. Unsupported config versions or v2 semantic fields require per-flag remote fallback in auto mode and an unsupported result in strict local mode.
- `schemas/flags_response_v4.schema.json` is the exact producer schema. It forbids `experiment_id` on v2 records; only version 1 records carry one. `schemas/flags_response_v4_presence.schema.json` adds the terminal-reason presence matrix from `rules/response_presence.json`. The companion also requires the failure envelope flag whenever a record failed and rejects a version 1 `experiment_id` alongside `has_experiment: false`; `has_experiment: true` alone does not imply an unambiguous experiment ID. Validate against both, then check map-key equality and recursive seed absence. A schema-only success does not establish a valid producer response.
- `schemas/management_warning.schema.json` describes a diagnostic with a required warning code from the registry's five `warning_codes` and optional presentation `detail` and field `attr`. `schemas/management_error.schema.json` preserves the management validation-error envelope (`type`, `code`, `detail`, `attr`); error codes remain endpoint-owned. These artifacts do not install a warning response envelope or an acknowledgement protocol.
- `schemas/feature_flag_called_context.schema.json` and `schemas/experiment_exposure_properties.schema.json` describe **closed property projections**, not entire capture envelopes. The generic call supports v1 diagnostics and v2 terminal context. A config version 2 call carries the reason code in `$feature_flag_reason_code` and never `$feature_flag_reason`, the reason description. Version 1 calls keep `$feature_flag_reason` with any value; the schema declares it only so that a version 2 call that carries it is rejected. A malformed-split diagnostic may retain config version 2 and the split reason while omitting the whole rule and variant attribution tuple; a partly populated tuple is rejected. A direct exposure carries exactly `$feature_flag`, `$feature_flag_response`, `$feature_flag_variant`, `$feature_flag_rule_id` and `locally_evaluated` in the projection. The closed schema rejects a config version, rule type, reason code, source marker or experiment ID on it, and a `not` clause rejects `$feature_flag_forced_variant`, which the schema declares with its producer shape. Scan the full event for assignment seeds before extracting the projection; normal capture identity, library, group and optional transport properties remain outside it.
- `fixtures/wire/` contains definitions entry and producer response fixtures, and separate tolerant-reader expectations. Fixture sets are added together with the first consumer that runs them; the definitions feed, management diagnostic and event schemas have no fixtures and are specifications. The meta-tests check the event schemas against projections they build: an exposure without each required property, an exposure with each forbidden property, and a version 2 call with `$feature_flag_reason` for every presence-matrix row. A case deep-copies a named template, removes existing members by JSON Pointer, and then sets members by JSON Pointer (the parent must exist). Invalid cases declare the validation layer, keyword and instance path; reader cases separately declare producer validity; optional message fragments disambiguate required fields. `schema`, `presence`, `semantic` and `seed` are fixture validation layers, not new wire error codes. The meta-tests generate each presence-matrix row's per-field response negatives (each missing required field, each forbidden field and a wrong condition index) from `rules/response_presence.json` and the row's template; the committed invalid response cases are the hand-made ones, plus `responses.holdout_missing_rule_id`, which keeps a presence-layer `required` failure in the committed set for consumers that port the validator.
- `schemas/feature_flag_called_context.schema.json` and `schemas/experiment_exposure_properties.schema.json` describe **closed property projections**, not entire capture envelopes. The generic call supports v1 diagnostics and v2 terminal context. A config version 2 call carries the reason code in `$feature_flag_reason_code`, when the record has one, and never `$feature_flag_reason`, the reason description. Version 1 calls keep `$feature_flag_reason` with any value; the schema declares it only so that a version 2 call that carries it is rejected. `$feature_flag_rule_id` and `$feature_flag_rule_type` are sent together or not at all, and only with a rule type the reason allows. A tolerant reader drops the whole attribution of a record that breaks this, as it does for a malformed split: for `targeting_match`, `experiment_split`, `rollout_miss`, `experiment_paused` and `holdout`, a diagnostic call may retain config version 2 and the reason code while omitting every rule, variant and experiment attribution field. A partly populated tuple, such as a rule ID without its rule type, and a rule type the reason does not allow, such as `targeting_match` with `experiment`, are rejected. A config version 2 call whose reason code is missing, or is not one of the registry's `reason_codes`, keeps config version 2 and the raw code, if any, and carries no rule, variant or experiment attribution. A registered code is checked only against its own reason's branches, so a `targeting_match` call with a half pair is still rejected. A direct exposure carries exactly `$feature_flag`, `$feature_flag_response`, `$feature_flag_variant`, `$feature_flag_rule_id` and `locally_evaluated` in the projection. The closed schema rejects a config version, rule type, reason code, source marker or experiment ID on it, and a `not` clause rejects `$feature_flag_forced_variant`, which the schema declares with its producer shape. Scan the full event for assignment seeds before extracting the projection; normal capture identity, library, group and optional transport properties remain outside it.
- `fixtures/wire/` contains definitions entry and producer response fixtures, and separate tolerant-reader expectations. Fixture sets are added together with the first consumer that runs them; the definitions feed, management diagnostic and event schemas have no fixtures and are specifications. The meta-tests check the event schemas against projections they build: an exposure without each required property, an exposure with each forbidden property, a version 2 call with `$feature_flag_reason` for every presence-matrix row, the complete call for every row, a call without attribution for every reason that names a rule, for the local `flag_disabled`, for an unregistered reason code and for a missing one, each half rule pair and disallowed rule type, and each attribution field added to a call without attribution. A case deep-copies a named template, removes existing members by JSON Pointer, and then sets members by JSON Pointer (the parent must exist). Invalid cases declare the validation layer, keyword and instance path; reader cases separately declare producer validity; optional message fragments disambiguate required fields. `schema`, `presence`, `semantic` and `seed` are fixture validation layers, not new wire error codes. The meta-tests generate each presence-matrix row's per-field response negatives (each missing required field, each forbidden field and a wrong condition index) from `rules/response_presence.json` and the row's template; the committed invalid response cases are the hand-made ones, plus `responses.holdout_missing_rule_id`, which keeps a presence-layer `required` failure in the committed set for consumers that port the validator.

Definitions retain canonical rule and holdout assignment seeds because local evaluation needs them. Evaluation responses and events prohibit assignment seeds recursively, including inside arrays, diagnostic conditions and opaque transport objects. The response's `flags` map keys are user-defined flag names, so names such as `seed` are allowed; every record and the rest of the envelope are still scanned. Likewise, the immediate keys of an event's `$groups`, `$set` and `$set_once` maps are customer-chosen group type and person property names; their values are still scanned.

Producer and reader contracts serve different purposes. Missing split context is invalid producer output, but a reader preserves a usable value and emits no experiment exposure. Known fields with wrong JSON types produce a per-record parse error and leave siblings usable. Unknown fields are ignored. A record without `value` uses the old-server `variant ?? enabled` interpretation, config version 1, and no carried-over v2 context. Reader expectations never loosen producer validation. Reader fixtures are contract data; they do not run SDK execution or activate capabilities.
Producer and reader contracts serve different purposes. Missing split context is invalid producer output, but a reader preserves a usable value and emits no experiment exposure. Known fields with wrong JSON types produce a per-record parse error and leave siblings usable. Unknown fields are ignored. A record without `value` uses the old-server `variant ?? enabled` interpretation, config version 1, and no carried-over v2 context. A failed record reads as disabled, so the legacy payload accessor yields no payload for it, even when the record carries `metadata.payload`. A reader case pins this with an optional `legacy_payload` expectation: `null` for no payload, otherwise the decoded JSON value the accessor returns, not the encoded string on the wire. `readers.v1_at_v4` declares `{"columns": 2}` for the wire payload `"{\"columns\":2}"`. Every reader case whose failed record carries a payload declares it, as `readers.v1_failed` does with `null`. Reader expectations never loosen producer validation. Reader fixtures are contract data; they do not run SDK execution or activate capabilities.

The first event release permits only booleans and non-empty strings in the response property. Reserved number/object response-schema cases do not enable writers or event emission. Equal-valued arms remain separate analytical identities through their variant keys. A holdout, pause, rollout miss, default, missing or failed result cannot produce a direct exposure.

Expand Down
8 changes: 6 additions & 2 deletions contracts/feature_flag_rules_v2/fixtures/wire/readers.json
Original file line number Diff line number Diff line change
Expand Up @@ -992,7 +992,10 @@
"value": "compact",
"config_version": 1,
"experiment_exposure": false,
"error_code": null
"error_code": null,
"legacy_payload": {
"columns": 2
}
},
"producer_expected": "valid"
},
Expand All @@ -1018,7 +1021,8 @@
"value": true,
"config_version": 1,
"experiment_exposure": false,
"error_code": "GENERAL"
"error_code": "GENERAL",
"legacy_payload": null
},
"producer_expected": "valid"
},
Expand Down
2 changes: 1 addition & 1 deletion contracts/feature_flag_rules_v2/manifest.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"contract": {
"name": "feature_flag_rules_v2",
"version": "8.0.0",
"version": "9.0.0",
"config_version": 2
},
"artifacts": [
Expand Down
Loading
Loading