Repository navigation
docs(compliance): weekly SDK compliance matrix - #148
Draft
posthog[bot] wants to merge 3 commits into
Draft
posthog[bot] wants to merge 3 commits into
posthog[bot] wants to merge 3 commits into
Conversation
turnipdabeets
changed the base branch from
docs/acceptance-step-catalogue
to
main
October 27, 2026 22:00
posthog
Bot
force-pushed
the
posthog-code/weekly-sdk-compliance-matrix
branch
from
November 2, 2026 07:22
70def4b to
45c47cc
Compare
Generated-By: PostHog Desktop Task-Id: 86b4f98b-a338-40c2-912e-9e3a9dc366fd
Deep re-audit of posthog-php, posthog-ruby and posthog-go against the 69 cross-SDK contracts, at sdk-specs 5160f2f. Selection: php and ruby were the two stalest pointers (6 weeks, outside the ~4-week window) and had the largest backfill; go is the remaining matrix SDK named in the mcp-analytics applicability table, which five of this cycle's 19 spec commits touch. All three are server SDKs, where the Emails, Exception Event Metadata, Tracing Headers and config-version-2 flag deltas land. Contract surface grew to 69 with Experiment Exposure (#162). Main findings: - Emails went from one implementation to four. All three SDKs ship it and all three pass, including #163's flattened send body, Cloud app-host resolution, and a once-per-call idempotency key. - #129's span-context bridge and #123's trace context on exception events landed in all three. posthog-go is alone in getting the detached-work scenario right, and alone in the trace-context path being reachable only through an opt-in helper in a separate module. - None of the three reads /flags?v=4, so Experiment Exposure is ➖ on all three by the spec's own applicability sentence — raised as a scope question. - posthog-ruby ships a 3,798-line MCP package the spec's applicability table records as "not yet"; scored 🟡, with the table flagged for update. - The #132 jitter requirement is now unmet on five of seven SDKs checked; posthog-go passes jitter: 0 at every ingestion call site while its own Emails client jitters. Scope: compliance/ only. Generated-By: PostHog Desktop Task-Id: 86b4f98b-a338-40c2-912e-9e3a9dc366fd
This was referenced Nov 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Weekly SDK compliance matrix
The living per-SDK compliance matrix under
compliance/, maintained by a bounded weekly audit:≤3 SDKs are deeply verified per run (spec-affected → code-changed → pending-initial-audit →
staleness-backstop), so every SDK is re-checked roughly every 4 weeks rather than all at once.
Why: the cross-SDK contracts in
openspec/specs/are the point of this repo, but nothingrecorded how far each SDK actually is from them. This gives that a durable, per-contract home with
evidence and a backwards-compatibility verdict attached to every gap, so the specs can be used to
plan SDK work instead of only to settle arguments.
Scope:
compliance/only. No spec content, no SDK code, nothing outside that directory.This run (2026-11-09) — posthog-php, posthog-ruby, posthog-go
The contract surface grew from 68 to 69: Experiment Exposure
(#162) — for a config version 2 flag, a read landing
on an experiment split emits
$experiment_exposuredirectly. sdk-specs advanced 19 commits over23 spec files.
Selection. posthog-php and posthog-ruby were the two stalest pointers (6 weeks, the only two
outside the ~4-week window) and had the largest backfill — they were the top of last run's queue.
posthog-go is the only remaining matrix SDK in the
mcp-analyticsapplicability table, which five ofthis cycle's commits touch. All three are server SDKs, which is where the Emails, Exception Event
Metadata, Tracing Headers and config-version-2 flag deltas land. All three had also drifted hard
(+3,409, +1,695 and +11,341 non-test lines respectively).
Highlights
three SDKs audited this run ship it and all three pass — the flattened
template_idbody from#163, Cloud app-host resolution with a custom-host
refusal, and an idempotency key built once per call outside the attempt loop. posthog-ruby
reproduces the spec's error-name table line for line.
#129's span enrichment and
#123's trace context on exception events are
implemented in each, with the same shape. posthog-go alone gets the detached-work scenario
right (an
activeflag the middleware clears withdefer), and posthog-go alone has thetrace-context path reachable only through an opt-in helper in a separate module — a Go design
question rather than a bug, recorded as such.
/flags?v=4. Experiment Exposure is therefore ➖ on all three by thespec's own applicability sentence, and the config-version-2 attribution requirements never
trigger. Raised as a scope question rather than scored as a gap.
lib/posthog/mcp/is 3,798 linesacross 22 files and grew again this cycle, while the applicability table records posthog-ruby as
"not yet". Scored 🟡; the table is flagged for a human.
Gaps discovered this run
$mcp_interfacemissing, no client conversation idsjitter: 0at every ingestion call site$mcp_interfacemissing, no client conversation idsThe jitter requirement (#132, SHOULD-tier) is now
unmet on six of the seven SDKs checked against it. posthog-go is the sharpest case:
NewBackoffsupports jitter and every ingestion and flag call site passes
0, while its own new Emails clientpasses
0.2. posthog-php has the same split. posthog-ruby is the one that passes, and is thereference.
Gaps resolved this run
cycle, are satisfied on arrival — neither became a gap.
evaluation_runtime: "client"warningexclusion is implemented in all three, five days after it merged.
Needs a human
mcp-analyticsapplicability table is stale in two places. It records posthog-ruby as"not yet" against a 3,798-line shipped package, and lists two Go adapter packages where posthog-go
now ships three (
posthogmcpgo, 1,137 lines, new this cycle). The ruby entry changed a row'sstatus: this run scored the package anyway and gave it 🟡, which a maintainer may want reversed
to ➖ until the table is updated.
/flags?v=2keep scoring ➖ on Experiment Exposureindefinitely? Three server SDKs cannot produce correct experiment analytics for a config
version 2 flag, and the matrix shows no gap for it, because the applicability sentence sanctions
abstaining. If
v=4adoption is expected, that sentence is the place to say so.Running state across all runs
Audited at least once: all 12 in-scope SDKs. Pending initial audit: none.
9c059270c08fd1b92f949f188309f8378c832754dca22075c5e788bd1032c1023ddaa654f5cbe87c6538babcae4d019cQueued for next run, in order: posthog-dotnet and posthog-js (now the stalest at 5 weeks, and the
two whose backfill contains a real scoring question — dotnet's Emails row and js's MCP Analytics
row), then posthog-ios. A half-slot sweep can add the four ➖ rows to the six 64-row files and the
single Experiment Exposure row to java/node/python cheaply — but posthog-js's Experiment Exposure row
should be scored, not swept, since it is
both-scoped and js reads/flags.This PR is the accumulating baseline and is intentionally left as a draft; it is not meant to be
merged mid-rotation.
Created with PostHog Desktop
🤖 Generated with Claude Code