Skip to content

fix(aio): don't record zero cost when a generation has no token counts - #90211

Merged
trunk-io[bot] merged 18 commits into
masterfrom
posthog/ai-cost-unknown-not-zero
Sep 1, 2026
Merged

trunk-io[bot] merged 18 commits into
masterfrom
posthog/ai-cost-unknown-not-zero

Conversation

@bernatixer

@bernatixer bernatixer commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor

Problem

LLM analytics reports $0.00 for a generation whose cost was never known, which reads as a call that was free.

A generation arrives with no token counts whenever the provider never reported usage. The common case is a user cancelling a streaming response: usage arrives only in a stream's final chunk, so cancelling loses it.

The cost pipeline then resolved the model, found its rate, and multiplied that rate by an absent count. An absent value coerces to 0, so the product was 0, and that zero was written to $ai_total_cost_usd.

The rate is small but steady, and close to identical across every major provider, so it is not specific to one model, provider, or SDK.

Changes

  • The trace tree shows no cost where the cost is unknown, instead of $0.00. This covers a generation row and the span that sums it. The visual review run on this PR captures the before and after on the trace scene story: the __start__ span row drops its / $0.00.
  • Ingestion leaves the cost properties unset when a generation reports no token counts.
  • A token count that is present and zero still prices at zero. Presence is the test rather than value: 0 is a usage report, an absent property is silence.
  • A span with some priced children keeps the partial sum, because a cost some children reported is still known.
  • Request and web search charges survive absent token counts. Neither depends on a token count, so neither becomes unknown.
  • Costs the client computed themselves survive absent token counts too: an event carrying only $ai_input_cost_usd or $ai_request_cost_usd keeps its total, as it did before this branch, and is not counted as unknown.
  • Input and output costs are each written only when that side's own token counts exist. A side without counts is unknown, and one reported side must not fabricate a $0.00 for the other: an interrupted Anthropic stream reports input tokens from message_start while the output count only arrives in the final delta. The total sums the components that exist.
  • llma_ai_cost_outcome_total gains an unknown outcome, so the rate is measurable as soon as this deploys.

The ingestion change alone does not remove the $0.00. Both the generation row and the span above it coerced a missing cost to 0 on their own, so all three layers are needed to deliver the first bullet.

The runtime ingestion diff is deliberately one file (costs/index.ts) plus a metric help text. The token validation step stays at its production behavior: widening it to sanitize the modality token counts was considered and dropped, because it would mutate event properties on traffic unrelated to this fix while the cost calculators already sanitize those values at read time. It can ship separately.

Note

On affected events, $ai_total_cost_usd is absent rather than 0, and $ai_input_cost_usd/$ai_output_cost_usd are absent when their side reported no token counts. A one-sided event's total is the sum of its known components, a documented lower bound, because billing and trace-level views sum this property. sum() over these is unchanged. avg() and count() will move.

$ai_cache_reporting_exclusive is no longer written on an unpriced event, because the early return skips the input cost calculator that sets it. Nothing reads that flag for an event carrying no token counts.

How did you test this code?

  • Each new test fails when its fix is reverted and passes with it. Verified by reverting the ingestion guard, the client-cost check, the input/output gate, the per-side token gates, the generation null check, the span aggregation, and entries in the token property lists in turn.
  • Those are the regressions the tests exist to catch: a guard broadened back to dropping known non-token or client-sent costs, unknown token costs written as $0.00 components, one reported token side writing the other side's cost as $0.00, a || 0 coercion that restores $0.00 on a generation, a span that sums unknowns to zero, and a token property list narrowed back to the core counts.
  • The cases guarding that list name their properties directly. A case that read the list would disappear when an entry went missing, rather than fail.
  • Nothing here guards the opposite drift, a calculator that starts reading a token property nobody added to the list. The list's comment states the invariant; closing it needs one source of truth across the pipeline, which is a larger change than this one.
  • Ran the AI ingestion suite and the products/ai_observability/frontend suite. Both pass. Eslint passes on every touched file.
  • The frontend typecheck reports no errors in the files this change touches.
  • The S3-backed cases in blob-offload/blob-store.integration.test.ts fail here for want of a running object store, identically before and after.
  • hogli ci:preflight passes (no failures, staleness advisory only). hogli review is not available on this checkout, so the Greptile pass happens server-side.

Automatic notifications

  • Publish to changelog?

Changelog copy: LLM analytics now shows no cost, instead of $0.00, for calls whose provider never reported usage. Average cost per generation may look different from before.

An in-app notice was considered per /announcing-behavior-changes and rejected: "this user is affected" cannot be expressed from client state, so a banner would show to everyone about a minority's events. The permanent affordance, if wanted later, is a computed tooltip on the missing cost, not a time-boxed notice.

Docs update

None.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Written by Claude Code, directed by a PostHog engineer working a support question about a trace showing no cost.

Skills invoked: /writing-tests, /writing-pr-descriptions, /query-clickhouse-via-metabase, /adding-ingestion-warnings.

Four explanations were proposed and rejected against data before this one: a trace with no generation events, a model missing from the pricing table, a broken OTel attribute mapping, and early abandonment of the stream. The cause was confirmed by driving the Vercel AI SDK against a mock provider and capturing its OpenTelemetry spans: an aborted stream emits a span carrying model, provider and prompt, and no usage attributes.

Review then caught nine defects across six rounds. The early return skipped the request and web search calculators, which read no token counts, so it turned two known costs into unknowns; a later round found it also dropped one-sided costs the client had computed themselves. The generation row coerced the now-absent cost back to 0. The span above it summed unknown children to zero, which is the shape a Vercel AI SDK or OTel trace actually has. Events priced only by a non-token charge had their unknown token costs written as $0.00 components. The outcome counter labelled a client-supplied total as unknown. The single token gate let one reported side write the other side's cost as 0, so the sides now gate independently. Omitting the total on one-sided events was tried and reverted: the AI credit usage report bills only positive totals, and trace-level views sum event totals, so a lower-bound total is the least wrong value for a property every consumer aggregates.

A final scope pass deliberately shrank the ingestion diff: a round of consolidation had widened the token validation step from seven sanitized properties to thirteen, and that widening was reverted because it changed behavior on events unrelated to this fix. Minimizing production ingestion risk won over deduplicating the property list.

An ingestion warning was designed and dropped. The fleet baseline for missing usage is high enough that a per-event warning would fire for nearly every project with meaningful volume, which is noise rather than signal. A health check with a proportion threshold is the better instrument and needs separate design, because operations that are not billed per token, such as image generation and transcription, report no tokens correctly.

Public artifact: the investigation used a customer's project data through internal tooling. No customer data, project identifier, or operational figure appears in this PR, its commits, or its tests. The reproduction used a mock provider and an invented prompt.

A generation that arrives without any token counts was priced by multiplying the model rate by an absent count, which coerces to 0. The resulting $0.00 is indistinguishable from a call that genuinely cost nothing.

Absent token counts now leave the cost properties unset, so the difference between "cost nothing" and "cost unknown" survives to the UI. A token count that is present and zero still prices at zero, because that is a real usage report.

Tracked as a new `unknown` outcome on llma_ai_cost_outcome_total.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
@trunk-io

trunk-io Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

😎 Merged successfully - details.

@github-actions

github-actions Bot commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

✅ Trunk lane — non-backend lane

This PR is assigned to the non-backend lane. It does not run backend Python tests and may merge in parallel with PRs in other lanes.

⚠️ Complexity (TypeScript) — 8 functions above 10 (max 66)

Cyclomatic complexity above 10 in changed typescript files. Warn only: worth simplifying when you next touch these functions.

Function Location Complexity
<anonymous> products/ai_observability/frontend/AIObservabilityTraceScene.tsx:1493 66
TraceWorkflowPanel products/ai_observability/frontend/AIObservabilityTraceScene.tsx:725 47
TraceNode products/ai_observability/frontend/AIObservabilityTraceScene.tsx:1268 26
aggregateSpanMetrics products/ai_observability/frontend/aiObservabilityTraceDataLogic.ts:875 26
QueueTraceNavigationButtons products/ai_observability/frontend/AIObservabilityTraceScene.tsx:197 25
TraceSceneWrapper products/ai_observability/frontend/AIObservabilityTraceScene.tsx:448 17
<anonymous> products/ai_observability/frontend/aiObservabilityTraceDataLogic.ts:563 14
buildFeedbackAttachmentMap products/ai_observability/frontend/aiObservabilityTraceDataLogic.ts:115 12
⚠️ Bundle size — 🔺 +12 B (+0.0%)

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 68.07 MiB · 🔺 +12 B (+0.0%)

No file changed by more than 1000 B.

Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report

✅ Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root Eager (shipped) Δ vs base Budget
entry (logged-out pages, app bootstrap)
src/index.tsx
1.42 MiB · 22 files no change ███░░░░░░░ 31.5% of 4.51 MiB
authenticated shell (every logged-in page)
src/scenes/AuthenticatedShell.tsx
8.84 MiB · 3,245 files no change █████████░ 91.0% of 9.71 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx

Largest files eagerly shipped from src/index.tsx
Size File
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.5 KiB ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
3.9 KiB ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.4 KiB ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.3 KiB src/RootErrorBoundary.tsx
912 B ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js
789 B src/scenes/ChunkLoadErrorBoundary.tsx
762 B src/index.tsx
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
Size File
307.0 KiB ../node_modules/.pnpm/posthog-js@1.422.5_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/rrweb.js
267.7 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
263.5 KiB ../node_modules/.pnpm/posthog-js@1.422.5_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.js
253.2 KiB src/taxonomy/core-filter-definitions-by-group.json
154.2 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
104.7 KiB src/lib/api.ts
95.2 KiB ../packages/quill/packages/quill/dist/index.js
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
90.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js

Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479

✅ Toolbar bundle — eager 2.26 MiB within budget

What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.

Metric Size Δ vs base Budget
Eager (shipped)
entry + static imports
2.26 MiB · 18 files no change ████░░░░░░ 39.5% of 5.72 MiB
Deferred (lazy) 2.11 MiB · 45 files no change n/a — loads on demand
Loader dist/toolbar.js 1.1 KiB no change █░░░░░░░░░ 5.8% of 19.5 KiB
Largest eagerly-shipped chunks
Size File
751.0 KiB dist/toolbar/toolbar-app-2F4EKOCO.css
588.3 KiB dist/toolbar/chunk-chunk-7FNJZP6R.js
484.7 KiB dist/toolbar/chunk-chunk-OQ4CRAT2.js
134.1 KiB dist/toolbar/chunk-chunk-MZDGT3DD.js
131.8 KiB dist/toolbar/chunk-chunk-FDH2IBXT.js
71.3 KiB dist/toolbar/toolbar-app-YIDQNIPP.js
69.0 KiB dist/toolbar/chunk-chunk-TSAL54PB.js
35.6 KiB dist/toolbar/chunk-chunk-L2QPXV4Z.js
21.0 KiB dist/toolbar/chunk-chunk-3RON5VMO.js
6.8 KiB dist/toolbar/chunk-chunk-DV7IWQNF.js

Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile

✅ Dist folder size — 🔺 +1.7 KiB (+0.0%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 1446.19 MiB · 🔺 +1.7 KiB (+0.0%)

✅ Playwright — all passed

All tests passed.

View test results →

The absent and zero token-count cases become one parameterized test, since they are the same behavior distinguished by input.

Adds a case for an event that reports only cache tokens. The pricing gate reads thirteen token properties, but the tests exercised two, so narrowing that list would silently stop pricing events that report only cache or modality tokens.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
@trunk-io

trunk-io Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

Static Badge   Static Badge   Static Badge

View Full Report ↗︎ ⋅ Docs

Skipping cost calculation for a generation with no token counts also dropped the two costs that never depended on tokens. A per-request charge bills for the call itself, and a web search charge bills a count the provider did report, so both were still known.

That turned known costs into unknowns: an event priced per request, or one reporting web searches, lost a cost it previously carried. 580 model entries carry a web search price, so that half was live.

The early return now applies only when nothing at all priced the call. The two non-token costs are computed first and consulted by the guard.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
The trace tree coerced a missing cost to 0, so a generation whose provider never reported usage still displayed $0.00 — the symptom the ingestion change set out to remove. Leaving the cost properties unset was necessary but not enough on its own.

The tree now carries a null cost through and renders nothing for it, matching the metadata header and event card, which already hide the chip. A cost of 0 still renders, because a call that genuinely cost nothing is a different fact from one whose cost was never reported.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
An event carrying its own $ai_total_cost_usd, with no token counts and no input or output costs, reaches the early return. The cost survives untouched, but the outcome counter labelled it unknown, which overstates how often the cost is not known.

The increment now skips a total the client already sent, matching the other paths that leave a passthrough total untracked.

The token property list gains a note on the invariant it carries, and the case covering it names its properties directly rather than reading the list. Iterating the list would only drop a case when an entry goes missing, never fail one.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
A span sums the costs its children reported, starting from zero. A span whose generations all went unpriced summed to zero and rendered $0.00, so the symptom survived one level up from the generation row.

That is the shape a Vercel AI SDK or OTel trace has, with the generation nested under a span, so a cancelled stream still read as free on the parent row.

The aggregation now tracks whether any descendant reported a cost and returns null when none did. A span with a mix keeps the partial sum, because a cost some children reported is still known.

The trace root already behaved this way: its query wraps the sum in nullIf, so an all-unpriced trace comes back null.

Three assertions expected the summed zero. They now expect null, which is what those fixtures describe.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
@posthog

posthog Bot commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

✅ Visual changes approved by @bernatixer — baseline updated in 7d61e1e.

View this run in PostHog

1 changed.

The set of token properties the cost pipeline reads was written out by hand in three places: the validation step that sanitizes them, that step's test, and the guard deciding whether an event reported usage. The step's copy listed seven of the thirteen.

They now share one definition. The step sanitizes the six modality counts it previously skipped, so a malformed audio or image count is nulled and reported instead of reaching js-big-decimal untouched. The cost outcome is the same either way, because the calculators already treat an unusable value as absent.

The step's test keeps a representative subset for its value matrix rather than the whole list, since the step applies identical checks to every entry and the full matrix runs 1184 cases. One case covers a property outside that subset, and fails if the shared list loses it.

Also names the two outcomes the cost counter's help text left out. Both predate this branch: "error" counts a total that came out NaN.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
The span aggregation tracked whether any child reported a cost in a separate boolean, then chose between the running total and null at the end. Two variables held one piece of state.

The running total is now nullable itself. It starts null and the first known cost promotes it, so null means unknown at every point rather than only after the return.

No behavior change. Every case the boolean covered follows from the arithmetic: a span with its own cost keeps it, a span whose children all went unpriced stays null, and a child reporting zero still produces a known zero.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
The guard asked whether the event carried token counts or a computed request or web search charge, but never whether the client had already sent a cost. An event with only $ai_input_cost_usd or only $ai_request_cost_usd lost the total it previously received, and was counted as unknown. One-sided costs never reach the passthrough early return, which needs both input and output present.

Any client-sent cost component now keeps the event on the pricing path, so its total is computed as before. This subsumes the earlier client-total check on the unknown counter.

Input and output costs are also written only when token counts exist. They are token rates times token counts, so without counts they are unknown, and writing 0 put "Input $0.00" in the cost breakdown on the same class of event this branch fixes. The total sums the components that exist.

Also moves token-properties.ts from costs/ to the pipeline root, since the validation step and the cost calculators both read it.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
The token validation step returns to its production scope. Widening it from seven sanitized properties to thirteen was hardening beyond this fix: it mutated malformed modality counts on every AI event, while the cost calculators already sanitize those values at read time. It can ship on its own if wanted.

With the step reverted, the cost guard is the only reader of the token property list again, so the list moves back inline next to the guard. This also removes the parent-relative import that failed the ingestion lint rule.

The runtime ingestion diff is now one file plus a metric help text.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
The unknown outcome's comment described only the token-count condition, but the label fires when nothing priced the event at all: no token counts, no request or web search charge, and no client-sent cost. The comment now says so, since this counter is how the change is measured after deploy. The error label's comment states when it can fire: only an Infinity-sized component survives the write guards, so it should stay at zero.

The token property list's comment cross-references the validation step, which sanitizes only the core subset. The gap is safe because every calculator read goes through finiteNumberOrUndefined, and the comment now says that instead of reading as if the step covered the full list.

The span aggregation seeds its running cost from extractTotalCost and derives its gate from that seed, so one function decides what counts as a known cost. Previously the seed and the gate tested the property two different ways twenty lines apart from the arbiter the tree rows use.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
@github-actions

github-actions Bot commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

🦔 Hogbox preview · ✅ ready

▶ Open the preview

🔑 Login test@posthog.com / 12345678 (demo data)
🧩 Running this PR's backend and frontend, on the PostHog :master base
🔗 Link stable across rebuilds — a re-push swaps the box underneath, the URL stays
🔒 Access tailnet only (PostHog VPN)
🛠️ Admin inspect & debug state in hogland
💤 Idle sleeps after ~30 min idle (snapshot to S3, zero node cost) and wakes on your next visit in ~30s, behind a brief "waking up" screen

commit 80fa8b1 · box box-e0fdfd1b4ea1 · ready in 579s (push → usable) · build log · rebuilds on every push, torn down on close

@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested review from a team August 28, 2026 13:45
The leaf-child branch of aggregateSpanMetrics tested the cost property directly while the recursive branch and the seed already go through extractTotalCost. All three now use the same arbiter of what counts as a known cost.

No behavior change: extractTotalCost applies the identical typeof check.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
The token gate was one boolean over all thirteen count properties, so an event reporting only one side still ran both calculators and wrote the other side's cost as 0. A failed generation with input tokens but no output count carried $ai_output_cost_usd: 0, the same asserted zero this branch removes, one level finer.

The list splits by the side each calculator prices: the input calculator reads eight of the thirteen properties, the output calculator the other five, with no overlap. Each side is now computed and written only when one of its own counts is present, and the component-sum total picks up whichever sides exist.

This shape is common rather than exotic: an interrupted Anthropic stream reports input tokens from message_start while the output count only arrives in the final delta.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
$ai_total_cost_usd reads downstream as the whole cost, but a one-sided event summed to the cost of its known side alone: a lower bound presented as a total. The per-side gating made this shape common, since an interrupted stream often reports input tokens without an output count.

The total is now written only when the input and output sides are equally known, mirroring how token counts are omitted when unknown. A partial event keeps its known components without a total. The gate reads the cost properties rather than the token counts, so a side the client priced directly still counts as known.

Events priced without either side, such as a per-request or web search charge, keep their totals: those charges cover the call by themselves. A client-sent lone input cost loses only the derived total, not the component.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
@veria-ai

veria-ai Bot commented Aug 31, 2026 •

Copy link
Copy Markdown
Contributor

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

Reverts the unequal-sides total guard. Omitting $ai_total_cost_usd on one-sided events breaks three consumers that sum the property: the AI credit usage report bills only positive totals, so a repeatedly aborted stream's known input cost would go unbilled; both trace query runners sum event totals, so the trace header would silently drop the known side; and cost insights sum the property, losing known costs on a common event shape.

A lower bound is the least wrong value for a sum-shaped property that every consumer aggregates. The one-sided semantic stays documented at the write site, and a partial-cost flag or a billing-specific property is the durable fix if the total's completeness needs to be machine-readable.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
bernatixer added a commit to PostHog/posthog-python that referenced this pull request Aug 31, 2026
Every integration wrote $ai_input_tokens: 0 and $ai_output_tokens: 0 when the provider never reported usage, so an interrupted stream priced as a free call downstream. Token counts are now written only when they trace back to a provider report: the shared capture paths, both embedding events, the LangChain callback and the Agents and Claude Agent SDK processors omit unreported counts, and the stream accumulators and response extractors no longer seed at zero.

A zero the provider reported is still sent as 0, so zero keeps meaning a real report of nothing. LangChain cache and reasoning counts follow the same rule instead of coalescing to 0, and the Agents processor's $ai_total_tokens is the sum of the reported sides, omitted when neither side reported.

Mirrors the posthog-js change (PostHog/posthog-js#4664). Ingestion distinguishes absent from zero since PostHog/posthog#90211 and older ingestion coalesces absent to 0 at read time, so this ships independently.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
1 updated
Run: 13ba67a7-5b5d-4d6b-bf72-0c1dacad8dd5

Co-authored-by: bernatixer <2202231+bernatixer@users.noreply.github.com>
bernatixer added a commit to PostHog/posthog-python that referenced this pull request Sep 1, 2026
* fix(ai): omit token counts the provider never reported

Every integration wrote $ai_input_tokens: 0 and $ai_output_tokens: 0 when the provider never reported usage, so an interrupted stream priced as a free call downstream. Token counts are now written only when they trace back to a provider report: the shared capture paths, both embedding events, the LangChain callback and the Agents and Claude Agent SDK processors omit unreported counts, and the stream accumulators and response extractors no longer seed at zero.

A zero the provider reported is still sent as 0, so zero keeps meaning a real report of nothing. LangChain cache and reasoning counts follow the same rule instead of coalescing to 0, and the Agents processor's $ai_total_tokens is the sum of the reported sides, omitted when neither side reported.

Mirrors the posthog-js change (PostHog/posthog-js#4664). Ingestion distinguishes absent from zero since PostHog/posthog#90211 and older ingestion coalesces absent to 0 at read time, so this ships independently.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf

* chore(ai): update public API snapshot for Optional embedding count

extract_gemini_embedding_token_count now returns Optional[int], returning None when no embedding carried a token count. Regenerated with make public_api_snapshot.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
@bernatixer
bernatixer removed request for a team and richardsolomou September 1, 2026 10:54
@trunk-io
trunk-io Bot merged commit f35164b into master Sep 1, 2026
223 checks passed
@trunk-io
trunk-io Bot deleted the posthog/ai-cost-unknown-not-zero branch September 1, 2026 11:39
bernatixer added a commit that referenced this pull request Sep 1, 2026
Two review findings.

The trace JSON export guarded its cost keys with `if (cost)`, so a generation that genuinely priced to zero — now possible since #90211 and shown as $0.00 on the trace header — dropped its cost from the export. The guard is now `!= null`, at both the trace total and each event, so the export agrees with the header. A frontend case pins this and fails on the old guard.

The list-runner test asserted `assertIsNone(unpriced_trace.requestCost)`, which was null for both fixtures, so it pinned nothing. It now asserts `inputCost` is null for the unpriced trace, mirroring the zero trace's `inputCost == 0`.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
bernatixer added a commit that referenced this pull request Sep 1, 2026
The session view returned null for a trace that genuinely cost $0.00, where the single-trace header (#90211) and the traces list now return 0 for the same events. The session runner still wrapped its seven sums in nullIf(sum, 0), which collapses a reported zero into null.

Each of the seven summed fields now uses a count guard: null when no event carried the field, the real sum including zero when one did. This matches trace_query_runner and traces_query_runner.

Adds session_query_runner.py to the "update all three query runners together" rule in the product guide. That rule listed only two runners, which is why this copy fell behind.

Generated-By: PostHog Desktop
Task-Id: ec0cc27f-fc40-4994-b153-2bfe74de1edf
@deployment-status-posthog

deployment-status-posthog Bot commented Sep 1, 2026 •

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-09-01 12:06 UTC Run
prod-us ✅ Deployed 2026-09-01 12:28 UTC Run
prod-eu ✅ Deployed 2026-09-01 12:35 UTC Run

dmarticus added a commit that referenced this pull request Sep 4, 2026
#90211)

Co-authored-by: posthog[bot] <206114724+posthog[bot]@users.noreply.github.com>
Co-authored-by: bernatixer <2202231+bernatixer@users.noreply.github.com>
dmarticus added a commit that referenced this pull request Sep 4, 2026
#90211)

Co-authored-by: posthog[bot] <206114724+posthog[bot]@users.noreply.github.com>
Co-authored-by: bernatixer <2202231+bernatixer@users.noreply.github.com>
dmarticus added a commit that referenced this pull request Sep 4, 2026
#90211)

Co-authored-by: posthog[bot] <206114724+posthog[bot]@users.noreply.github.com>
Co-authored-by: bernatixer <2202231+bernatixer@users.noreply.github.com>

This branch was successfully deployed

1 active deployment
preview-pr-90211 — 80fa8b13 Deployed Sep 1, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants