[wizard-ci-bot]

Automated wizard CI run

Source: wizard-pr
Trigger ID: 3f39e0e
App: groq/node-chat
App directory: apps/groq/node-chat
Workbench branch: wizard-ci-3f39e0e-groq-node-chat
Wizard branch: release-please--branches--main--components--wizard
Context Mill branch: main
PostHog (MCP) branch: master
Timestamp: 2026-09-23T21:39:00.086Z
Duration: 289.0s

YARA Scanner

✓ 49 tool calls scanned, 0 violations detected

No violations: ✓ 49 clean scans

[wizard-ci-bot]

Now I have all the information needed to produce the evaluation.

---

PR Evaluation Report

Summary

This PR instruments a Groq-backed Node.js chat CLI with PostHog AI observability using the @posthog/ai/openai wrapper. The implementation correctly swaps the OpenAI client for the PostHog wrapper, threads per-call identity (posthogDistinctId, posthogTraceId, ``) through every LLM call, and flushes before exit. A graceful fallback pattern preserves the app's functionality when POSTHOG_API_KEY is not configured.

Files changed	Lines added	Lines removed
4	+86	-23

Confidence score: 4/5 👍

• Missing .env.example: No .env.example was created to document the new POSTHOG_API_KEY and POSTHOG_HOST environment variables. [MEDIUM]

---

File changes

Filename	Score	Description
apps/ai-observability/groq/node-chat/src/index.ts	4/5	Core instrumentation: swaps client to @posthog/ai/openai, adds CallContext type, threads posthogDistinctId/posthogTraceId/`` per call, adds fallback pattern and shutdown
apps/ai-observability/groq/node-chat/package.json	5/5	Adds @posthog/ai and posthog-node dependencies
apps/ai-observability/groq/node-chat/.gitignore	5/5	Ignores .env file
apps/ai-observability/groq/node-chat/.posthog-wizard-cache/.posthog-ai.json	5/5	Wizard metadata cache

---

App sanity check ⚠️

Criteria	Result	Description
App builds and runs	Yes	TypeScript compiles, no missing types or modules
Preserves existing env vars & configs	Yes	GROQ_API_KEY still read from env; no pre-existing PostHog client to conflict with
No syntax or type errors	Yes	Clean TypeScript with proper as const assertions and optional chaining
Correct imports/exports	Yes	@posthog/ai/openai, posthog-node, NativeOpenAI from openai — all valid
Minimal, focused changes	Yes	Changes are focused on instrumentation; fallback pattern adds some complexity but preserves behavior
Pre-existing issues	None	Base app is clean

Issues

• No .env.example for PostHog variables: POSTHOG_API_KEY and POSTHOG_HOST are read from the environment but not documented in any .env.example file. Should add one. [MEDIUM]

Other completed criteria

• Dependency versions (@posthog/ai ^6.6.0, posthog-node ^5.53.0) are valid and should contain the required APIs
• posthog?.shutdown() called in finally block — correct for this CLI script
• No credentials hardcoded — API key and host read from environment

---

PostHog implementation ✅

Criteria	Result	Description
PostHog SDKs installed	Yes	@posthog/ai ^6.6.0 and posthog-node ^5.53.0 added
PostHog client initialized	Yes	new PostHog(posthogApiKey, { host, privacyMode: false }) at module scope, conditionally on env var presence
capture()	Yes	Generations captured automatically via @posthog/ai/openai wrapper
identify()	N/A	Server-side CLI; identity via per-call posthogDistinctId
Error tracking	No	No exception capture configured
Reverse proxy	N/A	Server-side CLI

Issues

None critical.

Other completed criteria

• Instrumentation initialized once at module scope — not inside request handlers
• @posthog/ai/openai wrapper correctly wraps the OpenAI client with posthog passed to constructor
• baseURL preserved as https://api.groq.com/openai/v1 — Groq routing intact
• : 'groq' explicitly set in posthogProperties — ensures correct provider attribution
• posthogTraceId generated once per ask() via randomUUID() and shared across condense() and reply() — groups multi-call turns into single traces
• `` uses thread_abc (the app's `threadId`) — correct key name and correct cardinality
• Per-call identity via posthogDistinctId: context.userId — correct scope for this app
• No hand-authored spans — moderate() left as a plain guard function
• No pre-existing PostHog integration to disturb

---

PostHog insights and events ✅

Filename	PostHog events	Description
src/index.ts	`` (×3 per run)	Three LLM calls: one for first ask, two for second ask (condense + reply). All grouped by posthogTraceId per turn, under session thread_abc, attributed to user_123

Issues

None.

Other completed criteria

• Event tree matches README: 1 session → 2 traces → 3 generations (1 + 2)
• Person attribution uses the app's own user_123 identifier — not random or placeholder
• No fabricated structure — sessions map to Thread, traces map to ask() calls
• No secrets or PII in event properties — only model prompts and responses (expected for AI observability)

---

Reviewed by wizard workbench PR evaluator