Skip to content

fix(experiments): match AI summary metric order to the UI - #84318

Merged
trunk-io[bot] merged 3 commits into
masterfrom
experiments/max-summary-metric-ordering
Aug 19, 2026
Merged

trunk-io[bot] merged 3 commits into
masterfrom
experiments/max-summary-metric-ordering

Conversation

@jurajmajerik

Copy link
Copy Markdown
Contributor

Problem

The AI experiment summary numbers metrics as "inline metrics first, then shared metrics", ignoring primary_metrics_ordered_uuids / secondary_metrics_ordered_uuids. When the user has reordered metrics, "Metric 3" in the summary is a different metric than #3 in the app's metrics list.

Changes

Sort the combined metric list by the experiment's ordered uuids in ExperimentSummaryDataService before numbering. Metrics missing from the ordering keep their relative position at the end, and experiments without an ordering (null) keep the current behavior.

How did you test this code?

Extended test_fetch_experiment_data_combines_inline_and_saved_metrics with an ordering that puts the saved metric first and asserted the numbered names come out in that order. Without the fix, the inline metric stays first. Ran the full test_experiment_summary_tool.py suite (28 passed).

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Claude Code found this while investigating why the AI summary referenced metrics not visible in the metrics list (#84316). Skills invoked: /wt, /writing-tests. Sibling PRs: #84316 (saved metric names), and a third for the summary metric cap.

@jurajmajerik jurajmajerik self-assigned this Aug 18, 2026
@trunk-io

trunk-io Bot commented Aug 18, 2026 •

Copy link
Copy Markdown

😎 Merged successfully - details.

@jurajmajerik
jurajmajerik marked this pull request as ready for review August 18, 2026 12:29
@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team August 18, 2026 12:30
@greptile-apps

greptile-apps Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "fix(experiments): match AI summary metri..." | Re-trigger Greptile

@jurajmajerik jurajmajerik added the stamphog Request AI approval (no full review) label Aug 18, 2026
@trunk-io

trunk-io Bot commented Aug 18, 2026 •

Copy link
Copy Markdown

Static Badge   Static Badge   Static Badge

View Full Report ↗︎ ⋅ Docs

stamphog[bot]
stamphog Bot previously approved these changes Aug 18, 2026

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small, contained bugfix outside risky territory, with a test demonstrating the new ordering behavior, authored by a team member with strong familiarity in these exact files.

  • Author wrote 100% of the modified lines and has 49 merged PRs in these paths (familiarity STRONG).
  • 👍 on the PR from greptile-apps[bot].
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✓ no deny categories matched
size ✓ 15L, 1F substantive, 30L/2F incl. docs/generated/snapshots — within ceiling
tier ✓ T1-agent / T1b-small (30L, 2F, single-area, fix)
stamphog 2.0.0b4 .stamphog/policy.yml @ 7cf4b93 · reviewed head bab5b64

@github-actions

github-actions Bot commented Aug 18, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Trunk lane — runs backend Python tests

This PR is assigned to a lane that runs backend Python tests.

✅ Backend coverage — all changed backend lines covered

🧪 Backend test coverage

Patch coverage — changed backend lines (products + core): ████████████████████ 100.0% (12 / 12)

All changed backend lines are covered ✅

Per-product line coverage (touched products)
Product Coverage Lines
platform_features ██░░░░░░░░░░░░░░░░░░ 12.1% 7 / 58
warehouse_sources_queue ███████░░░░░░░░░░░░░ 36.1% 91 / 252
batch_exports ████████░░░░░░░░░░░░ 39.2% 8,864 / 22,593
demo ███████████░░░░░░░░░ 56.8% 1,516 / 2,667
data_tools █████████████░░░░░░░ 66.2% 90 / 136
tasks ██████████████░░░░░░ 70.3% 39,520 / 56,187
ai_gateway ███████████████░░░░░ 75.0% 9 / 12
signals █████████████████░░░ 83.0% 28,895 / 34,793
apm █████████████████░░░ 83.3% 1,234 / 1,481
cdp █████████████████░░░ 84.3% 3,959 / 4,694
actions █████████████████░░░ 86.6% 717 / 828
wizard █████████████████░░░ 86.7% 1,090 / 1,257
data_modeling █████████████████░░░ 87.0% 9,619 / 11,054
notebooks █████████████████░░░ 87.2% 8,553 / 9,812
cohorts ██████████████████░░ 87.6% 6,528 / 7,452
product_tours ██████████████████░░ 87.9% 1,303 / 1,482
business_knowledge ██████████████████░░ 89.0% 4,385 / 4,929
data_warehouse ██████████████████░░ 89.0% 12,485 / 14,032
exports ██████████████████░░ 89.3% 7,702 / 8,627
managed_warehouse ██████████████████░░ 89.5% 7,149 / 7,992
dashboards ██████████████████░░ 89.5% 6,221 / 6,954
engineering_analytics ██████████████████░░ 89.5% 6,828 / 7,632
visual_review ██████████████████░░ 89.8% 6,057 / 6,745
alerts ██████████████████░░ 90.4% 4,699 / 5,199
mcp_analytics ██████████████████░░ 90.4% 3,968 / 4,388
links ██████████████████░░ 90.6% 183 / 202
streamlit_apps ██████████████████░░ 90.7% 2,630 / 2,901
conversations ██████████████████░░ 90.8% 20,304 / 22,363
error_tracking ██████████████████░░ 91.3% 11,363 / 12,452
stamphog ██████████████████░░ 91.3% 5,031 / 5,510
canvas ██████████████████░░ 91.9% 4,401 / 4,788
slack_app ██████████████████░░ 92.5% 11,743 / 12,701
early_access_features ███████████████████░ 92.5% 1,323 / 1,430
managed_migrations ███████████████████░ 92.6% 1,556 / 1,681
web_analytics ███████████████████░ 92.8% 16,513 / 17,802
notifications ███████████████████░ 92.9% 1,065 / 1,146
ai_observability ███████████████████░ 93.0% 17,367 / 18,674
mcp_store ███████████████████░ 93.1% 7,411 / 7,957
surveys ███████████████████░ 93.3% 5,943 / 6,373
posthog_ai ███████████████████░ 93.3% 1,340 / 1,436
reminders ███████████████████░ 93.4% 468 / 501
marketing_analytics ███████████████████░ 93.5% 16,431 / 17,577
approvals ███████████████████░ 93.5% 3,491 / 3,734
product_analytics ███████████████████░ 93.6% 7,133 / 7,622
billing_alerts ███████████████████░ 93.9% 2,047 / 2,181
legal_documents ███████████████████░ 94.0% 1,914 / 2,037
endpoints ███████████████████░ 94.3% 8,814 / 9,349
review_hog ███████████████████░ 94.3% 9,962 / 10,562
workflows ███████████████████░ 94.4% 9,118 / 9,661
tracing ███████████████████░ 94.4% 2,678 / 2,837
skills ███████████████████░ 94.7% 3,869 / 4,085
annotations ███████████████████░ 95.1% 732 / 770
experiments ███████████████████░ 95.5% 29,048 / 30,416
replay_vision ███████████████████░ 95.7% 21,667 / 22,644
messaging ███████████████████░ 95.7% 3,545 / 3,704
logs ███████████████████░ 95.7% 13,001 / 13,580
feature_flags ███████████████████░ 96.0% 19,029 / 19,827
customer_analytics ███████████████████░ 96.2% 13,777 / 14,325
revenue_analytics ███████████████████░ 96.3% 1,841 / 1,911
user_interviews ███████████████████░ 96.5% 2,638 / 2,734
data_quality ███████████████████░ 96.5% 2,688 / 2,785
growth ███████████████████░ 96.7% 6,505 / 6,729
access_control ███████████████████░ 96.7% 873 / 903
warehouse_sources ████████████████████ 97.5% 401,435 / 411,575
analytics_platform ████████████████████ 98.1% 2,286 / 2,330
data_catalog ████████████████████ 98.1% 2,976 / 3,033
metrics ████████████████████ 98.2% 2,491 / 2,536
pulse ████████████████████ 98.4% 2,017 / 2,049
live_debugger ████████████████████ 99.2% 613 / 618
field_notes ████████████████████ 99.4% 158 / 159

Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.

…mary-metric-ordering

# Conflicts:
#	products/experiments/backend/experiment_summary_data_service.py
#	products/experiments/backend/test/test_experiment_summary_tool.py
@stamphog
stamphog Bot dismissed their stale review August 19, 2026 08:12

New commits pushed (delta classified non_trivial_delta) — stamphog approval dismissed; re-review running automatically.

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small, focused, well-tested fix in a file the author has strong recent familiarity with and owns via team membership; no risky territory touched.

  • Author wrote 100% of the modified lines and has 54 merged PRs in these paths (familiarity STRONG).
  • 👍 on the PR from greptile-apps[bot].
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✓ no deny categories matched
size ✓ 15L, 1F substantive, 30L/2F incl. docs/generated/snapshots — within ceiling
tier ✓ T1-agent / T1b-small (30L, 2F, single-area, fix)
stamphog 2.0.0b4 .stamphog/policy.yml @ 58a0411 · reviewed head 4719533

@trunk-io
trunk-io Bot merged commit 0ae6edf into master Aug 19, 2026
264 checks passed
@trunk-io
trunk-io Bot deleted the experiments/max-summary-metric-ordering branch August 19, 2026 12:28
@deployment-status-posthog

deployment-status-posthog Bot commented Aug 19, 2026 •

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-08-19 13:09 UTC Run
prod-us ✅ Deployed 2026-08-19 13:30 UTC Run
prod-eu ✅ Deployed 2026-08-19 13:27 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stamphog Request AI approval (no full review)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant