Skip to content

fix: require enough snapshots for regression analysis - #81

Merged
alexshapalov merged 1 commit into
pgrundev:mainfrom
DivyamTalwar:fix/require-sufficient-regression-history
Oct 12, 2026
Merged

alexshapalov merged 1 commit into
pgrundev:mainfrom
DivyamTalwar:fix/require-sufficient-regression-history

Conversation

@DivyamTalwar

Copy link
Copy Markdown
Contributor

What and why

Closes #80.

Four snapshots are needed to produce the three interval observations required by the existing sustained-shift detector. Share that minimum across analysis, CLI rendering, wider-window guidance and the MCP description so an insufficient history is not reported as a clean diagnosis.

A real-store CLI regression exercises three snapshots inside the window plus older available history. A positive control proves that the smallest sufficient four-snapshot series still detects a genuine slowdown. Historical changelog entries and design records are left untouched.

Verification

Verified commit: d59be1de91ba16551828a87e04ee4ab1695375af.

  • New actual analyzer/CLI regressions fail on upstream 282ffe8fdd7acdd7b6e1dbd8cf04c155401d3a92 for missing guidance and the false clean result.
  • go test -race ./internal/why ./cmd/pgbot -count=1: passed.
  • Positive boundary: four snapshots produce three intervals (8, 26, 26) and one detected sustained slowdown.
  • bash scripts/gate.sh: passed on committed HEAD, including pinned lint, unit tests and all six release-target builds.
  • go test -race ./...: passed; formatting and git diff --check clean.

Scope and risk

This corrects the data-sufficiency boundary, not the shift algorithm. Four total snapshots remain necessary rather than sufficient when measurements are missing or reset. Tests use actual SQLite history and actual CLI/analyzer code; no PostgreSQL connection is required. No report-field, schema, dependency or SQL changes. Revert the commit to restore the previous threshold.

Checklist

  • scripts/gate.sh passes on committed HEAD; full race suite passes.
  • Read-only, privacy and deterministic-finding invariants preserved.
  • No new finding or model.Context field; catalog/model schema unchanged.

Three snapshots yield only two interval measurements, but the sustained
shift detector requires three. The earlier guard treated this as sufficient
history and the CLI could report that nothing got worse without an
analyzable baseline.

Share the four-snapshot minimum across analysis, offline/live rendering,
window guidance and the MCP tool description. Keep report field shapes and
valid-history detection unchanged. Cover actual stored-history CLI output
and a smallest-sufficient four-snapshot case that detects a real shift.

Verification: new regressions fail on 282ffe8; full why and CLI race suites
pass. No new queries, dependencies or schema changes.
@alexshapalov
alexshapalov merged commit b5643b2 into pgrundev:main Oct 12, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Three snapshots can produce a misleading clean regression diagnosis

2 participants