Skip to content

fix(db): include techniques in result-row uniqueness - #2

Merged
aistackdev merged 1 commit into
masterfrom
fix/techniques-in-uniqueness
May 26, 2026
Merged

fix(db): include techniques in result-row uniqueness#2
aistackdev merged 1 commit into
masterfrom
fix/techniques-in-uniqueness

Conversation

@aistackdev

Copy link
Copy Markdown

Summary

PR #1 left benchmark_results_unique and eval_results_unique unchanged when it demoted spec_method to a per-measurement techniques jsonb. With two MTP variants (mtp_layers=4 vs 6) now sharing one config_id, rows with the same (workflow_run, config, isl, osl, conc) but different techniques collide on the existing constraint and the later insert overwrites the earlier one.

First post-deploy ingest of gemma4 run 26387431571 landed 6 of ~24 expected rows for exactly this reason. The remaining 18 silently got clobbered.

Migration 007 adds techniques to both unique constraints and re-truncates so re-ingest from scratch is clean. The bulk insert paths also need techniques in their ON CONFLICT clause and in their in-batch dedup discriminator — fixed in benchmark-ingest.ts and eval-ingest.ts.

Verification (post-merge)

# Re-dispatch the same run after the auto-deploy
gh workflow run auto-ingest.yml -R vngcloud/InferenceX-app -f run_url=https://github.com/vngcloud/InferenceX/actions/runs/26387431571

# Check: total bmk rows should be ~24 (was 6); distinct techniques should be 2
ssh -p 234 hoanq333@61.28.228.19 'docker compose -f /opt/docker-compose/docker-compose.yml exec -T postgres psql -U inferencex -d inferencex -tAc "
SELECT count(*) FROM benchmark_results;
SELECT techniques::text, count(*) FROM benchmark_results GROUP BY techniques;"'

Test plan

  • typecheck / lint / fmt clean
  • Post-merge: re-ingest of gemma4 lands ~24 rows, with both mtp_layers=4 and mtp_layers=6 represented

🤖 Generated with Claude Code

PR #1 left benchmark_results_unique and eval_results_unique unchanged
when it demoted spec_method to a per-measurement jsonb. With two MTP
variants (mtp_layers=4 vs 6) now sharing one config_id, rows with the
same (workflow_run, config, isl, osl, conc) but different techniques
collided on the existing constraint and the later insert overwrote the
earlier one. First post-deploy ingest of gemma4 run 26387431571 landed
6 of ~24 expected rows for exactly this reason.

Migration 007 adds techniques to both unique constraints and re-
truncates so re-ingest from scratch is clean. The bulk insert paths
also need techniques in their ON CONFLICT clause and in their in-batch
dedup discriminator — fixed in benchmark-ingest.ts and eval-ingest.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@aistackdev
aistackdev merged commit eb4f11d into master May 26, 2026
3 of 9 checks passed
@aistackdev
aistackdev deleted the fix/techniques-in-uniqueness branch May 26, 2026 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant