Declare Qwen-3.5-27B model so its benchmarks ingest + display - #14
Merged
Conversation
Benchmark runs for the dense Qwen3.5-27B (model-prefix `qwen3.5-27b`, vLLM + SGLang on 1x H100) were force-ingested successfully but every row was dropped: the ingest log shows `Skipped: 18 rows / qwen3.5-27b`. `resolveModelKey` returned null because `qwen3.5-27b` wasn't a known model key — it's a new prefix, deliberately distinct from the 397B `qwen3.5` so the dashboard doesn't conflate the two. Declare it in the three registries: - constants/models.ts DB_MODEL_TO_DISPLAY: `qwen3.5-27b` -> `Qwen-3.5-27B` (the single source of truth; DB_MODEL_KEYS + DISPLAY_MODEL_TO_DB derive from it, so the ETL now resolves the prefix instead of skipping). - app data-mappings.ts: new Model.Qwen3_5_27B enum member + MODEL_CONFIG entry (label "Qwen3.5 27B", prefix `qwen3.5-27b`) for frontend display. - db etl/normalizers.ts MODEL_TO_KEY: map the HF paths Qwen/Qwen3.5-27B[-FP8] -> `qwen3.5-27b` as a fallback when the prefix field is absent (e.g. eval artifacts). No migration: `model` is free-form lowercase text and `qwen3.5-27b` already satisfies the lowercase CHECK. Distinct display name keeps it a separate dashboard bucket from the 397B (no point-release grouping). After deploy, re-ingest the two qwen3.5-27b runs (vLLM 26717611277, SGLang 26717831054) so their previously-skipped rows land. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Benchmarks for the dense Qwen3.5-27B (model-prefix
qwen3.5-27b, vLLM + SGLang on 1× H100) force-ingested "successfully" but every row was dropped. The ingest log:resolveModelKeyreturnednullbecauseqwen3.5-27bisn't a known model key. It's a new prefix, deliberately kept distinct from the 397Bqwen3.5so the two don't conflate on the dashboard — but it was never declared. (gemma4 ingested fine becausegemma4is declared.)Fix — declare the model in the three registries
constants/models.tsDB_MODEL_TO_DISPLAY:qwen3.5-27b→Qwen-3.5-27B. Single source of truth —DB_MODEL_KEYSandDISPLAY_MODEL_TO_DBderive from it, so the ETL now resolves the prefix instead of skipping.app/src/lib/data-mappings.ts: newModel.Qwen3_5_27Benum member +MODEL_CONFIGentry (label: 'Qwen3.5 27B',prefix: 'qwen3.5-27b') for frontend display/labeling.db/src/etl/normalizers.tsMODEL_TO_KEY: mapQwen/Qwen3.5-27B[-FP8]→qwen3.5-27bas a fallback when the prefix field is absent (e.g. eval artifacts).Notes
configs.modelis free-formtextwith a lowercase CHECK;qwen3.5-27balready satisfies it.Qwen-3.5-27B, separate from the 397BQwen-3.5-397B-A17B) → its own dashboard bucket, not point-release grouped.Record<Model,…>isMODEL_CONFIG(updated);MODEL_ARCHITECTURESisPartial<…>. Localtsc --noEmitpasses.After merge + deploy
Re-ingest the two skipped runs so their rows land:
…/InferenceX/actions/runs/26717611277…/InferenceX/actions/runs/26717831054🤖 Generated with Claude Code