Skip to content

Declare Qwen-3.5-27B model so its benchmarks ingest + display - #14

Merged
aistackdev merged 1 commit into
masterfrom
feat/declare-qwen3.5-27b
Jun 1, 2026
Merged

Declare Qwen-3.5-27B model so its benchmarks ingest + display#14
aistackdev merged 1 commit into
masterfrom
feat/declare-qwen3.5-27b

Conversation

@aistackdev

Copy link
Copy Markdown

Problem

Benchmarks for the dense Qwen3.5-27B (model-prefix qwen3.5-27b, vLLM + SGLang on 1× H100) force-ingested "successfully" but every row was dropped. The ingest log:

=== ingesting …/runs/26717611277 ===
  bmk_qwen3.5-27b_1k1k_fp8_vllm_…   (all 18 artifacts read)
  Skipped: 18 rows
    qwen3.5-27b

resolveModelKey returned null because qwen3.5-27b isn't a known model key. It's a new prefix, deliberately kept distinct from the 397B qwen3.5 so the two don't conflate on the dashboard — but it was never declared. (gemma4 ingested fine because gemma4 is declared.)

Fix — declare the model in the three registries

  • constants/models.ts DB_MODEL_TO_DISPLAY: qwen3.5-27bQwen-3.5-27B. Single source of truth — DB_MODEL_KEYS and DISPLAY_MODEL_TO_DB derive from it, so the ETL now resolves the prefix instead of skipping.
  • app/src/lib/data-mappings.ts: new Model.Qwen3_5_27B enum member + MODEL_CONFIG entry (label: 'Qwen3.5 27B', prefix: 'qwen3.5-27b') for frontend display/labeling.
  • db/src/etl/normalizers.ts MODEL_TO_KEY: map Qwen/Qwen3.5-27B[-FP8]qwen3.5-27b as a fallback when the prefix field is absent (e.g. eval artifacts).

Notes

  • No migration: configs.model is free-form text with a lowercase CHECK; qwen3.5-27b already satisfies it.
  • Distinct display name (Qwen-3.5-27B, separate from the 397B Qwen-3.5-397B-A17B) → its own dashboard bucket, not point-release grouped.
  • Only exhaustive Record<Model,…> is MODEL_CONFIG (updated); MODEL_ARCHITECTURES is Partial<…>. Local tsc --noEmit passes.

After merge + deploy

Re-ingest the two skipped runs so their rows land:

  • vLLM: …/InferenceX/actions/runs/26717611277
  • SGLang: …/InferenceX/actions/runs/26717831054
gh workflow run auto-ingest.yml -R vngcloud/InferenceX-app -f run_url=<run-url>   # one at a time

🤖 Generated with Claude Code

Benchmark runs for the dense Qwen3.5-27B (model-prefix `qwen3.5-27b`,
vLLM + SGLang on 1x H100) were force-ingested successfully but every row
was dropped: the ingest log shows `Skipped: 18 rows / qwen3.5-27b`.
`resolveModelKey` returned null because `qwen3.5-27b` wasn't a known
model key — it's a new prefix, deliberately distinct from the 397B
`qwen3.5` so the dashboard doesn't conflate the two.

Declare it in the three registries:
- constants/models.ts DB_MODEL_TO_DISPLAY: `qwen3.5-27b` -> `Qwen-3.5-27B`
  (the single source of truth; DB_MODEL_KEYS + DISPLAY_MODEL_TO_DB derive
  from it, so the ETL now resolves the prefix instead of skipping).
- app data-mappings.ts: new Model.Qwen3_5_27B enum member + MODEL_CONFIG
  entry (label "Qwen3.5 27B", prefix `qwen3.5-27b`) for frontend display.
- db etl/normalizers.ts MODEL_TO_KEY: map the HF paths
  Qwen/Qwen3.5-27B[-FP8] -> `qwen3.5-27b` as a fallback when the prefix
  field is absent (e.g. eval artifacts).

No migration: `model` is free-form lowercase text and `qwen3.5-27b`
already satisfies the lowercase CHECK. Distinct display name keeps it a
separate dashboard bucket from the 397B (no point-release grouping).

After deploy, re-ingest the two qwen3.5-27b runs (vLLM 26717611277,
SGLang 26717831054) so their previously-skipped rows land.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@aistackdev
aistackdev merged commit a0986f1 into master Jun 1, 2026
3 of 9 checks passed
@aistackdev
aistackdev deleted the feat/declare-qwen3.5-27b branch June 2, 2026 03:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant