Repository navigation
File-level: retrain per-language ("local") archetype model — needs a broader corpus #3085
Description
Description
Split off from #1158 (parent epic: #1160). #1158's global file-archetype model was fixed in #3061 (post-assembly classification against a rebaked, self-describing GENERAL_FILE_INFERENCE_MODEL) — that part is done.
The per-language ("local micro-species") model was explicitly left out of that fix and is still broken exactly as #1158 originally described:
SPECIFIC_FILE_INFERENCE_MODELis 74-dimensional and was only ever trained for 5 of the 59 languages this engine parses (python, javascript, typescript, c, cpp).- Even for those 5 languages, the live vector length doesn't match the trained dimensionality (same truncation-class bug as the now-fixed global model).
- Every other language never gets a
local_archetypeat all.
Currently signal_processor.py just emits a placeholder (local_archetype = None) — nothing crashes, the field is simply always empty outside the 5 covered languages.
Why this is a separate issue, not part of #1158
Fixing this properly needs the same treatment #3061 gave the global model: a retrained, self-describing brain (real FEATURE_NAMES/FEATURE_WEIGHTS/centroids) built from an offline corpus in gitgalaxy-population-analyses, extended to cover meaningfully more than 5 languages. That needs a decent per-language corpus to train against — we don't have enough repo data across the long tail of languages to do that credibly right now. Tracking this separately so it doesn't block closing out the rest of #1158, and so it's picked up once there's enough corpus breadth to retrain against.
Acceptance criteria
- Live per-language feature vector and
SPECIFIC_FILE_INFERENCE_MODELdimensionality agree by construction (same self-describing-brain contract as the global model), not by truncation. - Language coverage meaningfully expanded beyond the current 5 (python/javascript/typescript/c/cpp), gated on adequate corpus size per added language.
-
local_archetypeno longer silentlyNonefor covered languages.
Blocked on: sufficient per-language corpus data to retrain against.
Activity
squid-protocol commentedon Oct 3, 2026
Obsolete: #4106 (PR #4117) retired
SPECIFIC_FILE_INFERENCE_MODELand thelocal_archetypeplaceholders, so there's nothing left to retrain. Bringing back a per-language model would be a new issue, once there is enough corpus per language.