Skip to content

Census 2026-09-27 - #171

Merged
bong-water-water-bong merged 1 commit into
mainfrom
census/2026-09-27
Sep 27, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
census/2026-09-27

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

415695 text-generation models on HF, 332726 with an architecture (2616 architectures).

models share
mapped (a backend's code accepts the architecture) 315701 94.88%
checked (passed serve_e2e on Strix Halo) 214033 64.33%
backend mapped checked
vulkan 94.88% 64.33%
hrx 94.78% 64.32%
zinc 69.83% 10.28%
npu 9.29% 9.29%

Unmapped architectures with the most models:

  • Step1MoEForCausalLM: 2882
  • ParlerTTSForConditionalGeneration: 1586
  • ChessForCausalLM: 970
  • LlavaLlamaForCausalLM: 451
  • DynamicAlibiForCausalLM: 245
  • OpenLMForCausalLM: 245
  • DynamicForgettingForCausalLM: 240
  • BertLMHeadModel: 212
  • MobileLLMForCausalLM: 176
  • BartForConditionalGeneration: 174

…rchitectures): mapped 315701 (94.88%), checked 214033 (64.33%)
@context7

context7 Bot commented Sep 27, 2026

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan. —

Commit a012d0b

@bong-water-water-bong
bong-water-water-bong merged commit 7e5c6fe into main Sep 27, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the census/2026-09-27 branch September 27, 2026 10:59
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Possible Performance Issue

The PR introduces a new census of models with various metrics. One potential issue is that the "checked_pct" is not consistent with other metrics in some parts of the JSON. For example, in the first hunk, the "checked_pct" is 64.33%, but in the old hunk, the "checked_pct" is 64.19%. This might indicate a data inconsistency or a bug in the code that calculates or populates these values.

{
 "date": "2026-09-27",
 "coverage": {
  "models": 415695,
  "with_architecture": 332726,
  "architectures_seen": 2616,
  "mapped_models": 315701,
  "mapped_pct": 94.88,
  "checked_models": 214033,
  "checked_pct": 64.33,
  "per_backend": {
   "vulkan": {
    "mapped": 315701,
    "checked": 214033,
    "mapped_pct": 94.88,
    "checked_pct": 64.33
   },
   "hrx": {
    "mapped": 315374,
    "checked": 214016,
    "mapped_pct": 94.78,
    "checked_pct": 64.32
   },
   "zinc": {
    "mapped": 232345,
    "checked": 34214,
    "mapped_pct": 69.83,
    "checked_pct": 10.28
   },
   "npu": {
    "mapped": 30900,
    "checked": 30900,
    "mapped_pct": 9.29,
    "checked_pct": 9.29
   }
  },
Possible Data Integrity Issue

There are discrepancies in the "checked_pct" values between the old and new hunks in the same section. For example, in the per_backend section, the Vulkan backend has "checked_pct": 64.33% in the new hunk and 64.19% in the old hunk. This could indicate a problem with how the data is being collected or updated.

"per_backend": {
 "vulkan": {
  "mapped": 315701,
  "checked": 214033,
  "mapped_pct": 94.88,
  "checked_pct": 64.33
Possible Data Integrity Issue

The "checked_models" and "checked_pct" fields do not align correctly in some parts of the JSON. For example, in the first hunk, "checked_models": 214033 and "checked_pct": 64.33%. However, in the old hunk, the same metrics are "checked_models": 213516 and "checked_pct": 64.19%. This inconsistency might suggest a data entry error or a logical error in the code.

  "mapped_pct": 9.29,
  "checked_pct": 9.29
 }
},
"top_unmapped": [
 {
  "architecture": "Step1MoEForCausalLM",
  "models": 2882
 },
 {
  "architecture": "ParlerTTSForConditionalGeneration",
  "models": 1586
Possible Data Integrity Issue

The "checked_pct" field in the top_unmapped architectures section is inconsistent with the "checked_pct" field in the main section. In the main section, "checked_pct": 9.29, but in the top_unmapped section, there is no "checked_pct" value provided. This might indicate that the data in the top_unmapped section is incomplete or there is an error in the aggregation.

},
{
 "architecture": "ParlerTTSForConditionalGeneration",
 "models": 1586
},
{
 "architecture": "ChessForCausalLM",
 "models": 970
},
{
 "architecture": "LlavaLlamaForCausalLM",
 "models": 451
},
Possible Data Integrity Issue

The "checked_pct" field in the raw counts section does not match the "checked_pct" field in the main section. For example, in the raw counts, "checked_pct": 10.28 for Zinc, but in the main section, the overall checked_pct is 64.33%. This inconsistency could indicate a problem with how the data is being categorized or updated.

"raw": {
 "total": 415695,
 "no_arch": 82969,
 "pages": 416,
 "complete": true,
 "counts": {
  "LlamaForCausalLM": 100476,
  "Qwen2ForCausalLM": 46501,
  "GPT2LMHeadModel": 31746,
  "Qwen3ForCausalLM": 30900,
  "MistralForCausalLM": 27915,
  "GPTNeoXForCausalLM": 8056,
  "StableLmForCausalLM": 6681,
  "Gemma3ForCausalLM": 6073,
  "Gemma2ForCausalLM": 5580,
  "GemmaForCausalLM": 5518,
  "Phi3ForCausalLM": 3224,
  "Qwen3_5ForConditionalGeneration": 3052,
  "Step1MoEForCausalLM": 2882,
Possible Data Integrity Issue

The "checked_pct" field in the per_backend section for Zinc is 10.28, but in the top_unmapped architectures, there is no corresponding "checked_pct" value. This might indicate that the top_unmapped architectures have not been checked or there is an error in the reporting.

"OpenLMForCausalLM": 245,
"DynamicForgettingForCausalLM": 240,
"Gemma4UnifiedForConditionalGeneration": 235,
"SeedOssForCausalLM": 227,
"Glm4ForCausalLM": 224,
"BertLMHeadModel": 212,
"Qwen3_5MoeForCausalLM": 197,
"Mistral3ForConditionalGeneration": 196,
"OlmoeForCausalLM": 190,
"ExaoneForCausalLM": 189,
"Cohere2ForCausalLM": 183,
Possible Data Integrity Issue

The "checked_pct" field in the per_backend section for NPUs is 9.29, but in the main section, the overall checked_pct is 64.33%. This discrepancy could indicate that the NPUs are either not included in the main section or there is an error in the calculation of checked_pct for different backends.

"FalconH1ForCausalLM": 163,
"MambaForCausalLM": 158,
"MixFormerSequentialForCausalLM": 156,
"LLaMAForCausalLM": 153,
"RobertaForCausalLM": 139,
"ApertusForCausalLM": 136,
"BaichuanForCausalLM": 131,
"DeciLMForCausalLM": 129,
"DFlashDraftModel": 128,
"LlamaForCausalLMEagle3": 126,
"NanochatGPTForCausalLM": 121,
Possible Data Integrity Issue

The "checked_pct" field in the per_backend section for NVidia GPUs (if present) is missing, but present in the main section. This might indicate that the data for certain backends is incomplete or not properly aggregated.

"Llama4ForCausalLM": 81,
"DynamicSlidingWindowForCausalLM": 79,
"K2HorizonForCausalLM": 79,
Possible Data Integrity Issue

The "checked_pct" field in the per_backend section for some backends (e.g., Vulkan, HRX, Zinc) is present, but the corresponding values in the old hunk do not match. This suggests that the new data might have been updated incorrectly, or there is a mismatch in how the data is stored.

"AfmoeForCausalLM": 83,
"Llama4ForCausalLM": 81,
"DynamicSlidingWindowForCausalLM": 79,
"K2HorizonForCausalLM": 79,
"XGLMForCausalLM": 79,
"MiMoV2ForCausalLM": 78,
"Glm5NextForConditionalGeneration": 77,
"MarianForCausalLM": 76,
"LISAForCausalLM": 74,
"NMMaskMoELLaVAQwen3ForCausalLM": 73,
"KORMoForCausalLM": 72,
"NanoChatForCausalLM": 71,
"TransformerForCausalLM": 68,
"Qwen4ExpForConditionalGeneration": 67,
Possible Data Integrity Issue

The "checked_pct" field in the per_backend section for some backends is inconsistent with the overall checked_pct. For example, Vulkan has 64.33% checked, but the overall checked_pct is 64.33%, which might be coincidental or indicate a specific issue with Vulkan's checked count.

 "checked_pct": 64.33
},
"hrx": {
 "mapped": 315374,
 "checked": 214016,
 "mapped_pct": 94.78,
 "checked_pct": 64.32
Possible Data Integrity Issue

The "checked_pct" in the per_backend section for Vulkan is 64.33%, which matches the overall checked_pct. This might be intentional, but it's worth verifying if these values are meant to be the same or if there is a logical error in how they are calculated.

"vulkan": {
 "mapped": 315701,
 "checked": 214033,
 "mapped_pct": 94.88,
 "checked_pct": 64.33

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant