Skip to content

About

Artificial Analysis Intelligence Index vs parameter count for open-weights LLMs, plus a guide to what you can run locally. Unofficial.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Latest commit

 

History

31 Commits

Folders and files

Repository files navigation

Open-weights: intelligence vs size

Artificial Analysis scores open-weights models on its Intelligence Index, but plots it against price and speed, not size. This site plots the index against parameter count for every open-weights model it scores, then shows which ones your GPU, RAM or unified memory can run. Unofficial: not affiliated with Artificial Analysis or Hugging Face.

License Live Demo Deploy Data: Artificial Analysis

Intelligence vs size · What fits your hardware · Refreshed every 3 days

Artificial Analysis Intelligence Index against parameter count for open-weights models, with Claude Opus and GPT Astra as dashed reference lines

What it shows

  • Intelligence vs parameters. The Intelligence Index against total or active parameters, log or linear. The dashed line is the frontier: no smaller model scores higher. Frontier models are always shown, whichever model set you pick. A MoE's active parameters set its speed, its total parameters set its memory.
  • Lookup-table models (n-gram or per-layer embedding tables, such as Qwen3.8-Flash-Next) appear as two linked points: the ring is the size without the tables, the filled dot the full size. The tables take memory but almost no bandwidth, so they can sit on SSD. The frontier uses the ring.
  • Closed models. They publish no sizes, so the optional "Closed models" switch draws the latest model of each product tier at the leading closed labs as dashed score lines.
  • Two Intelligence Index versions. Artificial Analysis replaced its index in late October 2026 (different components, roughly 0.5–0.7× the old scale) and has scored only a minority of older models on the new one. The default view is the old index, a frozen snapshot of 2026-10-25 (data/models_v1.json, never regenerated); the toggle switches to the new index (data/models.json, refreshed every 3 days, far fewer models). The scales are not comparable. On the v2 side, an "Estimate unscored" button adds the models Artificial Analysis has not scored on v2, with a score projected from their v1 score (least-squares v2 ≈ a·v1 + b·v1² over the models scored on both; residual SD about 1.2 index points, refitted on every refresh). Projected points are faint and dashed, frontier segments that rest on one are drawn fainter, they are never offered as a "what should I run" pick, and they are replaced by the real score as soon as Artificial Analysis publishes one.
  • Model sets. Featured (hand-curated), Recent, Artificial Analysis's own picks, All, or choose models one by one.

What should I run?

Hardware picker, a quadrant of coding versus knowledge strength against speed, and top picks for the chosen hardware
  • Pick your hardware: GPU only, GPU plus system RAM, or unified memory; total VRAM, RAM size and precision. Weights fill VRAM, then RAM, then SSD if you allow it. A model counts as usable if a bandwidth estimate gives it a few tokens per second; the estimate is rough and not shown.
  • Precision. On RTX 5000 / Blackwell, "4-bit" means NVFP4, elsewhere GGUF Q4_K_M. Artificial Analysis scored hosted endpoints, so quantized local copies will score lower.
  • Quadrant. Horizontal: coding and agentic percentile minus knowledge and instruction-following percentile (Artificial Analysis has no creative-writing eval). Vertical: speed, approximated by fewer active parameters. Dot size: Intelligence Index. A ring marks models that nothing beats on both speed and intelligence.

Where the data comes from

What Source
Scores, release dates, total and active parameters, model-picker defaults Artificial Analysis leaderboard and model pages
Exact total sizes, stored sizes, repos, licenses Hugging Face (safetensors headers); AA's figure is kept where the two differ by more than 8%, with a note
Lookup-table sizes, the Featured set Hand-curated: data/overrides.json, data/curated.json
Hardware capacities and bandwidths Rough representative values in assets/hardware.js

A GitHub Action refreshes data/ every three days and redeploys. It refuses to publish if a scrape returns under 80% of the previous model count, and its log lists recent open models that are not in the Featured set.

Getting started

No build step: a static site, with stdlib-only Python for the data refresh.

python scripts/update.py        # refresh data (optional)
python -m http.server 8000      # then open http://localhost:8000

Open these via the local server; each prints PASS or FAIL:

  • tests/core.test.html: fit and placement maths
  • tests/ui.test.html: interactions
  • tests/responsive.test.html: no overflow from 320 to 2560 px
index.html, assets/          the site (core.js: fit logic, hardware.js: presets, app.js: UI)
data/                        models.json (generated), curated.json and overrides.json (hand-edited), caches
docs/screenshots/            README images
scripts/                     scrape_aa.py, hf.py, update.py
.github/workflows/pages.yml  refresh and deploy

Corrections and curation

  • Wrong parameter count or missing lookup-table size: edit data/overrides.json (keyed by Artificial Analysis family id, for example qwen3-8-flash-next).
  • Featured set and picker order: edit data/curated.json.
  • Hardware numbers: assets/hardware.js. Fit logic: assets/core.js.

Pull requests welcome.

Support

Free and open source. If it saved you time, you can chip in toward its upkeep.

Ko-fi Liberapay PayPal Crypto donation button by NOWPayments

Stars, bug reports (wrong sizes, missing models) and suggestions for the Featured list help just as much.

License and credits

The code and hand-curated files in this repository are MIT licensed. Benchmark data belongs to Artificial Analysis; sizes and model metadata come from Hugging Face. Please credit them if you reuse it, and follow their terms.

About

Artificial Analysis Intelligence Index vs parameter count for open-weights LLMs, plus a guide to what you can run locally. Unofficial.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Sponsor this project

Packages

Contributors

Languages