feat(enrichment): treat ML model/checkpoint weights as binary assets - #3204
Conversation
The asset-weight analyzer flags heavy binary blobs whose byte sizes never appear in the textual diff, but its extension set covered images/fonts/media/ archives and missed serialized ML model and checkpoint weights — safetensors, gguf, onnx, pt, pth, ckpt. These are among the heaviest artifacts a PR can commit (routinely hundreds of MB to multi-GB), so a PR that adds or grows one slipped past the size-bloat signal. Add them as one centralized group. The lookup already lowercases and matches only the final extension, so casing and compound names are handled by the existing path. src/review/rag.ts already classifies these six as binary; this brings the size analyzer in line. The 100 KB flag threshold means a small text `.pth` path-config file is never weighed, so including `.pth` cannot produce a false finding.
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
|
Warning 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 ⏸️ Gittensory review result - manual review recommendedReview updated: 2026-07-04 19:52:19 UTC
⏸️ Suggested Action - Manual Review Review summary Nits — 5 non-blocking
Review context
Contributor next steps
Signal definitions
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
Summary
The
asset-weightanalyzer flags heavy binary blobs whose byte sizes never appear in the textual diff (they show as "Binary files differ"). Its extension set already covers images, fonts, media, and archives, but missed serialized ML model and checkpoint weight formats —safetensors,gguf,onnx,pt,pth,ckpt. These are among the heaviest artifacts a PR can commit (routinely hundreds of MB to multi-GB), so a PR that added or grew one slipped past the size-bloat signal entirely.This adds the six formats as one centralized group in
BINARY_EXTS:.SAFETENSORS) and compound names are handled by the existing path — no new logic.src/review/rag.ts'sBINARY_EXT_REalready classifies all six as binary, so the repo already treats them as binary assets — this brings the size analyzer in line with that existing decision..pthsafety:.pthis normally a PyTorch weight file but can also be a small Python site-packages path-config (text). The analyzer only flags blobs ≥ 100 KB, so a tiny text.pthis never weighed — including it cannot produce a false finding, and it matchesrag.ts.No linked issue
This is a no-issue PR by design: a self-contained detection-coverage improvement that adds six related extensions to the
asset-weightanalyzer'sBINARY_EXTSset plus unit tests, touching onlyreview-enrichment/. There is no behavior change beyond recognizing the new extensions, so no tracking issue is needed. It mirrors the accepted no-issue precedent for the very same file —webp/avif/heic/heifand the recently-mergedzst(#3128).Scope
type(scope): short summaryConventional Commit format.CONTRIBUTING.mdand does not reintroduce GitHub Pages, VitePress,site/, orCNAME.Validation
git diff --checknpm run actionlintnpm run typechecknpm run test:coverage— nosrc/**lines changed (this change is underreview-enrichment/, which Codecov does not measure), socodecov/patchhas no diff to gate; suite is green.npm run test:workersnpm run build:mcpnpm run test:mcp-packnpm run ui:openapi:checknpm run ui:lintnpm run ui:typechecknpm run ui:buildnpm audit --audit-level=moderate.safetensors/.gguf/.onnx/.pt/.pth/.ckptand an uppercase.SAFETENSORScase are added to theisBinaryAssetunit test, in the same assertion cluster;npm run rees:testpasses (all review-enrichment tests green, analyzer-metadata check clean).Validated green against the full GitHub CI
validate-codecheck set —actionlint,db:migrations:check,db:schema-drift:check,cf-typegen:check,selfhost:validate-observability,typecheck,test:coverage,test:workers,build:mcp,test:mcp-pack,build:miner,rees:test(689 tests pass, analyzer-metadata check clean),ui:openapi:check,ui:openapi:settings-parity,ui:version-audit,ui:lint,ui:typecheck,ui:test,ui:build. Branch rebased on latestmain.Safety
Notes
Analogues followed end-to-end: the merged
feat(enrichment): treat HEIC/HEIF images as binary assets(#3089),feat(enrichment): treat Zstandard (.zst) archives as binary assets(#3128), and the original asset-weight analyzer (#1621) — same file, same additive shape, same test pattern. Data/array formats (npy/npz/parquet/sqlite) are intentionally left out of scope here as a separate concern.