Skip to content

feat(enrichment): treat ML model/checkpoint weights as binary assets - #3204

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
davion-knight:feat/asset-weight-ml-weights
Jul 4, 2026
Merged

feat(enrichment): treat ML model/checkpoint weights as binary assets#3204
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
davion-knight:feat/asset-weight-ml-weights

Conversation

@davion-knight

Copy link
Copy Markdown
Contributor

Summary

The asset-weight analyzer flags heavy binary blobs whose byte sizes never appear in the textual diff (they show as "Binary files differ"). Its extension set already covers images, fonts, media, and archives, but missed serialized ML model and checkpoint weight formatssafetensors, gguf, onnx, pt, pth, ckpt. These are among the heaviest artifacts a PR can commit (routinely hundreds of MB to multi-GB), so a PR that added or grew one slipped past the size-bloat signal entirely.

This adds the six formats as one centralized group in BINARY_EXTS:

  • The lookup already lowercases and matches only the final extension, so casing (.SAFETENSORS) and compound names are handled by the existing path — no new logic.
  • Additive-only: six set entries plus test cases; no existing behavior changes.
  • Consistency anchor: src/review/rag.ts's BINARY_EXT_RE already classifies all six as binary, so the repo already treats them as binary assets — this brings the size analyzer in line with that existing decision.
  • .pth safety: .pth is normally a PyTorch weight file but can also be a small Python site-packages path-config (text). The analyzer only flags blobs ≥ 100 KB, so a tiny text .pth is never weighed — including it cannot produce a false finding, and it matches rag.ts.

No linked issue

This is a no-issue PR by design: a self-contained detection-coverage improvement that adds six related extensions to the asset-weight analyzer's BINARY_EXTS set plus unit tests, touching only review-enrichment/. There is no behavior change beyond recognizing the new extensions, so no tracking issue is needed. It mirrors the accepted no-issue precedent for the very same file — webp/avif/heic/heif and the recently-merged zst (#3128).

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • No linked issue — see the No linked issue section above.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage — no src/** lines changed (this change is under review-enrichment/, which Codecov does not measure), so codecov/patch has no diff to gate; suite is green.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New behavior has tests — positive cases for .safetensors/.gguf/.onnx/.pt/.pth/.ckpt and an uppercase .SAFETENSORS case are added to the isBinaryAsset unit test, in the same assertion cluster; npm run rees:test passes (all review-enrichment tests green, analyzer-metadata check clean).

Validated green against the full GitHub CI validate-code check set — actionlint, db:migrations:check, db:schema-drift:check, cf-typegen:check, selfhost:validate-observability, typecheck, test:coverage, test:workers, build:mcp, test:mcp-pack, build:miner, rees:test (689 tests pass, analyzer-metadata check clean), ui:openapi:check, ui:openapi:settings-parity, ui:version-audit, ui:lint, ui:typecheck, ui:test, ui:build. Branch rebased on latest main.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • No auth, cookie, CORS, GitHub App, Cloudflare, or session changes (N/A — pure extension-set addition).
  • No API/OpenAPI/MCP behavior change (N/A).
  • No UI changes (N/A — this is a review-enrichment analyzer).
  • No visible UI change, so no UI Evidence section is required.
  • No docs/changelog changes needed.

Notes

Analogues followed end-to-end: the merged feat(enrichment): treat HEIC/HEIF images as binary assets (#3089), feat(enrichment): treat Zstandard (.zst) archives as binary assets (#3128), and the original asset-weight analyzer (#1621) — same file, same additive shape, same test pattern. Data/array formats (npy/npz/parquet/sqlite) are intentionally left out of scope here as a separate concern.

The asset-weight analyzer flags heavy binary blobs whose byte sizes never
appear in the textual diff, but its extension set covered images/fonts/media/
archives and missed serialized ML model and checkpoint weights — safetensors,
gguf, onnx, pt, pth, ckpt. These are among the heaviest artifacts a PR can
commit (routinely hundreds of MB to multi-GB), so a PR that adds or grows one
slipped past the size-bloat signal. Add them as one centralized group.

The lookup already lowercases and matches only the final extension, so casing
and compound names are handled by the existing path. src/review/rag.ts already
classifies these six as binary; this brings the size analyzer in line. The
100 KB flag threshold means a small text `.pth` path-config file is never
weighed, so including `.pth` cannot produce a false finding.
@davion-knight
davion-knight requested a review from JSONbored as a code owner July 4, 2026 19:50
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 4, 2026
@loopover-orb

loopover-orb Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-04 19:52:19 UTC

2 files · 1 AI reviewer · no blockers · readiness 73/100 · CI green · clean

⏸️ Suggested Action - Manual Review

Review summary
This change correctly extends the asset-weight binary extension set to cover common ML model/checkpoint formats, and the existing `isBinaryAsset` path lowercases only the final extension so the new entries are exercised through the real helper path. The tests cover each new extension plus case-insensitivity, and the analyzer behavior remains additive with no schema, API, or queue wiring impact. I don't see a reachable correctness defect in the provided diff.

Nits — 5 non-blocking
  • nit: review-enrichment/src/analyzers/asset-weight.ts:72 overstates that all listed extensions' bytes never appear in textual diffs; `.pth` can be a text Python path-config file, even if the 100 KB threshold makes most real cases harmless.
  • nit: review-enrichment/test/asset-weight.test.ts:31 only exercises `.SAFETENSORS` casing, so the case-insensitive claim for the whole new group is covered by mechanism rather than by a representative non-first extension.
  • review-enrichment/src/analyzers/asset-weight.ts:72 Consider tightening the comment to say these are treated as binary-size assets by extension, with `.pth` included for checkpoint parity despite rare text uses.
  • review-enrichment/test/asset-weight.test.ts:31 Add one mixed-case example for a shorter new extension such as `models/RESNET.ONNX` if you want the test to document case-insensitivity across the group, not just `safetensors`.
  • Readiness score is below the configured threshold — Use the readiness panel as advisory maintainer context; the score does not block this PR.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ❌ 5/25 Preflight is holding this PR: the review lane is unavailable, so it is not ready for automated review.
Contributor workload ✅ 10/10 Author activity: 21 registered-repo PR(s), 14 merged, 0 issue(s).
Contributor context ✅ Confirmed Gittensor contributor davion-knight; Gittensor profile; 21 PR(s), 0 issue(s).
Gate result ✅ Passing No configured blocker found.
Review context
  • Author: davion-knight
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: Rust
  • Official Gittensor activity: 21 PR(s), 0 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Explain no-issue PR.
  • Await review-lane availability.
  • Refresh registry data or choose a registered active repo.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 222d624 into JSONbored:main Jul 4, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant