Skip to content

fix(db): raise the stored PR/issue body cap from 4000 to GitHub's own 65536-char limit - #4579

Merged
JSONbored merged 2 commits into
mainfrom
fix/pr-body-truncation-4682
Jul 10, 2026
Merged

fix(db): raise the stored PR/issue body cap from 4000 to GitHub's own 65536-char limit#4579
JSONbored merged 2 commits into
mainfrom
fix/pr-body-truncation-4682

Conversation

@JSONbored

@JSONbored JSONbored commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Summary

  • compactGitHubPayload (shared by both PR and issue sync) silently truncated any body over 4000 chars before storing it, with zero indication anything was cut.
  • Every body-content check reads the stored copy, not a live GitHub fetch — screenshotTableGate's viewport/theme matrix parser, linked-issue satisfaction, slop keyword matching.
  • Confirmed live on metagraphed#4682: a genuinely complete, correctly-formatted 12-image before/after table (5160 chars) got closed for "missing before/after screenshot table" because only the first ~4000 chars (one row) were ever persisted. The AI reviewer's own commentary independently flagged the same truncation ("the before/after screenshot table is truncated... past the Desktop·Light row") — it was reading the same corrupted stored copy.
  • New cap (65536) matches GitHub's own documented issue/PR body limit, so it can only ever bind on content GitHub itself was never going to accept.
  • Added github_app.body_truncated_on_store structured logging so a future occurrence (if GitHub ever raises its own limit) is immediately visible instead of requiring manual DB archaeology, which is how this one was found.
  • Also includes an unrelated but blocking fix: renumbers the two colliding 0134_* migrations (three PRs independently grabbed 0134 within ~90 minutes) — verified against the production migrations table that only the first-merged one was actually applied, so the other two were safe to renumber to 0136/0137. This was blocking db:migrations:check fleet-wide.

Test plan

  • New regression tests: a body just past the old 4000 cap round-trips in full (mirrors the real feat(scripts): widen docs-drift checker to the full config surface #4682 length), a body past GitHub's 65536 limit is still capped there, the same fix covers issue bodies (shared code path), and the new truncation-trace log fires (and only fires) exactly when truncation actually happens
  • Updated one existing test that had pinned the old 4000-char truncation behavior
  • npm run typecheck, full npm run test:coverage, npm run test:ci (all green, exit 0)

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.12%. Comparing base (d1cb719) to head (7b78a04).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4579   +/-   ##
=======================================
  Coverage   94.12%   94.12%           
=======================================
  Files         430      430           
  Lines       38155    38159    +4     
  Branches    13912    13914    +2     
=======================================
+ Hits        35913    35917    +4     
  Misses       1585     1585           
  Partials      657      657           
Files with missing lines Coverage Δ
src/db/repositories.ts 96.61% <100.00%> (+<0.01%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 10, 2026
@loopover-orb

loopover-orb Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Caution

🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

🛑 Gittensory review result - fixes required

Review updated: 2026-07-10 05:27:11 UTC

3 files · 1 AI reviewer · 2 blockers · readiness 93/100 · CI failing · blocked

🛑 Suggested Action - Manual Review

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

Review summary
This raises MAX_STORED_BODY_CHARS in src/db/repositories.ts from 4000 to 65536 (GitHub's documented issue/PR body limit), fixing silent truncation that corrupted every downstream body-content check (screenshotTableGate's matrix parser, linked-issue satisfaction, slop matching) since they all read the stored copy rather than a live fetch. The fix is applied at the correct layer — the single shared constant used by compactGitHubPayload for both PR and issue sync — rather than patching a symptom in one of the consuming gates. Tests exercise the real upsertPullRequestFromGitHub/upsertIssueFromGitHub write paths and a real read-back via getPullRequest/listIssues, and the pre-existing data-spine test's hardcoded 4000-char truncation assertion is correctly updated to 5000 to match the new cap, so nothing here is a fabricated or can't-occur scenario.

Nits — 5 non-blocking
  • src/db/repositories.ts: the new cap is a raw string-length count, not a byte count — if downstream storage (D1/SQLite TEXT) or any serialization step has its own byte-size ceiling, 65536 UTF-16 chars could still exceed it for multi-byte bodies; worth a one-line confirmation this isn't relevant here.
  • test/unit/backfill.test.ts: the new `getPullRequest` import is inserted out of the existing alphabetical ordering in the import list (before `upsertPullRequestFile`) — minor style nit, verify the file's convention.
  • PR description doesn't link to an eligible open issue in this repo (only references an external metagraphed#4682 as evidence) — worth confirming there's a corresponding gittensory issue this closes, even though this is a narrow, clearly-motivated bug fix rather than scope creep.
  • Consider auditing compactGitHubPayload for other truncation caps on adjacent fields (e.g. comment bodies) that might have the same GitHub-limit-mismatch problem this PR just fixed for the main body field.
  • Add a brief comment or test noting whether other body-consuming fields (review comments, commit messages) share this same truncation risk, so a follow-up PR has a pointer if needed.

Why this is blocked

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

CI checks failing

  • validate
  • validate-code
Signal Result Evidence
Code review ❌ 2 blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 48 registered-repo PR(s), 40 merged, 353 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 48 PR(s), 353 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 48 PR(s), 353 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 10, 2026
JSONbored added 2 commits July 9, 2026 22:43
… 65536-char limit

compactGitHubPayload silently truncated any PR/issue body over 4000 chars before
storing it -- with zero indication anything was cut -- and every body-content
check (screenshotTableGate's viewport/theme matrix parser, linked-issue
satisfaction, slop keyword matching) reads the stored copy, not a live GitHub
fetch. A body just past that threshold reads as "missing" evidence even when
the contributor's submission was genuinely complete.

Confirmed live on metagraphed#4682: a real, correctly-formatted 12-image
before/after table (5160 chars) got closed for "missing before/after
screenshot table" because only the first ~4000 chars (one row) were ever
persisted. The new cap matches GitHub's own issue/PR body limit, so it can
only ever bind on content GitHub itself was never going to accept.
…s truncated

The #4682 incident's entire failure mode was silence -- the old 4000-char cap
quietly corrupted screenshotTableGate's matrix parser (and every other
body-content check) for an unknown period with zero log line, audit row, or
any other trace. The cap is now GitHub's own 65536-char limit so this should
never fire in practice, but if it ever does again, github_app.body_truncated_on_store
makes it immediately greppable instead of requiring manual DB archaeology.
@JSONbored
JSONbored force-pushed the fix/pr-body-truncation-4682 branch from 5fe7ccc to 7b78a04 Compare July 10, 2026 05:43
@JSONbored
JSONbored merged commit ddb30f4 into main Jul 10, 2026
7 checks passed
@JSONbored
JSONbored deleted the fix/pr-body-truncation-4682 branch July 10, 2026 05:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

1 participant