Skip to content

feat(engine): add a load-testing harness for iterate-loop under concurrent load - #5781

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
nghetienhiep:fix/issue-5224
Jul 14, 2026
Merged

feat(engine): add a load-testing harness for iterate-loop under concurrent load#5781
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
nghetienhiep:fix/issue-5224

Conversation

@nghetienhiep

Copy link
Copy Markdown
Contributor

Reuses the fake-driver injection seam iterate-loop.test.ts already exercises to
run many simulated tenant attempts through runIterateLoop concurrently, without
spawning a real subprocess or spending API budget, and reports throughput per
concurrency level. Publishes a baseline results doc alongside it.

Closes #5224

…rrent load

Reuses the fake-driver injection seam iterate-loop.test.ts already exercises to
run many simulated tenant attempts through runIterateLoop concurrently, without
spawning a real subprocess or spending API budget, and reports throughput per
concurrency level. Publishes a baseline results doc alongside it.

Closes JSONbored#5224
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.06%. Comparing base (7d59907) to head (050a980).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5781   +/-   ##
=======================================
  Coverage   95.06%   95.06%           
=======================================
  Files         580      580           
  Lines       46156    46156           
  Branches    14804    14804           
=======================================
  Hits        43880    43880           
  Misses       1516     1516           
  Partials      760      760           
Flag Coverage Δ
shard-1 43.93% <ø> (ø)
shard-2 35.77% <ø> (ø)
shard-3 32.26% <ø> (ø)
shard-4 32.95% <ø> (+<0.01%) ⬆️
shard-5 31.75% <ø> (ø)
shard-6 44.63% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 14, 2026
@loopover-orb

loopover-orb Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-14 15:26:20 UTC

5 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This adds a standalone load-testing script for runIterateLoop that reuses the existing fake-driver injection seam, plus a supporting doc and npm script. The harness itself is correct: it batches concurrent attempts, aggregates throughput, and the tests exercise the real exported functions and a real CLI run rather than fabricating unreachable state. It's execution/measurement tooling only — it doesn't touch runIterateLoop's production code path — and closes #5224 as claimed.

Nits — 5 non-blocking
  • packages/loopover-engine/scripts/load-test-iterate-loop.mjs:139 uses console.log directly in the CLI entrypoint; fine for a script but worth a lint-disable comment or noting it's intentional given repo conventions elsewhere.
  • packages/loopover-engine/scripts/load-test-iterate-loop.mjs:60-61 hardcodes maxIterations: 3 and maxTurnsPerIteration: 20 with no named constant explaining why those values were chosen for the synthetic attempt.
  • The reported 'depth 5' nesting in load-test-iterate-loop.mjs:115 is really just an arrow-function body inside runConcurrencyLevel's for-loop plus Promise.all/Array.from — readable as written, not worth restructuring.
  • docs/iterate-loop-load-test.md's baseline numbers are machine-dependent and will drift; consider marking them clearly as illustrative only (already partially done) so nobody treats them as a regression gate.
  • Consider extracting maxIterations/maxTurnsPerIteration in load-test-iterate-loop.mjs:60-61 into named constants alongside SYNTHETIC_ISSUE_NUMBER for consistency.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #5224
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 11 registered-repo PR(s), 6 merged, 0 issue(s).
Contributor context ✅ Confirmed Gittensor contributor nghetienhiep; Gittensor profile; 11 PR(s), 0 issue(s).
Gate result ✅ Passing No configured blocker found.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence. LLM value judgment: moderate — It's a well-scoped, non-production-risk addition (pure tooling + docs) that gives the team real signal on iterate-loop's concurrent-scheduling overhead, directly closing the linked issue without touching runIterateLoop itself.
Linked issue satisfaction

Addressed
The PR adds a checked-in load-testing harness that reuses the fake-driver injection seam to run concurrent runIterateLoop attempts without real API spend, and publishes a baseline results document with concurrency/throughput numbers, matching both deliverables and boundaries in the issue.

Review context
  • Author: nghetienhiep
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 11 PR(s), 0 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 4b6b5c6 into JSONbored:main Jul 14, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Execution: load-testing harness and published capacity numbers for AMS's iterate-loop under concurrent load

1 participant