Repository navigation
fix(arxiv-researcher): optimize token usage and raise max-ai-credits - #49965
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…ts limit Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
@copilot use sub agents with smaller models as much as possible to filter |
…ore large-model extractor Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Added a The updated pipeline is now:
Both cheap small-model stages run before any large-model call, so the expensive extractor is only invoked for the highest-value papers. |
There was a problem hiding this comment.
Pull request overview
Optimizes the daily arXiv workflow’s AI-credit usage and increases its budget ceiling.
Changes:
- Reduces fetched papers and abstract size.
- Adds small-model ranking before extracting the top three opportunities.
- Lazily loads the discussion template and raises the budget to 300 AIC.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/daily-arxiv-researcher.md |
Updates limits, ranking, template loading, and budget. |
.github/workflows/daily-arxiv-researcher.lock.yml |
Regenerates the compiled workflow. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Suppressed comments (2)
.github/workflows/daily-arxiv-researcher.md:164
- The PR description says the extractor cap changes from 8 to 5, but this implementation adds an undocumented ranking stage and ultimately caps extraction at 3. That materially changes both output coverage and the cost estimate; either implement the documented five-paper cap or update the PR description and estimates to describe the ranker/top-three design.
For each relevant paper, invoke the `relevance-ranker` sub-agent with:
.github/workflows/daily-arxiv-researcher.md:169
- Keeping only three ranked papers leaves the remaining relevant papers without opportunity or area values, but Step 4 still requires those fields for every paper marked relevant (lines 188–197). Runs with more than three relevant papers therefore cannot satisfy the ledger instructions without inventing data. Track “selected for extraction” separately and define how unselected relevant papers are recorded, or update the ledger schema accordingly.
Sort ranked papers by `score` descending. Keep only the top 3.
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Balanced
| fs.mkdirSync(BASE_DIR, { recursive: true }); | ||
|
|
||
| const ARXIV_URL = 'https://export.arxiv.org/api/query?search_query=(cat:cs.AI+OR+cat:cs.SE+OR+cat:cs.LG)+AND+(agentic+OR+%22multi-agent%22+OR+%22llm+agent%22+OR+%22workflow+automation%22+OR+%22code+generation%22+OR+%22ai+agent%22)&max_results=40&sortBy=submittedDate&sortOrder=descending'; | ||
| const ARXIV_URL = 'https://export.arxiv.org/api/query?search_query=(cat:cs.AI+OR+cat:cs.SE+OR+cat:cs.LG)+AND+(agentic+OR+%22multi-agent%22+OR+%22llm+agent%22+OR+%22workflow+automation%22+OR+%22code+generation%22+OR+%22ai+agent%22)&max_results=25&sortBy=submittedDate&sortOrder=descending'; |
|
🎉 This pull request is included in a new release. Release: |
The daily arXiv researcher workflow consistently hits the 250 AIC budget (253.7 used on last run), triggering an AI-credits guardrail failure.
Token optimizations
paper-screener(model: small) gets a smaller contextopportunity-extractorcalls: max relevant papers capped 8 → 5; this is the most expensive step (model: large), so this is the highest-leverage reduction (~37% worst-case cost)max_resultsreduced 40 → 25; directly reduces screener fan-out## skill: discussion-templateblock — the runtime only injects it when the agent explicitly retrieves the skill, keeping early turns (Steps 1–4) leanBudget
max-ai-creditsraised 250 → 300 (20% headroom after the optimizations above).