[2211427551]

Summary

Closes #5306

This PR contributes an independent 1,500-item third-party evaluation report (English & Chinese) and visualization suite for Billion Context (v0.1.202) across three orthogonal long-context and agent memory benchmarks:

1. LongMemEval-S (500 items, ~110k tokens): Multi-session conversational memory, knowledge updates, and temporal reasoning (bc: 79.20%, +16.60% over best RAG 62.60%, -71.1% independent-session cost vs full $2.30 vs $7.97).
2. ContextBench (500 items, 58 repos, Pi CLI v0.73.1): Multi-turn SWE coding agent bug localization (bc: 0.3097 Utilized F1 — 1st place overall, beating full 0.2928 with -58.3% lower cost $4.61 vs $11.05).
3. MemoryAgentBench (500 items, 6k–1.48M tokens): Extreme context scale stress test (bc survives 100% of the 1.48M-token TTL split at 18.58% Recall@10 and 30× steady-state compression where full crashes 199/199, while empirically documenting the verbatim needle retrieval trade-off in AR).

Files Changed

• benchmarks/third-party-1500/README.md & benchmarks/third-party-1500/README.zh-CN.md: Complete bilingual evaluation report, Full Cost Attribution (arXiv:2609.22114) tables, and per-task analysis.
• benchmarks/third-party-1500/*.png:
  • 4 publication-grade 5-arm comparison figures (all_benchmarks_5arms_summary.png, longmemeval_5arms_comparison.png, contextbench_5arms_comparison.png, mab_5arms_comparison.png)
  • 3 plain-language 7-card metric explainers (longmemeval_metric_guide_simple.png, contextbench_metric_guide_simple.png, mab_metric_guide_simple.png)
• README.md & README.zh-CN.md: Added a brief link under the Paper/Preprint section pointing to the bilingual third-party benchmark reports.

[ranxianglei]

thinks for your evaluation !

[github-actions]

📦 Merged Build Published

This merge to master was published as 0.0.943-master.a34a65a (rolling master tag).

npm install -g billion-context@0.0.943-master.a34a65a  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.