PaperScope: A Multi-Modal Multi-Document Benchmark for Agentic Deep Research Across Massive Scientific Papers

PaperScope addresses a critical gap in AI evaluation: the transition from single-document RAG to 'Deep Research' agents that must synthesize information across multiple multi-modal PDFs. While the project is extremely new (4 days old) and currently lacks public traction (0 stars), its value lies in the data curation of multi-paper reasoning chains which are significantly harder to automate than single-doc Q&A. However, it faces high frontier-lab risk as companies like OpenAI (with their 'Deep Research' model) and Google (with NotebookLM/Gemini) are internally developing similar evaluation sets to refine their flagship models. The defensibility is low because benchmarks are easily superseded by newer, larger datasets or 'official' benchmarks from established entities like the Allen Institute for AI (AI2) or major labs. The '5 forks' suggests early interest from the research community, likely associated with the paper's authors. Its long-term survival depends on whether it can become a community-accepted leaderboard standard before a platform-native benchmark from Google Scholar or Semantic Scholar emerges.