ClaimDB: A Fact Verification Benchmark over Large Structured Data

ClaimDB addresses a significant gap in current LLM evaluation: the transition from 'Table-QA' (usually single, small tables like WikiTableQuestions) to 'Database-QA' (millions of rows across multiple tables). Its defensibility is currently low (4) because, as a 6-day-old project with zero stars, it lacks the 'researcher gravity' and community adoption required to become a standard like FEVER or TabFact. However, the effort required to curate 80 unique, real-world databases across diverse domains provides a moderate barrier to entry for individual developers. Frontier labs are a 'medium' risk; while they prioritize general reasoning, they are increasingly focused on 'Agentic RAG' and structured data tool-use. They are likely to absorb these datasets into their internal evaluation rigs, potentially making the benchmark obsolete if it doesn't gain rapid academic traction. The primary competition comes from existing benchmarks like UnifiedSKG, TabFact, and Bird-SQL. The displacement horizon is set to 1-2 years, as the field of LLM evaluation moves rapidly toward more dynamic, 'live' web or agentic benchmarks that go beyond static datasets.