Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces

OmniBehavior addresses a critical gap in LLM evaluation: moving from synthetic or narrow-task benchmarks to 'holistic' human simulation. While the project is extremely new (8 days old), the 14 forks against 0 stars suggest it is being actively scrutinized by the academic community, likely following an Arxiv release. Its defensibility is low because benchmarks, while hard to build, are easy to adopt and supersede; the moat is entirely in the 'real-world' dataset quality. Frontier labs (Google, Meta, Apple) pose a high risk here because they sit on the largest troves of real-world human behavioral telemetry (OS logs, app usage, social interaction) and could release much larger-scale versions of this benchmark if they chose to. This project is a 'novel combination' because it integrates cross-scenario data which is typically siloed. It is a necessary tool for the current 'agentic' shift in AI, but it faces rapid obsolescence as more comprehensive industry datasets become the standard for training personal AI agents. Current competitors include AgentBench and older simulation frameworks like Generative Agents, but OmniBehavior's focus on heterogeneous real-world traces gives it a temporary niche.