Seeing Isn't Orienting: A Cognitively Grounded Benchmark Reveals Systematic Orientation Failures in MLLMs

DORI addresses a specific, high-value blind spot in current MLLM evaluation: spatial orientation reasoning. While models like GPT-4o and Gemini are excellent at scene description, they frequently fail at fine-grained orientation tasks essential for robotic manipulation. The project is currently in the 'paper release' stage (0 stars, 7 forks, 7 days old), which explains its low defensibility score; benchmarks only gain a moat through community adoption and becoming a 'standard' for leaderboard prestige. Frontier labs are unlikely to build this themselves—they prefer to compete on benchmarks created by academia—but they will likely optimize their models to perform better on DORI if it gains traction. The primary competition comes from broader spatial benchmarks like RealWorldQA (xAI) or MMMU. The 7 forks so soon after release suggest initial interest from the research community, likely peer reviewers or collaborators. The defensibility is low because the methodology can be replicated, but the first-mover advantage in a specific niche like 'orientation perception' provides a temporary intellectual moat.