SUPERGLASSES: Benchmarking Vision Language Models as Intelligent Agents for AI Smart Glasses

SUPERGLASSES addresses a critical gap in the VLM evaluation landscape: the shift from static, general-purpose image analysis to the dynamic, egocentric, and knowledge-dependent context of smart glasses. With 0 stars but 7 forks within just 8 days of release, it shows immediate interest from the research community (likely academic peers of the paper authors). However, the defensibility is low (score 3) because benchmarks only survive through mass adoption and becoming a 'standard' (e.g., ImageNet, MMLU). Currently, it lacks the network effects or massive scale required to fend off competitors. The platform domination risk is high because Meta (with Ray-Ban Meta glasses) and Google (Project Astra) control the hardware and the data streams; they are likely to release their own internal or public benchmarks that leverage proprietary, high-fidelity user data. Its novelty lies in the 'agentic' framing and the inclusion of external knowledge (RAG-VQA), but this is an incremental step rather than a breakthrough. Major competitors include established egocentric datasets like Meta's Ego4D, which already possesses significantly deeper moats in terms of data volume and industry backing.