SemanticScanpath: Combining Gaze and Speech for Situated Human-Robot Interaction Using LLMs

SemanticScanpath addresses a critical bottleneck in Human-Robot Interaction (HRI): the grounding of underspecified language (e.g., 'give me that') in physical reality using gaze cues. While the 'Semantic Scanpath' representation is a clever way to bridge the gap between low-level gaze data and high-level LLM reasoning, the project's defensibility is low (score 3) due to its status as a fresh academic release (9 days old, 0 stars) with no established ecosystem. The primary threat comes from frontier labs like OpenAI and Google, who are moving toward native multimodal processing (GPT-4o, Gemini Multimodal Live) where gaze and spatial video data could be ingested directly into the model's latent space, potentially rendering intermediate 'representations' like scanpaths obsolete. The 5 forks indicate early academic replication or internal team use, but without a robust software framework or proprietary dataset, the 'moat' is purely the novelty of the algorithm which is easily replicated by any robotics lab with gaze-tracking hardware.