MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion

MobiFlow addresses a critical bottleneck in mobile agent research: the 'verification gap.' While tools like AndroidWorld provide robust testing environments, they rely on system-level state checks (e.g., checking if a file exists or a specific database entry changed), which isn't possible for the vast majority of closed-source third-party apps. MobiFlow's use of trajectory fusion—comparing the agent's visual/action path against reference human or expert trajectories—is a theoretically sound way to bridge this. However, the project currently lacks any public traction (0 stars), despite 8 forks which likely indicate peer researchers or internal team members. Its defensibility is low because the value of a benchmark is entirely derived from its adoption as a standard; without a leaderboard or community buy-in, it remains a purely academic exercise. Furthermore, Google (Android) and Apple are incentivized to build their own 'Agentic Testing' suites that might expose the very APIs MobiFlow circumvents, creating a high platform domination risk. If Google releases an 'Agent-Ready' developer mode for Android that provides success signals, the need for trajectory-based verification significantly diminishes.