jeffasante/cellm

cellm is an extremely early-stage research project (8 days old, 3 stars, 0 velocity) addressing a real problem—efficient LLM inference on mobile devices with severe memory constraints. The paged KV cache and multi-session scheduling are known techniques, but their combination for sub-512MB mobile inference is a reasonable research contribution. However, defensibility is critically weak: (1) No production users or evidence of adoption; (2) No clear differentiation from existing mobile LLM frameworks (TensorFlow Lite, MLX, CoreML optimizations, or proprietary solutions from Apple/Google); (3) Rust + Metal/Vulkan is a defensible implementation choice but not a moat—these are commodity tools; (4) The specific RAM constraint (512MB) is narrow and context-dependent. Platform domination risk is HIGH: Apple and Google are aggressively investing in on-device LLM inference (Apple Intelligence, Pixel Neural Core). Either platform could trivially integrate equivalent capabilities into their OS or development frameworks within 6-12 months, especially given their ability to co-design hardware and software. Market consolidation risk is MEDIUM: Incumbents like Qualcomm (for Android), Apple (for iOS), and specialized mobile ML startups (e.g., Hugging Face's on-device optimizers, Anthropic's potential mobile play) could quickly outpace a solo research project. Displacement horizon is 6 MONTHS because platform vendors are actively shipping mobile LLM inference products now, and this project has zero traction, no community, and no defensible moat beyond the novelty of the technical approach. If the author can rapidly build adoption (proving real mobile use cases and users), gain academic visibility, or open-source a uniquely better implementation, the horizon could extend. As-is, it's a research sketch with high risk of obsolescence by major platform players.