Doubly Sparse: Sparse Mixture of Sparse Experts for Efficient Softmax Inference

While the paper presents a sophisticated mathematical approach to softmax efficiency using MoE principles, the project has zero community traction (0 stars) and is over 7 years old. The techniques described (sparsity in MoE) have since been subsumed or superseded by modern LLM architecture optimizations (like FlashAttention and PagedAttention) and standard MoE implementations used by frontier labs.