fkodom/soft-mixture-of-experts

This project is a clean, third-party implementation of a specific research paper from Google Brain. While technically sound, it lacks a defensive moat. With only 83 stars and zero recent velocity, it serves primarily as a reference for researchers rather than a production-grade library. The 'Soft MoE' technique itself is a significant architectural shift (moving from discrete routing to soft weighting), but this repository is easily replaceable. Frontier labs like OpenAI or Google develop these architectures internally; if Soft MoE gains wider traction, it will be natively integrated into major frameworks like Hugging Face Transformers, DeepSpeed, or Megatron-LM, rendering a standalone implementation obsolete. The displacement horizon is very short because any team serious about training MoE models would likely implement this logic directly into their specialized training harness to optimize for distributed throughput, rather than relying on a small, unmaintained repository.