Collected sources and patterns will appear here. Add from search, explore, or the patterns library.
Tensor<HiddenStates> -> List<Tensor<HiddenStates>>
Segment large expert networks into multiple smaller sub-experts and route tokens across the expanded pool.
Problem it solves
Coarse-grained routing restricts the flexibility of expert combinations, limiting parameter specialization.
Consumes
Emits
The real projects this mechanism was found in. Attribution is the point — this is how the best teams actually do it.