Collected sources and patterns will appear here. Add from search or the patterns library.
Tensor<Tokens> -> List<Tensor<ExpertTokens>>
Route mixed-modality tokens to the top-k most relevant neural network experts based on router probability scores.
Problem it solves
Processing all tokens through all experts is computationally expensive; sparse routing reduces active parameters per forward pass.
Consumes
Emits
The real projects this mechanism was found in. Attribution is the point — this is how the best teams actually do it.