Collected sources and patterns will appear here. Add from search, explore, or the patterns library.
Tensor<RoutingLogits> -> Tensor<LossScalar>
Calculate a routing loss using auxiliary load-balancing and z-loss regularization to stabilize MoE training.
Problem it solves
Dynamic routing in sparse MoE models suffers from routing collapse or numeric instability during training.
Consumes
Emits
The real projects this mechanism was found in. Attribution is the point — this is how the best teams actually do it.