Collected sources and patterns will appear here. Add from search, explore, or the patterns library.
List<Tensor<FeatureMap>> -> List<Tensor<FeatureMap>>
Apply self-attention exclusively to high-level multi-scale features to capture semantic context, then use cheaper convolution-based fusion steps to propagate these context representations to lower-level feature maps.
Problem it solves
Applying multi-scale self-attention across all feature scales in encoder-decoder architectures is computationally prohibitive for real-time inference.
Consumes
Emits
The real projects this mechanism was found in. Attribution is the point — this is how the best teams actually do it.