Collected sources and patterns will appear here. Add from search or the patterns library.
Tensor<KV_States> -> Tuple<Tensor<Latent_KV>, Tensor<Decoupled_Query_Key>>
Compress Key-Value matrices into a low-rank latent joint vector during attention calculations to minimize inference memory usage.
Problem it solves
Standard Multi-head Attention (MHA) creates huge Key-Value caches that bottle-neck generation throughput and maximum batch sizes.
Consumes
Emits
The real projects this mechanism was found in. Attribution is the point — this is how the best teams actually do it.