Collected sources and patterns will appear here. Add from search or the patterns library.
LogicalTokenSequence -> PhysicalMemoryPointers
Map logical sequence tokens to non-contiguous physical GPU memory blocks using a virtual page table to eliminate KV cache fragmentation.
Problem it solves
Static pre-allocation of maximum-sequence KV caches causes high memory fragmentation and limits serving concurrency.
Consumes
Emits
The real projects this mechanism was found in. Attribution is the point — this is how the best teams actually do it.