Collected sources and patterns will appear here. Add from search, explore, or the patterns library.
QuantizedWeights -> ActivationTensor
Execute matrix multiplication on quantized weights (INT4/INT8) using Intel AMX, AVX512, or AVX2 hardware instructions.
Problem it solves
Standard CPU execution of quantized LLM weights is too slow to sustain real-time offloaded inference.
Consumes
Emits
The real projects this mechanism was found in. Attribution is the point — this is how the best teams actually do it.