Making LLMs Fast and Small: A Guide to Inference Optimization Research in 2026
Five approaches to making LLMs faster and cheaper — compression, diffusion decoding, architecture, KV cache, and sparse attention — explained with real numbers.
Kv-Cache
4.7x Faster Long-Context LLM Inference: AsyncTLS Explained
Long-context LLM inference collapses past 96k tokens. AsyncTLS pairs two-level sparse attention with async KV offloading for 1.3-4.7x more throughput.
TriAttention Compresses KV Cache 10.7x — How Trigonometry Fixed Long-Context Reasoning
TriAttention uses pre-RoPE vector concentration and trigonometric scoring to compress KV cache 10.7x while matching full attention accuracy on reasoning tasks.