Making LLMs Fast and Small: A Guide to Inference Optimization Research in 2026
Five approaches to making LLMs faster and cheaper — compression, diffusion decoding, architecture, KV cache, and sparse attention — explained with real numbers.
Llm-Inference
AsyncTLS: 4.7x Faster Long-Context LLM Inference With Two-Level Sparse Attention
AsyncTLS sparse attention fuses block filtering, token selection, and async KV cache offloading for 1.3-4.7x throughput gains at 48k-96k token contexts.