Speculative Decoding and KV-Cache Compression: Accelerating LLM Inference Throughput by 4x
Overcoming the memory bandwidth bottleneck in LLM serving: draft model speculative verification, dynamic KV-cache eviction (StreamingLLM, SnapKV), and FlashAttention-3 kernels.

