/tag
Autoregressive decoding is slow because it's sequential and memory-bandwidth bound — the GPU spends its time hauling multi-gigabyte weight matrices out of HBM, not doing math. Speculative decoding exploits that: a small draft model guesses the next few tokens, the big model verifies them all in a single pass, and a clever accept/reject rule makes the output byte-for-byte identical to the big model alone. Here's the whole trick.