When a model 'doesn't know X' or 'won't behave how I want,' there are three completely different fixes — and picking the wrong one wastes months. RAG changes what the model KNOWS, fine-tuning changes how it BEHAVES, prompting steers within what it already can do. Here's the decision framework, why they compose instead of competing, and where LoRA fits.
Autoregressive decoding is slow because it's sequential and memory-bandwidth bound — the GPU spends its time hauling multi-gigabyte weight matrices out of HBM, not doing math. Speculative decoding exploits that: a small draft model guesses the next few tokens, the big model verifies them all in a single pass, and a clever accept/reject rule makes the output byte-for-byte identical to the big model alone. Here's the whole trick.
LLM outputs are non-deterministic and open-ended, so 'does it pass the test' has no exact-match answer — which means every prompt tweak you ship on vibes is silently regressing cases you never look at. Evals are the fix: a repeatable, quantified measure of output quality, the way unit tests are for code. Here's the taxonomy, how LLM-as-judge works and where it lies to you, and how to wire it all into a regression gate.
Asking an LLM to 'reply in JSON' and then JSON.parse-ing the result is the retry-and-pray pattern, and at scale a 1% parse-failure rate is a paging incident. There's a better move: constrained decoding masks the token distribution at every step so the model literally cannot emit a syntactically invalid character. Here's how it works, what it does and doesn't guarantee, and how to wire it in as a backend engineer.
An agent with a 1M-token context still meets you as a total stranger tomorrow — the context window is RAM, wiped the moment the session ends. Real memory is an engineering system you build around the model: what to save, when to update it, and what to forget. This is the taxonomy, the storage trade-offs, and the write policy that separates a gimmick from a colleague.
An AI SDK doesn't make the model smarter — it deletes the streaming parser, the tool-calling loop, the JSON-validation prayer, and the per-provider glue you'd otherwise hand-roll against a raw HTTP endpoint. Here's exactly what you get over `fetch`, using the Vercel AI SDK, and where the abstraction leaks.