When a model 'doesn't know X' or 'won't behave how I want,' there are three completely different fixes — and picking the wrong one wastes months. RAG changes what the model KNOWS, fine-tuning changes how it BEHAVES, prompting steers within what it already can do. Here's the decision framework, why they compose instead of competing, and where LoRA fits.
Semantic search and RAG rest on one deceptively simple operation: find the nearest vectors to a query. At a few thousand rows that's a boring loop; at a hundred million it's the whole ballgame. Here's what a vector index actually does inside — why brute force falls over, why HNSW's layered graph wins, and which three knobs decide your recall, latency, and RAM bill.
Autoregressive decoding is slow because it's sequential and memory-bandwidth bound — the GPU spends its time hauling multi-gigabyte weight matrices out of HBM, not doing math. Speculative decoding exploits that: a small draft model guesses the next few tokens, the big model verifies them all in a single pass, and a clever accept/reject rule makes the output byte-for-byte identical to the big model alone. Here's the whole trick.
LLM outputs are non-deterministic and open-ended, so 'does it pass the test' has no exact-match answer — which means every prompt tweak you ship on vibes is silently regressing cases you never look at. Evals are the fix: a repeatable, quantified measure of output quality, the way unit tests are for code. Here's the taxonomy, how LLM-as-judge works and where it lies to you, and how to wire it all into a regression gate.
Asking an LLM to 'reply in JSON' and then JSON.parse-ing the result is the retry-and-pray pattern, and at scale a 1% parse-failure rate is a paging incident. There's a better move: constrained decoding masks the token distribution at every step so the model literally cannot emit a syntactically invalid character. Here's how it works, what it does and doesn't guarantee, and how to wire it in as a backend engineer.
Prompt engineering was about wording one message. Context engineering is about managing the entire context window as a scarce budget — what goes in, in what order, and what gets evicted. For a backend engineer, it's working-set management applied to an LLM.