For thirty years the web has had exactly one safe way to ask a server a question — GET — and it can't carry a body. So every real search endpoint quietly cheats with POST and loses caching, idempotency, and honesty about what it's doing. QUERY is the proposed method that fixes the thirty-year-old workaround: safe and idempotent like GET, but with a request body like POST.
Prompt engineering was about wording one message. Context engineering is about managing the entire context window as a scarce budget — what goes in, in what order, and what gets evicted. For a backend engineer, it's working-set management applied to an LLM.
A normal cache keyed on the exact request string is almost useless for LLM calls, because every paraphrase is a miss. Semantic caching keys on meaning instead — embed the query, search for a near-identical past question, and return its answer with no model call. Here's the architecture, the threshold problem that makes or breaks it, and real pgvector code.
How resolving redundant API calls and leveraging caching transformed a sluggish billing generation process into a performant operation.