Compiled once, not retrieved every time

What RAG actually does

Retrieval-augmented generation keeps your raw documents - transcripts, PDFs, whatever - in an index. Ask a question, it retrieves the chunks that look relevant, stuffs them into a prompt, and the model answers from that context. Ask a related question five minutes later and it retrieves again, from scratch.

Nothing about the first answer sticks. The index does not get smarter, more organized, or more connected because you asked it something. It is a very good re-reading machine, and re-reading is the entire operation.

What a compiled network does instead

The method reads a source once - in the compile step - and writes what it learned onto pages organized by concept, linked to related concepts, headed by where each claim came from. The next question reads those pages, not the raw transcript.

The difference shows up on the second question. RAG re-derives the answer from raw text again. A compiled network already has the page, already has the links to adjacent concepts a raw-text search would never have surfaced, and answers faster because the reading is done.

The honest tradeoff

Compiling costs more upfront than indexing. You pay a model to read and write on the way in, before anyone has asked a single question - RAG defers that cost until query time and pays it, in a smaller way, every single time. If you only ever ask one question of a corpus, RAG is cheaper.

But a corpus you keep coming back to inverts the math. Every question against a compiled network reuses work already paid for. Every question against RAG pays the retrieval-and-reasoning cost again, and the answer still evaporates when it's done.

Where RAG still wins

Be honest about the cases: a corpus too large to compile - millions of documents where writing a page per concept is not a weekend job. A corpus that changes faster than you can recompile it - live logs, a feed that updates by the minute. A question that wants the exact raw text, not a synthesis of it - a legal quote, a precise transcript line, where compiling would paraphrase away the thing you actually needed.

And plainly: if there is no budget to run the compile pass at all, RAG with no upfront cost beats a compiled network that never gets built.

Pick by how often you'll ask

One-off question over a huge or fast-moving corpus: RAG. A body of sources you'll query, link and build on for months - a podcast archive, a research beat, a personal knowledge base: compile it. The method exists for the second case; it is not a universal replacement for the first.