Three ways to give a model knowledge

RAG isn't the only way to make a model "know" something it didn't learn in training. There are three main options, and picking the right one saves enormous effort:

• RAG — retrieve relevant text at question time and put it in the prompt.
• Fine-tuning — further-train the model on your data so the knowledge or style is baked in.
• Long context — simply paste everything into a large context window each time.

When RAG is the right call

Reach for RAG when your knowledge is large, changing, or needs citations:

• The data is too big to fit in a single prompt (thousands of documents).
• It updates often — prices, policies, docs, news. Re-indexing is instant; re-training isn't.
• You need traceability — showing which source an answer came from.
• You want to add or remove documents without touching the model.

This covers the majority of real products, which is why RAG is the default starting point.

When the others win

• Fine-tuning shines for teaching behavior and format — a consistent tone, a niche output structure, a specialized skill — not for memorizing facts, which it does unreliably and can't update cheaply.
• Long context is simplest when the relevant material is small and self-contained — a single contract, one report. No infrastructure, no retrieval step to get wrong.

They also combine well: fine-tune for style, use RAG for facts.

If the question is "does the model sound right?" consider fine-tuning. If it's "does the model know the right facts, right now, and can it prove it?" you want RAG.

Try this: List three things you'd want an AI assistant to know about your work or life. For each, ask: is this big and changing (RAG), a style or behavior (fine-tune), or small and fixed (just paste it)? You'll likely find most land squarely in RAG territory.