RAG and grounding an answer in your own data
Lesson 4 of 42 min read
Retrieval, chunking, citations — and the UI that makes a generated answer trustworthy.
A model does not know your catalogue, your policy document or yesterday's order. RAG — Retrieval Augmented Generation — is the practical fix: look up the relevant text and put it in the prompt.
The pipeline
- Chunk your documents into pieces of a few hundred words, keeping headings with their text.
- Embed each chunk into a vector and store it.
- At question time, embed the question and retrieve the nearest chunks.
- Prompt: "Answer using only the context below. If the answer is not there, say you do not know."
- Return the sources alongside the answer.
For a mobile app, steps 1-4 live on your backend. The app sends a question and renders an answer plus sources.
Why this beats fine-tuning for most app features
Your content changes weekly. With RAG you update an index; with fine-tuning you retrain. RAG also lets you show where an answer came from, which is the only reliable way to make users trust it.
The mobile developer's half of the work
- Stream the answer, then render the source chips when it completes.
- Make sources tappable — open the document at that section. This single affordance does more for trust than any wording in the answer.
- Render "I don't know" as a normal outcome, with a path to search or contact support. A model admitting ignorance is the system working.
- Cache recent answers keyed by the question, so repeating a query is instant and free.
- Never silently replace an authoritative value with a generated one. Prices, balances and dates come from your API; the model may describe them, not produce them.
Evaluating it before you ship
Write down twenty real questions with their correct answers, run them on every prompt change, and count how many are right, wrong, or correctly refused. It is not glamorous, and it is the only thing that stops a prompt tweak from quietly breaking the feature.
Treat retrieved text as data, never as instructions. If a document contains "ignore previous instructions", a careless prompt will obey it — keep retrieved content clearly delimited and your rules above it.