Retrieval-Augmented Generation

RAG is how you get an AI system to answer from your documents instead of guessing. This is the plain-English guide to what it is, when it beats fine-tuning, and what your content has to look like first.

Answers from your own documents, not from something the model half-remembers.

What is retrieval-augmented generation?

Two CodeFish team members talking through a problem across the meeting room table

The model looks it up before it answers.

A language model knows what it was trained on, not your procedures, contracts or last month's site reports, and when asked about them it often invents something plausible. Retrieval-augmented generation (RAG) finds the relevant passages in your own documents at the moment the question is asked, and answers from them. Every answer links back to its source, so anyone can check it. Your team stops interrupting the one person who knows, and acts on answers they can verify.

RAG, fine-tuning, a bigger model or a query?

Pick by where the answer lives.

RAG

For when the answer lives in your documents and those documents change. Update the procedure and the answers update with it, with nothing to retrain.

Fine-tuning

For behaviour, not facts: a house format or a consistent tone. Using it to fix a knowledge problem is the most common expensive mistake here.

A bigger model

Helps least. If retrieval misses the right passage, a stronger model just writes a more convincing wrong answer.

A plain database query

If the answer is a definite value in one of your systems, a query gets it right every time for a fraction of the cost. We will say so when it fits.

What is a RAG system made of?

Most of it is content work. The model is the last and least interesting piece.

Content worth retrieving

Current, findable, and not contradicting itself. Retrieval will surface a superseded procedure next to the live one with equal confidence, and cleaning that up is usually the real project.

Sensible chunking

Documents split so a retrieved passage still makes sense on its own. Split mid-procedure and the system answers with half a rule, which reads as authoritative and is wrong.

Retrieval that finds the right thing

Meaning-based search, usually alongside keyword search, because the exact part number and the vaguely worded question both have to work.

Permissions that follow the user

Someone asking a question should only get answers from documents they are allowed to read. This is designed in at the start — bolting it on afterwards means rebuilding the retrieval layer.

Citations back to the source

Every answer linked to the paragraph it came from. This is what makes an answer checkable, and it is the feature that decides whether people still use the system in month three.

A way to measure answers

A set of real questions with known-good answers, run whenever anything changes. Without it you are judging the system on impressions, and impressions do not catch a quiet regression.

How long does a RAG project take to ship on your data?

Collect the real questions (1–2 weeks)

What people actually ask, from the service desk, the inbox and the person everyone interrupts. Those questions become the test set the system is judged on rather than a guess at what it should handle.

A hand-drawn wireframe sketch on paper showing sidebar navigation, locations, devices and services

Sort out the content (2–6 weeks)

Find the authoritative version of each answer, retire the duplicates, fix the contradictions. The least technical stage and almost always the one that decides whether the system is any good.

The CodeFish Studio workspace in Adelaide, developers working at desks on the studio floor

Ship it with citations on

One team, real questions, every answer showing its source. Watch what people ask that it cannot answer — that list is the roadmap for the next round.

A designer sketching interface wireframes on paper beside a laptop

What does a RAG system cost?

It starts with a Rapid Insight Workshop at a fixed $9,000 + GST. We look at the questions people actually ask and the state of the content that should answer them, because the content sets the price. Where your procedures are current and organised, a first system takes weeks; where the same answer sits in four documents with three different dates, the project is mostly content clean-up. You know which one you have, and what it will cost, before the build is quoted.

RAG questions we get asked

Findable, current, and not contradicting itself. Retrieval will faithfully surface an out-of-date procedure sitting next to the current one, and the system has no way of knowing which you meant. Cleaning that up is usually the real project.

Yes, and it should. Being able to open the source paragraph is what makes the answer checkable, and it is the feature that decides whether people trust the system enough to keep using it after the first month.

It does not have to. The architecture can keep documents and queries inside an environment you control, and never send them to a public tool or use them to train someone else's model. For businesses with privacy or compliance obligations, that is a decision made at the start, not retrofitted. Under the Privacy Act 1988, the Australian Privacy Principles still apply when an AI system reads personal information in your documents, including APP 8 if anything is sent offshore.

It should say so, and a system that never does is a system that is guessing. Refusing to answer without a source is a design decision made up front, and it is what lets people act on the answers they do get. The unanswered questions are also the most useful list you will have for what to write next.

RAG accuracy is measured with a test set of real questions from your own staff, each with a known-good answer and the source it should come from. Every change to the content, the retrieval or the model is run against that set before it ships, so you can see whether answers got better or worse instead of guessing. Each answer is checked on two things: did it find the right passage, and did it say what that passage says. Questions it should refuse belong in the set too, because a confident wrong answer is the failure that costs you. This lines up with the testing and transparency guardrails in the Australian Government's Voluntary AI Safety Standard.

A RAG system's monthly running cost is driven by question volume, the model you choose and where it is hosted, and for most internal teams the usage fees are small next to the people time. Each question carries model and search charges that grow with how much text is retrieved. Hosting in an Australian cloud region adds a steady amount for the search index and the app itself. The larger ongoing cost is keeping your data current and re-running the test set whenever documents change. A system nobody maintains gets cheaper to run and less useful at the same pace, so budget the maintenance time before you budget the fees.

Ready to build something great?

Local partners, global standards – full service design and development.