Back-end18 min read
What is RAG? Indexing, hybrid search and a real case
What is RAG (Retrieval-Augmented Generation), explained with a real project: crawler, chunking, embeddings, vector store, hybrid search, re-ranking and cited sources.

If you landed here searching for "what is RAG", you've probably seen the acronym in job posts, in AI threads and in every product that promises "chat with your documents". The idea is much simpler than the name.
I built a RAG from scratch: Nixa AI, a chat assistant that answers questions about the public documentation of NICE CXone, a contact center platform. You ask in Portuguese, it finds the right passages in the official docs and answers telling you where it got them. The code is open and on GitHub.
This post explains what RAG is, how indexing works (crawler, chunks, embeddings, vector store) and the path a question takes (hybrid search, re-ranking, prompt and sources), with the project's real numbers.
TL;DR#
- RAG means searching before answering. The app finds relevant passages in your documents and hands them to the model along with the question.
- It has two halves: indexing, which runs ahead of time (fetch, split, embed, store), and the question, which runs on every message (search, rank, build the prompt, generate).
- An embedding is a vector of numbers that represents the meaning of a text. Similar texts become nearby vectors.
- Meaning-only search misses API names. In Nixa, the score is 75% meaning and 25% words, and then a re-ranking step adjusts it by source, page type and recency.
- One index, one embedding model. Change the model, reindex everything.
- RAG teaches the model nothing, it only gives it context. If the answer isn't in the documents, the right move is to say so.
The problem: the model hasn't read your docs#
An LLM knows what was in its training data up to a cutoff date. Your product's documentation, last week's release notes, your team's wiki: it saw none of that. When you ask anyway, it does what it does best, which is write a plausible text. Sometimes right, sometimes with an endpoint that never existed.
The CXone documentation is large, spread across several sites (help center, developer portal, SDKs on GitHub) and almost entirely in English. Finding an answer usually means opening ten tabs. It was the perfect setting for RAG.
What is RAG#
RAG stands for Retrieval-Augmented Generation. The name comes from a 2020 paper by researchers at Facebook AI, and the idea fits in one sentence: before generating the answer, retrieve the relevant passages and put them in the prompt.
It's an open-book exam. The model is still the one writing, but instead of answering from memory, it answers with the page open in front of it.
There are three steps:
- Retrieval: find, in a document base, the passages related to the question.
- Augmentation: build the prompt with those passages and the question.
- Generation: the LLM writes the answer using that context.
Steps 2 and 3 are almost trivial. Nearly all the work (and nearly every bug) lives in step 1. And to retrieve well, the base has to be prepared beforehand: that's indexing.
RAG, fine-tuning or pasting everything into the prompt?#
Before building, it's worth understanding why not the other two obvious options.
| Paste everything in the prompt | Fine-tuning | RAG | |
|---|---|---|---|
| How it works | Sends the full documents with every question | Trains the model again on your data | Finds the relevant passages and sends only those |
| New document | Just paste it | Train again | Index the new page |
| Cost per question | High: every token, every time | Low | Low: just a few passages |
| Shows the source | Hard | No | Yes, every passage knows where it came from |
| Good for | A few short documents | Teaching style, format, a task | A large base that changes and needs sources |
Pasting everything works when the content fits comfortably in the context window. Not my case: a single release notes page I indexed is over 140 thousand characters. Fine-tuning teaches behavior, isn't good at holding facts that change every week and won't tell you where the answer came from. RAG solves exactly the two things I needed: content that changes and a source in every answer.
The two halves of a RAG#
Every RAG has two halves that run at different times:
- Indexing, before any question: fetch the documents, split them into pieces, turn each piece into a vector and store it.
- Question, on every message: turn the question into a vector, find the closest pieces, rank them, build the prompt and call the LLM.
In Nixa, indexing is an API route I trigger from the UI, and it streams its progress back. The question is the chat route.
Indexing, step by step#
1. The crawler: fetch and clean#
The crawler starts from a list of 154 seed URLs and only follows links inside 11 allowed domains: the help center, the developer portal and the official SDKs on GitHub and npm. Login pages, error pages, binary files and third-party GitHub repos are left out.
For each page, it downloads the HTML and uses Cheerio to throw away everything that isn't content: menu, footer, cookie banner, "was this page helpful?". That junk looks harmless, but if it gets into the index, it shows up in every search result, because it's on every page.
import * as cheerio from "cheerio";
export async function crawlPage(url: string) {
const html = await fetch(url).then((res) => res.text());
const $ = cheerio.load(html);
// The same menu on every page means noise in every search result
$("nav, footer, header, script, style, .cookie-banner").remove();
const title = $("h1").first().text().trim();
const content = $("main, article").first().text().trim();
if (content.length < 150) return null; // empty or error page
const pageType = classify(url, title); // api, reference, guide, release
return { url, title, content, breadcrumb: getBreadcrumb($), pageType };
}Besides the text, the crawler stores metadata: title, breadcrumb (a path like "Agent > Copilot > Configure"), the page type (API, reference, guide, release note) and a SHA-256 hash of the content. The hash saves money: on the next indexing run, if the page hasn't changed, it doesn't go through the embedding again. And a page indexed more than two weeks ago is fetched again.
Keep the metadata in mind, it comes back in the re-ranking.
2. Chunking: split into pieces#
You don't index the whole page. An API page can talk about authentication, pagination and rate limits at once, and the vector of the whole page becomes an average of all that: it isn't close to any specific question. So the page is split into chunks.
In Nixa, the target is 900 characters per chunk, and each chunk starts by repeating the last sentences of the previous one, up to about 200 characters. That's the overlap.
The overlap exists because the answer doesn't respect your split boundary. If the sentence explaining the parameter ended up at the end of one chunk and the example at the start of the next, the repetition makes sure at least one of them has both.
Another detail: a short paragraph that looks like a title ("Configure the profile") doesn't become a chunk on its own. It's glued to the next paragraph, so the passage carries its subject with it.
export function chunkText(text: string, maxSize = 900, overlap = 200): string[] {
const chunks: string[] = [];
let current = "";
let pendingHeading = "";
for (const paragraph of text.split(/\n\n+/).map((p) => p.trim()).filter(Boolean)) {
if (looksLikeHeading(paragraph)) {
pendingHeading = paragraph; // glue it to the next paragraph
continue;
}
const section = pendingHeading ? `${pendingHeading}\n${paragraph}` : paragraph;
pendingHeading = "";
if (current && current.length + section.length > maxSize) {
chunks.push(current);
// the next chunk starts with the last sentences of this one
current = `${lastSentences(current, overlap)}\n\n${section}`;
} else {
current = current ? `${current}\n\n${section}` : section;
}
}
if (current.length > 80) chunks.push(current);
return chunks;
}Notice the split happens on paragraphs (\n\n). That has a consequence I'll get to at the end of the post.
3. Embeddings: turn text into a vector#
An embedding is a list of numbers that represents the meaning of a text. You send a text to an embedding model and get back a vector, like [0.012, -0.341, 0.088, ...]. The trick is that texts with similar meaning become vectors pointing in similar directions, even with no words in common: "how do I log in to the API" lands close to "authentication with OAuth".
The vector size depends on the model. Nixa's index has vectors of 384 numbers (all-minilm, which runs locally on Ollama) and of 3,072 numbers (the size gemini-embedding-001 returns). The app supports three embedding providers: Gemini (the default), OpenAI (text-embedding-3-small) and Ollama.
A cheap trick that helped: before generating the vector, the page title goes in front of the chunk. A passage that only says "set the value to true" means nothing on its own. With [Configure Copilot] in front, it does. Anthropic describes a more elaborate version of this idea, contextual retrieval.
const input = `[${chunk.title}] ${chunk.content}`.slice(0, 2000);
const { embedding } = await embed(input); // number[]4. The vector store: keep the vectors#
The vector store is where the chunks, vectors and metadata live. And here's the part that surprises anyone expecting a fancy vector database: in Nixa, it's a JSON file. The repository's data/vectorstore.json holds 425 chunks from 62 pages and weighs 11 MB.
To search, the app reads the file, computes the similarity between the question and every chunk, and sorts. Brute force. With 425 chunks, it's instant. With hundreds of thousands, that's when you want a real index, like pgvector on Postgres or a dedicated vector database. Start simple and switch when you measure that you need to.
The most common closeness measure is cosine similarity: how much two vectors point in the same direction, on a scale from -1 to 1.
function cosineSimilarity(a: number[], b: number[]): number {
if (a.length !== b.length) return 0; // vectors from different models
let dot = 0, normA = 0, normB = 0;
for (let i = 0; i < a.length; i++) {
dot += a[i] * b[i];
normA += a[i] * a[i];
normB += b[i] * b[i];
}
return dot / (Math.sqrt(normA) * Math.sqrt(normB));
}Remember that first line. It's the reason for one of the project's rules.
The question, step by step#
1. Query expansion: Portuguese to English#
The documentation is almost all in English and people ask in Portuguese. The embedding handles that reasonably well, but keyword search doesn't: "relatório" will never match "reporting".
So the question becomes two: the semantic one is the original question, and the lexical one gets English terms appended. There are 19 rules, one per subject:
const PT_EN_TERMS = [
{ pattern: /(relat[óo]rio|dashboard|m[ée]tricas)/i, en: "reporting analytics reports dashboard metrics KPI" },
{ pattern: /(fila|filas|roteamento|skill)/i, en: "queue queues skill skillset ACD routing" },
{ pattern: /(autentica[çc][ãa]o|login|oauth)/i, en: "authentication oauth bearer access login" },
// ...
];
function buildQuery(question: string) {
const extra = PT_EN_TERMS.filter((term) => term.pattern.test(question)).map((term) => term.en);
return { semantic: question, lexical: [question, ...extra].join(" ") };
}"Como gerar um relatório de filas?" ("how do I create a queue report?") goes to the embedding exactly as typed, and to the keyword search with "reporting", "queue", "ACD" and friends.
2. Hybrid search: meaning and words#
Semantic-only search is great for vague questions and bad at exact names. If someone asks about the agent-sdk, you want the passage that contains that word, not one about something similar. Keyword-only search has the opposite problem: it doesn't know "sign in" and "login" are the same thing.
Hybrid search adds both. In Nixa, each chunk's score looks like this:
const isShort = question.split(/\s+/).length <= 2;
const semanticWeight = isShort ? 0.5 : 0.75;
const lexicalWeight = isShort ? 0.5 : 0.25;
// lexicalOverlap: share of the query words found in the chunk (0 to 1)
const score =
cosineSimilarity(queryEmbedding, chunk.embedding) * semanticWeight +
lexicalOverlap(lexicalQuery, chunk.content) * lexicalWeight;Regular question: 75% meaning, 25% words. A question of up to two words goes half and half, because with so little text the embedding has little meaning to capture, and the exact word matters more.
The lexical part is as simple as it gets: the fraction of the question's words that appear in the chunk. You could swap it for BM25 and do better, but it was enough to get API and SDK names right.
This math yields the 20 best candidates (four times the number of passages that go into the prompt, with a minimum of 16), which move on to re-ranking.
3. Re-ranking: what similarity doesn't know#
Similarity only looks at the text. But I know things the text doesn't say: that the official API docs are worth more than some random README, that a technical question wants a reference page, that yesterday's release note is worth more than last year's. Re-ranking is where that comes in, adding bonuses and penalties to the score:
| Signal | Adjustment |
|---|---|
| Official developer portal | +0.15 |
| Official help center | +0.12 |
| GitHub that isn't one of the official SDKs | -0.40 |
| Technical question and API page | +0.12 |
| Technical question and reference page | +0.10 |
| Indexed up to 7, 30 or 90 days ago | +0.06, +0.04 or +0.02 |
| Each question word in the title | +0.025 |
| A question word in the breadcrumb | +0.08 |
| The whole question inside the title | +0.15 |
A "technical" question here is one with words like api, sdk, webhook or endpoint. After the adjustment, duplicate chunks are dropped, anything below 0.22 is discarded and the top 5 go into the prompt.
Rule-based re-ranking is the simplest kind there is. The next step would be a re-ranking model (a cross-encoder) that reads the question and the passage together and scores them. For a base this size, the rules did the job, and each one is easy to explain when a result looks odd.
4. Knowing when not to answer#
Before calling the LLM, Nixa looks at the best passage's score and rates the confidence: high if it's above 0.6 with at least two passages, medium above 0.4, low otherwise. With low confidence or no passages, it doesn't even call the model: it answers that it couldn't find that in the official documentation.
It sounds small, but it's the difference between a useful assistant and a guessing machine with nice-looking sources. If the context is weak, the model fills the gap with whatever sounds plausible.
5. The prompt#
The 5 passages go into the system prompt as XML, grouped by page type, each with title, URL and breadcrumb, and trimmed to 700 characters:
<context>
<section type="API">
<document index="1" type="api" url="https://docs.example.com/api/auth">
<title>Authentication</title>
<breadcrumb>Documentation > Getting Started</breadcrumb>
<content>...</content>
</document>
</section>
</context>And the rules say what to do with them: answer in the question's language, cite the source inline with [Title](URL) and, if the information isn't in the context, say it couldn't find it in the official documentation instead of making up an endpoint, field or version. When two sources disagree, the order is: API, then reference, then guide, then release note.
The LLM can be Gemini, OpenAI or Ollama (100% local). Switching changes nothing in the RAG: the prompt is the same.
6. Only the sources that were used#
A lot of RAG apps show below the answer everything the search returned. But the model may have used two of the five passages, and then three "sources" are decoration.
In Nixa, the answer is streamed and, when it ends, the server checks which URLs actually appear in the text and sends only those:
export function selectCitedSources(answer: string, candidates: Source[]): Source[] {
return candidates.filter((source) => answer.includes(source.url));
}One line. Since the prompt asks for citations with the link, the source shown below is the one the model really used.
One index, one embedding model#
Now, that first line of the cosine function. Vectors from different models can't be compared: they live in different spaces, sometimes not even of the same size. Between a 384-number vector from all-minilm and a 3,072-number one from Gemini there's no math to do, which is why the function returns 0. Even when the size matches, two models organize meaning differently: you'd get a score, but it would be garbage.
That's why Nixa's rule is a fixed embedding per index. The file records which provider, which model and which dimension produced the vectors, and the search always uses that same model to embed the question, no matter which LLM you picked for the chat. You can chat with Ollama and search with Gemini. If the question's dimension doesn't match the index, the search doesn't pretend: it falls back to keyword-only search and logs a warning to reindex.
const { embedding: queryEmbedding } = await getEmbedding(question, store.embeddingProvider);
if (store.dims && queryEmbedding.length !== store.dims) {
console.warn("Embedding dimension mismatch, reindex. Falling back to lexical search.");
return lexicalSearch(lexicalQuery);
}I learned this the annoying way. The index in the repository today mixes both: 330 chunks with 384-number vectors, from an indexing run with local Ollama, and 95 with 3,072, from Gemini. For a question embedded by Gemini, those 330 have zero similarity and only show up through the lexical part, which is worth 25% of the score. More than three quarters of the base almost invisible to semantic search, and not a single error on screen.
Changing the embedding model means changing the whole index. There's no partial migration.
The same search, outside the chat#
Since the search doesn't depend on any LLM, it became its own function, searchNiceDocs(), which only returns passages. The chat calls it and then generates the answer. But other clients can call it too: it's exposed as an MCP tool, search_nice_docs, and as a REST route.
That way ChatGPT, Claude or Cursor query the same base, and the client's own model writes the answer. If MCP is new to you, I wrote about it in What is MCP.
When RAG isn't the answer#
RAG is good at one thing: finding the right passage and answering from it. Beyond that:
- Questions that need the whole base. "How many APIs are there?" or "summarize every release this year" don't fit in 5 passages. That calls for another approach, like a structured query or a summary built ahead of time.
- Data that changes every minute. The index is a snapshot. In Nixa, the question "what's the latest version of the agent SDK?" doesn't even go through RAG: the app queries the npm registry on the spot and answers with the version and its publish date.
- Bad documents. If the answer isn't written anywhere, or is wrong, RAG will faithfully retrieve the wrong thing.
- Changing how the model behaves. Tone, format, a kind of task: that's prompting or fine-tuning, not retrieval.
- Poor splitting. If chunking separates the question from the answer, no re-ranking will save you. Which brings me to the next topic.
What I learned#
Splitting by structure has a price. The chunker splits on paragraphs (\n\n). When a page arrives with no blank lines at all, it becomes one paragraph, and one chunk. In the current index, 62 chunks are over 2 thousand characters, and the biggest, a release notes page, has 142,273 characters. The embedding only reads the first 2 thousand. The rest exists in the file and doesn't exist for semantic search. The fix is a safety cut: if a paragraph goes over the limit, split it by sentence.
Metadata is half the ranking. The title in front of the chunk, the page type, the breadcrumb and the domain made more difference in the answers than I expected. And they cost almost nothing to keep in the crawler.
Saying "I don't know" is a feature, not a failure. The minimum score and the confidence check before calling the LLM cut the made-up answer that fools people the most: the one that comes with sources and all.
Make the search observable. With RAG_DEBUG=true, each question logs the expanded query, each passage's score and the confidence. When an answer comes out wrong, the first place to look is the search, not the LLM: if the right passage never reached the prompt, no model gets it right.
Start with a JSON file. A file-based vector store, hand-written cosine, ten lines of keyword search. You can understand every number that comes out. A vector database is an optimization for when the base grows.
References#
Read next

Back-end12 min read
What is MCP (Model Context Protocol)? A practical guide

Front-end16 min read
Multilingual SEO in React: hreflang, prerendering, sitemap

Front-end15 min read
How I make AI videos with Remotion: a real case study