RAG from scratch, then Azure AI Search

Already ranking chunks by cosine similarity and using hybrid search? Skip to the next lesson.

Lesson 0.3 said the fix for made-up answers is putting true information in front of the model. This is how that is actually done, and it has a name.

What RAG means

RAG stands for Retrieval-Augmented Generation. Three words for one idea:

Look it up first, then answer.

That is genuinely all it is. Before the model writes anything, your code finds the relevant documents and puts them into the prompt.

The four steps

1. Chunk. Split your documents into small pieces, roughly one topic each.
2. Embed. Turn each chunk into coordinates, exactly as in lesson 0.7.
3. Rank. Turn the question into coordinates too, and find the closest chunks. Measuring that closeness is called cosine similarity.
4. Stuff. Put the winning chunks into the prompt and ask the question.

Chunking decides everything

This is the part people get wrong, so it is worth saying plainly.

If a chunk is one whole document, its coordinates are an average of everything in it. A page covering VPNs, printers and passwords points nowhere in particular, so it matches nothing well. Ask about VPNs and you either miss it, or you retrieve the whole page and waste most of the prompt on printers.

No amount of clever ranking later rescues badly cut chunks. Get this right and mediocre search still works. Get it wrong and nothing saves you.

Then the Azure version

Azure AI Search is Microsoft's managed version of the same pipeline.

Its main addition is hybrid search: running two searches at once. One is the coordinate search you just built. The other is keyword matching, which is still better when someone searches an exact error code or product name. RRF, reciprocal rank fusion, is how the two result lists get merged.

You build the simple version by hand first, because then the managed one is a service doing something you understand rather than a black box.

What to write

CORPUS is already split sensibly for you. embed_query is a small stand-in embedder.

1. cosine(a, b) and naive_search(query, k=2), returning the winning chunk ids
2. azure_hybrid_search(query), the Azure stand-in, mentioning RRF
3. TOOLS and execute_tool for both searches
4. The Module 3 loop, with tools=TOOLS and deployment=AZURE_DEPLOYMENT

The point

Chunk quality decides answer quality. Clever retrieval does not rescue bad chunks.

Break it on purpose

Replace CORPUS with one giant chunk containing the whole policy. Watch the VPN question stop finding the VPN answer.