RAG: an AI that knows your company's documents

What you'll see in this lesson

1. RAG: have the AI search before answering

   RAG (retrieval-augmented generation) finds the useful pieces in your documents and gives them to the AI along with the question.

2. First: does it all fit in the context?

   If your documents add up to less than about 200,000 tokens, roughly 500 pages, it's often better to include them whole with caching.

3. The fast track: the API's file search

   OpenAI and Google offer ready-made RAG: you upload the files and the model searches them on its own and cites the source.

4. Step 1: chunk the documents

   If you build your own RAG, first split the documents into chunks of a few hundred tokens: search and delivery happen chunk by chunk.

5. Step 2: embeddings and a vector database

   An embedding turns each chunk into a list of numbers that sums up its meaning: similar texts end up close together.

6. Hybrid search: words plus meaning

   Combine exact-word search (BM25, the one classic search engines use) with meaning search and merge the two lists.

7. Reranking: a second, smarter filter

   A reranker rereads the question with each candidate and sorts them by real relevance before passing them to the AI.

8. Contextual retrieval: each chunk with its context

   Before indexing, the AI adds one or two sentences to each chunk explaining which document and section it comes from.

9. Summary

   Start with the simplest thing that works and add techniques only when your tests show they're needed.