RAG pipeline design: from raw PDFs to grounded answers

A retrieval-augmented system can only be as good as what it retrieves. If a garbled sentence gets into a chunk, it gets into the retrieval and then into the answer, and no model downstream can repair it. I spent two full sessions cleaning PDFs before I ingested a single book. It felt like a delay. It was the best time I spent on the project.

This is a deep-dive from my travel RAG case study: how 38 guidebooks, and later 275 Wikivoyage articles, became a store of searchable chunks, and how a question becomes five good chunks in front of Claude.

1.Planning

Why RAG, not fine-tuning

The goal was answers grounded in specific guidebook text, with citations a reader can check. Fine-tuning teaches a model style and patterns; it doesn't give you a source to point at, and it would need retraining every time the content changes. Retrieval keeps the knowledge in a database I control and the model's job to reading and writing.

One chunk size does not fit every question

The classic RAG decision is chunk size. Small chunks are precise but lose context; large chunks keep context but dilute the match. Travel questions span that whole range, so I indexed every book three times, at three sizes, and let a router pick the right one per question.

Each chunk keeps a link to its parent, so retrieval can step up a level when a small chunk needs more context.

Decisions made before writing code

2.Implementation

Extraction and cleaning

I extracted the 38 PDFs with pdfplumber, with PyMuPDF as a fallback for the 0.4% of pages it handled badly. Then a custom cleaner did the real work:

The cleaned text is turned into a DocumentStructure: a tree of headings and paragraphs that knows which chapter, region and place each paragraph belongs to. Chunking works on that structure, never on raw text.

Chunking, embedding and storing

Each book is chunked at the three sizes from section 1 and embedded with Voyage voyage-3-lite into a vector(512) column. Every chunk carries metadata: its source book, its place in the heading tree and its parent chunk. An HNSW index makes approximate search over the whole corpus fast.

From question to five chunks

1. Route. A heuristic router reads the question, picks a chunk size and matches destination names against a gazetteer, a lookup table of place names. "Granada" plus Spanish context becomes a filter on Spain, not Nicaragua. A season or month in the question ("winter in Vienna") adds that season's months, weather and events to the search, so climate and event passages surface.
2. Search broadly. The question is embedded with the same Voyage model and pgvector returns the 50 nearest chunks within the filter.
3. Rerank precisely. A cross-encoder reranker reads the question and each candidate together and keeps the best 5. It is too slow to run on the whole corpus, but cheap on 50.
4. Generate. Claude receives the 5 chunks with their metadata, answers, and cites the guide and chapter each fact came from.

Swapping the corpus for launch

The guidebooks could not be served publicly, so I rebuilt the public corpus from Wikivoyage. The chunker needed no changes. A new adapter fetches articles through the MediaWiki API, strips the markup with mwparserfromhell and builds the same DocumentStructure. Wikivoyage listings for sights and restaurants are flattened into prose but keep their address, hours and price, because those are exactly what travellers ask for.

The new source brought a new structure problem. Wikivoyage puts detail on district subpages: the Sagrada Família's hours live on "Barcelona/Eixample", not on "Barcelona".

So every chunk gets a parent_destination field, and both the parent article and all its subpages share it. A question about Barcelona now reaches the district pages too. Verified end to end: Sagrada Família questions pull their top-ranked chunks from the Eixample subpage.

3.Challenges

Every book was broken differently

I tested the cleaner on the Spain guide and it looked perfect. Japan, Mexico and Iceland each broke in a new way. The worst was the reversed sidebar text: vertical labels like "PLAN YOUR TRIP" come out of the PDF backwards. The cleaner removed them from lines that were pure noise, but they also leaked inline into real sentences: "Best restaurants PIRT and hotels in Barcelona".

The fix was a quality audit across all 38 books before ingesting anything: 50 pages sampled per book, 17 minutes, with explicit gates. Four inline reversed tokens remained; fixing them took five minutes. Testing on one clean book would have shipped all of them.

Not everything was caught. One heading survived as nonsense and shows up as a cited source:

Two embedding models, one silent bug

The plan said BGE-large, run locally. My laptop is a fanless M2, and embedding 5.7 million tokens overnight would have throttled it badly, so ingestion moved to Voyage's API. Then the backend was about to embed questions with local BGE while the documents were embedded with Voyage. Vectors from different models live in different spaces: the search still returns results, they are just wrong. Nothing errors; quality is simply bad, and you might not find out for months. I moved embedding into one shared package that ingestion and the backend both import, so they cannot drift apart. Months later the same bug turned up inside a safety net: the shared package fell back to BGE whenever Voyage failed, quietly mixing the two spaces again. Questions now use Voyage only, and an outage shows an honest error instead.

Wikivoyage's redirects

Wikivoyage often redirects a subpage title away from its parent. Ask for "New York City/Manhattan" and the API answers with the canonical title "Manhattan".

Deriving the parent from the canonical title left 16 subpages orphaned: no question could reach them. Deriving it from the title I requested fixed all 16.

Parent articles hid their listings

Parent articles nest some listings inside wrapper templates, and the parser lost part of them: 18% of Madrid's listings and 12% of Tokyo's. Subpages kept almost all of theirs (Eixample: 69 of 69). That finding settled the scope question: ingest parents and subpages, or the most useful details are simply not there.

Secrets with quotes in them

The first Wikivoyage embedding run crashed on a database URL. The .env value had literal quotes; Python's loader stripped them, but the shell command I used kept them, which broke the URL parser. Worse, the error message printed the full connection string, password included, into a log file. I deleted the log, rotated the password, and added a scrubber that removes credentials from database errors before they are raised. The rule now: load secrets in Python, never with shell substitution.

4.Final results

The most important result is not in the tables: the second corpus went in without touching the chunker. Building DocumentStructure as the contract between ingestion and chunking meant a new source needed only a new adapter.

5.Learnings

1. Data quality sets the ceiling. Every artifact that slips past cleaning corrupts a chunk, then a retrieval, then an answer. The cleaner the data, the less you need from the model.
2. Audit across the whole corpus before scaling. One clean sample hides every pattern the other books break in. Seventeen minutes of auditing 38 books found what weeks of testing one would not.
3. Shared contracts belong in shared code. "Both sides use the same embedding model" is a contract. Put it in one package, so it cannot drift.
4. Make the middle of the pipeline a contract. DocumentStructure let two very different sources, PDFs and wikitext, feed one unchanged chunker.
5. Match chunk size to the question. One size is always wrong for someone. Three sizes and a router let each question choose its zoom level.
6. Broad first, precise second. Fast approximate search narrows 22,000 chunks to 50; a slower, smarter reranker picks 5. Each step does only what it is good at.
7. Know your source's shape. Wikivoyage keeps details on subpages and redirects their titles. A pipeline that ignores how its source is organised retrieves the wrong level of detail.
8. Your error messages are a data leak. A database error printed a password into a log. Scrub credentials where errors are raised, not where they are displayed.

The short version: in RAG, the unglamorous work before the model is the work that decides the answer.