Validate your data before you let an LLM near it

<p>Most retrieval-augmented generation projects that disappoint people do not fail because the wrong model was chosen. They fail because the data going in was never checked, and a language model is remarkably good at making unchecked data sound authoritative.</p> <p>I built a clinical dataset reporting tool for my Master's major project, and the single design decision that mattered most had nothing to do with the model: <strong>validation happens before generation, not after</strong>. Here is why, and what that looks like in practice.</p> <h2>The failure modes nobody mentions in the demo</h2> <p>A demo uses three tidy documents. Real data arrives as a spreadsheet somebody has been maintaining by hand since 2019. Four things go wrong, quietly.</p> <h3>1. Silent type coercion</h3> <p>You load a CSV, pandas infers the types, and a column that is 4,000 integers and one cell reading <code>N/A</code> becomes a column of strings. Identifiers with leading zeros lose them. A date column arrives in three formats and half of it parses as something plausible but wrong.</p> <p>Nothing raises an error. The pipeline runs. The numbers are subtly incorrect from that moment on.</p> <h3>2. <code>dropna()</code>, the quiet data loss</h3> <p>Somewhere in most data pipelines there is a line that drops rows with missing values. It is a reasonable thing to do, and it is almost never logged. Nobody can tell you afterwards whether it removed eleven rows or eleven hundred, or whether the removed rows shared something in common &mdash; like all coming from the one site that formats its exports differently.</p> <p>Dropping data is fine. Dropping data silently is not.</p> <h3>3. Duplicates that crowd out the answer</h3> <p>The same record exists three times with minor differences in whitespace. All three get embedded. A query retrieves all three, which means a third of your retrieved context is one record repeated, and something genuinely relevant got pushed out to make room.</p> <h3>4. Chunking that separates a value from its meaning</h3> <p>Split a table naively and you get a chunk full of numbers whose column headers and units stayed behind in the previous chunk. The model now sees <code>8.4</code> with no idea whether that is a percentage, a count, or a measurement in some unit. It will still produce a fluent sentence about it.</p> <h2>Why retrieving more is not the fix</h2> <p>The instinct when answers are poor is to widen retrieval &mdash; pull 40 chunks instead of 8 and let the model sort it out.</p> <p>There is good evidence this backfires. In <em>Lost in the Middle: How Language Models Use Long Contexts</em> (Liu et al., 2023, published in TACL), the authors found that model performance is highest when the relevant information sits at the beginning or the end of the context, and <strong>degrades significantly when the model has to find it in the middle</strong> of a long context. This held even for models explicitly built for long contexts.</p> <p>So padding the context with mediocre matches does not give the model more to work with. It buries the good match in the middle, which is the worst place for it to be. Precision in retrieval beats volume, and precision starts with clean, deduplicated source data.</p> <h2>The validation gate</h2> <p>What I settled on is a gate that data has to pass before it is allowed anywhere near an embedding model.</p> <ol> <li><strong>Declare the schema explicitly.</strong> In Python that means Pydantic models with real types, ranges and allowed values &mdash; not whatever pandas inferred on the day. A field that should be a positive integer under 200 says so.</li> <li><strong>Validate at ingestion, and quarantine failures.</strong> Rows that fail go to a rejects table along with the rule they broke and their original position in the file. They are not dropped and they are not silently fixed. Somebody can look at them and decide.</li> <li><strong>Deduplicate on a real key.</strong> Whole-row equality catches almost nothing, because real duplicates differ by a trailing space. Pick the fields that actually identify a record and normalise them first.</li> <li><strong>Normalise units and dates exactly once,</strong> at the boundary, and record what was converted.</li> <li><strong>Only validated rows get embedded.</strong> This is the whole point of the gate.</li> <li><strong>Carry provenance into the index.</strong> Every chunk keeps a reference to the row it came from, so an answer can be traced back rather than taken on trust.</li> <li><strong>Make the output cite its rows.</strong> Not a vague source list &mdash; the actual records the number came from.</li> </ol> <h2>The part that changes how people use it</h2> <p>That last point does more for trust than any amount of model tuning.</p> <p>When a summary arrives with the underlying rows attached, two useful things happen. A reader who doubts a figure can check it in seconds instead of dismissing the whole tool. And when the tool is wrong, you find out <em>why</em> &mdash; usually a data problem you can fix &mdash; instead of concluding vaguely that the AI is unreliable.</p> <p>An ungrounded answer that happens to be correct and a grounded answer that happens to be correct look identical. They are not the same thing, and the difference only shows up on the day it matters.</p> <h2>If you are commissioning one of these</h2> <p>Two questions worth asking whoever is building it:</p> <ul> <li><em>What happens to a row that fails validation?</em> If the answer is "we clean the data first", ask what "clean" means and where the rejected rows go. If they cannot tell you, they are being dropped.</li> <li><em>Can I trace any number in the output back to a source row?</em> If not, you have a system that produces confident text, which is a different product from one that produces answers.</li> </ul> <p>The model is genuinely the easy part now. The work &mdash; and the reason these projects succeed or quietly get abandoned &mdash; is almost entirely in the data that reaches it and what you can prove about where an answer came from.</p>