Open source · 2026
RAG systems don't crash, they rot
A changed PDF template, a parser upgrade, a stale index rebuild — and retrieval accuracy falls off a cliff while every dashboard stays green. ragdrift catches it before re-ingestion.
A retrieval system that breaks loudly is a good day. The failure that costs you is the quiet one: the corpus is re-ingested, a parser version moves, tables stop aligning, and the system keeps answering fluently from worse evidence. Latency is fine. Error rates are fine. The answers are wrong.
I hit this on a production legal system. The symptom was confident citations to the wrong authority; the cause was an index quietly covering a fraction of the corpus. Nothing in the stack was built to notice.
The detection lag is the problem
Without an instrument, the feedback loop runs through users. Someone notices answers got worse, someone escalates, and eventually a human bisects a re-ingestion. That path takes weeks.
ragdrift closes it by probing a corpus with golden queries before and after ingestion, then correlating retrieval scores against answer accuracy. A drop is attributed to a document, a heading shift, or a table that stopped parsing — not to a vague sense that things feel worse.
Run it in two commands
pip install ragdrift ragdrift demo --inject-drift
Zero config, zero API keys, no Docker. The demo copies twenty sample documents, corrupts some of them the way a real pipeline does, and shows the scan finding it: an 11.9 percentage-point accuracy drop across six documents, detected in 0.2 seconds.
Add --explain and a judge model diagnoses the root cause and suggests the prevention. That is what turns an alert into an action.
Where it fits
v0.2.0 ships an MCP server and LangSmith cost tracing, so drift surfaces inside the tools a team already has open rather than in a report nobody runs. That was the design constraint throughout: an instrument only works if it lives where the work happens.
It is deliberately not an observability platform. It answers one question — has retrieval quietly got worse since the last ingestion — and the README is explicit about when it is the wrong tool.