The RAG demo takes an afternoon. The RAG system that a compliance officer trusts takes engineering — and the difference is almost never the model.
Retrieval is the product
When a RAG system answers wrongly, autopsy the retrieval first: in our audits, four out of five failures happen before the model ever sees the context. Chunks split mid-thought. Tables shredded into noise. The 2023 policy outranking the 2026 revision because nobody indexed effective dates.
Permissions are not optional
Enterprise knowledge has boundaries — HR documents, deal rooms, board papers. Retrieval must respect the same ACLs as the source systems, enforced at query time, not hoped for at indexing time. This is the requirement that separates enterprise RAG architectures from tutorials.
Build the evaluation set before the system
Collect a hundred real questions with verified answers during discovery. That set becomes your speedometer: retrieval hit-rate, answer accuracy, and citation fidelity measured on every change. Teams without one are tuning blind and shipping vibes.
Freshness is a pipeline, not a batch job
Knowledge changes daily; quarterly re-indexing guarantees confident answers from expired truth. Production RAG treats ingestion like data engineering — event-driven updates, effective-date awareness, and deletion that actually deletes.