Most bad answers come not from the model but from the wrong context you fed it. How to make retrieval measurable.
It is the context, not the model
In RAG systems most wrong answers are not the model "hallucinating" — they are the wrong or incomplete chunk handed to it. Growing the prompt before fixing retrieval usually just burns money.
Chunking and metadata
Your chunking strategy decides the outcome. Chunks too large carry noise; too small and they sever context. Attaching metadata — source, title, date — to each chunk enables both filtering and citing the source in the answer.
Hybrid search
Vector similarity alone is not enough on most enterprise data. A hybrid approach that combines keyword (BM25) with vector search is markedly more accurate, especially for code, SKUs and domain-specific terms.
Measure retrieval
Before measuring answer quality, measure retrieval quality: did the right chunk land in the top-k (recall@k)? Without that number you will never know where the system breaks.
