lab: rag-lab / retrieval / evals
Context: an LLM can answer beautifully even when it has no right to be confident. For me, RAG starts not with an index but with the question: which answers can be verified and which sources must be visible to the user.
Problem: if you simply split documents and send top-k to the model, quality quickly becomes random. Poor chunking, noisy matches, no eval cases and no way to explain an answer turn the system into a black box.
Approach: I separate ingestion, retrieval, rerank and answer synthesis. Each layer needs logs: which document entered the context, why it was selected, which part of the answer it supports and where the model crossed the source boundary.
Conclusion: RAG becomes a production tool only when it has a small but living eval set. It catches regressions after changing the model, prompt, chunking or embedding provider.
Next step: build a quality dashboard: hit rate, citation coverage, hallucination flags and manual review for disputed answers.