Most RAG "evals" aren't evals

If your RAG evaluation process is "someone looks at ten outputs and says it looks fine," that's not an eval pipeline — it's a vibe check that happens to run once.

A real one needs a golden dataset of realistic queries with known-good answers, automated scoring for retrieval relevance (not just final answer quality), and regression gates so a chunking or prompt change that quietly breaks something gets caught before it ships, not after a user notices.

I wrote the full pipeline breakdown in The Complete RAG Evaluation Pipeline if you want the deeper version — this is the short one.