Definition
LLM evaluationRAGAS
Measuring the quality of an LLM or RAG system on a fixed set of real questions, with metrics such as faithfulness, relevance and retrieval hit-rate.
You can't improve what you don't measure. An evaluation set is a collection of representative questions with expected answers or sources, ideally written with domain experts. Every change to prompts, models or retrieval is scored against it.
Frameworks such as RAGAS provide metrics for RAG systems: whether the right context was retrieved, whether the answer is faithful to that context, and whether it addresses the question. Human review stays essential for high-stakes domains.
Related services
Related terms