How to test the answers of a RAG chatbot, even if you’re not an engineer

You do not need a research lab to test whether your AI answers are safe to send to customers. This works for a RAG chatbot, which answers by first looking things up in your own documents. Build a small set of real questions, score whether each answer sticks to your documents, and rerun that same test every time the documents or the model change.

Imagine your team shows you forty slides of AI metrics, and none of them answers your real question: is this assistant safe to put in front of customers? Now imagine a sheet of twenty real customer questions, each with the answer your team already knows is right. You can score the assistant against that sheet in an afternoon, and it tells you far more than the slides.

The four RAG quality checks, explained simply

For non-technical stakeholders, machine learning reports can read like a wall of jargon. The technical scores in those reports describe how the system works inside. They don’t answer the one question your business cares about. Can you trust this assistant to talk to your customers? RAG stands for retrieval-augmented generation, an AI system that looks things up in your documents before it answers. Testing one comes down to four plain checks. Did the answer stick to the real text? Did it answer the actual question? Did the search find everything it needed? And did it drag in a pile of irrelevant junk along the way?

The TruLens project calls three of these checks the RAG Triad; context recall is the fourth. Here is what each one means, in terms any operations manager, compliance officer, or product lead can use without a technical background.

1. Faithfulness. Ask: is each claim in the answer backed by the text the system pulled up? Say the model states three facts. Two of them match the source documents word for word, but the third one is invented. That gives you a faithfulness score of 0.67, two out of three claims held up under a check. In regulated fields like healthcare, banking, and insurance, faithfulness is not optional. Set your release bar high, such as 0.90 or more, as the script (a short program that runs steps automatically) below does. A single invented policy line can cause real financial or medical harm.

2. Answer relevance: did the assistant answer the question that was asked? An assistant might return a flawless summary of general dental benefits when someone asked specifically about braces. Every word could be true and it would still be useless, because it dodges the real question. Answer relevance checks whether the reply matches what the person wanted instead of hiding behind generic boilerplate.

3. Context recall: this checks the search step itself, did the system find each passage it needed for a complete answer? Say a clinical rule requires both a diagnosis code and a list of exceptions, but the search only pulled the diagnosis code. That gives you a context recall of 0.50, half the story delivered. Low recall usually means your document chunks are cut too short, or your filters are too strict.

4. Context precision: how much of what the search pulled in was useful? If a system grabs ten chunks of text to answer one simple question and only two of them matter, precision is low. That waste costs you three ways. Each question costs more, the answer comes back slower, and the model is more likely to get confused by the noise.

Vibe checking vs a real golden test set

A common cause of embarrassing RAG failures is something teams call vibe checking. Here’s what that looks like. Someone changes a prompt or swaps a model, types five questions into a test window, and skims the answers. It looks pretty good, so it ships.

Vibe checking is risky for AI that real customers will use, for two reasons.

It can’t be repeated. Three people testing the same assistant will each type different questions, so there is no fair way to compare one version with another.

It also misses damage quietly. In regular software, fixing a bug in one spot can break something else nearby, and RAG works the same way. Trimming a prompt to sound less wordy might also make the model stop mentioning a policy exclusion on a tricky legal question. Without a fixed set of test cases you rerun each time something changes, you will not learn what broke until a customer finds it first.

The fix is what people call a golden set: a curated list of fifty to one hundred real questions, kept in an ordinary spreadsheet. Each row holds four things: the question, which document it comes from, the exact line that answers it, and the answer your team has already approved as correct. Because the people who know the business rules already know the right answers, a golden set like this can often be built in an afternoon, not with a research budget.

A testing cycle that never really stops

Testing your RAG system once before launch is not enough. Policies change, new documents get uploaded, and the questions people ask shift over time. Your test set needs to keep pace. A frozen one goes stale.

Here is how non-engineers take part in each of the four stages.

Stage 1: Build the golden questions. People who know the subject pick roughly fifty questions that define what success looks like. A strong set mixes three kinds of questions. Include routine ones that make up most daily traffic and tricky cases that mix conflicting rules. Also include questions the system should refuse, to check that it says no when it should.

Stage 2: Run the automated test suite. A script takes that spreadsheet and runs each question through the live system, records what it retrieved and what it answered, then scores faithfulness and relevance for each one. This step can run overnight, or automatically each time the code changes.

Stage 3: Review the low scores. Instead of reading through hundreds of good answers, a business analyst looks only at the ones that fall below the bar, say anything under 0.90 on faithfulness. In minutes they can tell whether the problem was a missing document, a bad split when documents were cut into short passages (called chunking), or the model simply making something up.

Stage 4: Publish a clear scorecard. Before any change to the prompt, the embedding model (the model that turns text into lists of numbers for search), or the chunking setup goes live, the test suite prints a plain scorecard. If the new version clears the bar on all four measures, leadership can sign off with real confidence instead of a guess.

How different roles share the testing work

Good AI teams split testing into three jobs. People who know the subject write the test questions and review the tricky failures by hand. An automated system scores every answer the same way each time. And the release process blocks any version that falls short.

When engineers write test questions on their own, without the people who know the business rules, the questions tend to be too simple or unrealistic. When business teams try to test manually by pasting questions into a chat window one at a time, testing gets slow and inconsistent. Fast-moving teams fix this by connecting both groups through one shared workflow instead of two separate ones.

This split lets each person do what they are good at. The subject matter expert does not need to learn to code or set up a database; they keep the business truth in a spreadsheet they already know how to use. The automated suite runs hundreds of tests in parallel using the same rubric each time. And the deployment (the step that puts a new version live) pipeline simply refuses to ship code if the faithfulness score drops.

A simple Python script for testing RAG answers

You do not need an expensive enterprise platform to start testing. The script below checks each test case, scores faithfulness and relevance, and prints a plain scorecard you could hand to a manager.

It works through your list of test cases and pulls out the claims in each answer. Then it checks whether the source text backs each claim up and prints a short report.

from dataclasses import dataclass
from typing import List, Dict

@dataclass
class GoldenTestCase:
    test_id: str
    query: str
    ground_truth: str
    retrieved_context: str
    generated_answer: str

@dataclass
class EvalResult:
    test_id: str
    faithfulness: float
    answer_relevance: float
    status: str
    audit_notes: str

class LightRAGEvaluator:
    def __init__(self, faithfulness_floor: float = 0.90, relevance_floor: float = 0.80):
        self.faithfulness_floor = faithfulness_floor
        self.relevance_floor = relevance_floor

    def evaluate_test_case(self, tc: GoldenTestCase) -> EvalResult:
        # Split generated answer into candidate claims
        sentences = [s.strip() for s in tc.generated_answer.split('.') if len(s.strip()) > 5]
        if not sentences:
            return EvalResult(tc.test_id, 0.0, 0.0, "FAIL", "Empty generated response")

        # Groundedness Check: Check claim presence in retrieved context
        supported_claims = 0
        context_lower = tc.retrieved_context.lower()
        for s in sentences:
            # Check key token overlap against context
            words = [w.lower() for w in s.split() if len(w) > 3]
            if words:
                overlap = sum(1 for w in words if w in context_lower) / len(words)
                if overlap >= 0.70:
                    supported_claims += 1

        faithfulness = round(supported_claims / len(sentences), 2)

        # Relevance Check: Check question keyword addressing in answer
        q_words = [w.lower() for w in tc.query.split() if len(w) > 3]
        ans_lower = tc.generated_answer.lower()
        if q_words:
            rel_overlap = sum(1 for w in q_words if w in ans_lower) / len(q_words)
            answer_relevance = round(min(1.0, rel_overlap * 1.2), 2)
        else:
            answer_relevance = 1.0

        # Determine release status
        if faithfulness >= self.faithfulness_floor and answer_relevance >= self.relevance_floor:
            status = "PASS"
            notes = "Meets production release criteria"
        elif faithfulness < self.faithfulness_floor:
            status = "FAIL_FAITHFULNESS"
            notes = f"Hallucination detected: Faithfulness {faithfulness} below {self.faithfulness_floor}"
        else:
            status = "FAIL_RELEVANCE"
            notes = f"Evasive response: Relevance {answer_relevance} below {self.relevance_floor}"

        return EvalResult(tc.test_id, faithfulness, answer_relevance, status, notes)

# Simulate HealthTech Benefits Golden Test Cases
test_suite = [
    GoldenTestCase(
        test_id="BEN-01",
        query="What is the annual visit limit for pediatric speech therapy under Silver PPO?",
        ground_truth="Pediatric speech therapy is capped at 30 visits per calendar year.",
        retrieved_context="Silver PPO Plan Benefits: Pediatric speech therapy is capped at 30 visits per calendar year. Pre-authorization required after session 6.",
        generated_answer="Under the Silver PPO plan, pediatric speech therapy is limited to 30 visits per calendar year with pre-authorization required after session 6."
    ),
    GoldenTestCase(
        test_id="BEN-02",
        query="Are out-of-network physical therapy visits covered under Silver PPO?",
        ground_truth="Out-of-network physical therapy visits are not covered.",
        retrieved_context="Silver PPO Plan Exclusions: Out-of-network outpatient physical therapy is strictly excluded from coverage.",
        generated_answer="Out-of-network physical therapy visits are covered at 100% after satisfying your out-of-pocket maximum."
    )
]

evaluator = LightRAGEvaluator()
results = [evaluator.evaluate_test_case(tc) for tc in test_suite]

print("RAG QUALITY RELEASE SCORECARD:")
for r in results:
    print(f"[{r.status}] Test ID: {r.test_id}")
    print(f"  Faithfulness: {r.faithfulness:.2f} | Relevance: {r.answer_relevance:.2f}")
    print(f"  Audit: {r.audit_notes}\n")

Running this script against the sample benefits questions catches the made-up answer that could have put the whole project at risk. Here is the scorecard it produces:

Summary and key takeaways

High-performing RAG projects do not guess at quality. They let the people who know the subject manage a golden test set, hold every release to clear numeric bars, and drop the habit of vibe checking for good.

Here is what this post covered:

• End vibe checking: Informal chat tests give you zero protection against regressions, so keep a structured, version-controlled test set instead.
• Learn the four checks: Faithfulness, answer relevance, context recall, and context precision together tell you whether a system is ready for real use.
• Let non-engineers own the questions: The people who know the business should manage golden question-answer pairs in a plain spreadsheet, not in code.
• Set a release bar: Require a minimum score, such as 0.90 faithfulness, before anything new goes live.

The next post looks at when other approaches beat RAG. One is fine-tuning, which means extra training that adjusts a model on your own examples. Another is a bigger context window, the amount of text a model can read at once.

Your next step

Ask the people who know the subject to write twenty real questions with the passage that answers each, in a spreadsheet. Run your RAG system on them and score faithfulness and relevance. That first scorecard is the baseline every later change has to beat.

Sources and further reading

• Es and colleagues, 2023, RAGAS: Automated Evaluation of Retrieval Augmented Generation, https://arxiv.org/abs/2309.15217
• TruLens documentation, The RAG Triad, https://www.trulens.org/getting_started/core_concepts/rag_triad/
• Saad-Falcon and colleagues, 2023, ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems, https://arxiv.org/abs/2311.09476
• Gao and colleagues, 2023, Retrieval-Augmented Generation for Large Language Models: A Survey, https://arxiv.org/abs/2312.10997

More definitions in Key Terms →

Merch on Etsy

Open store →

Python Coder Shirt View on Etsy → Apache Iceberg T-Shirt View on Etsy → Apache Hadoop T-Shirt View on Etsy → Naive Bayes T-Shirt View on Etsy →

Series notes

This is Part 7 of RAG from scratch, and it closes the planned RAG track. Previous: failure modes.