Evals for humans

A human eval is a scored check on an AI tool’s output, run by a person. It uses a few simple pieces: quick spot checks, a set of frozen “golden” questions, and a small set of tests you re-run after every change. The point is to write down what a pass looks like and to run the checks before a fluent wrong answer ships.

Say you ask the model for “weekly revenue by region, excluding refunds, fiscal calendar, same as Finance.” It returns clean SQL, a short explanation, and a join at the wrong grain, which means one row stands for something other than what you intended. Nobody had a golden card on hand to catch it.

Evals without the research lab vibe

In AI product work, “eval” (short for evaluation) often means automated scoring. That includes exact-match checks, one AI model grading another, benchmark suites, and red-team harnesses, which are test rigs built to attack a system. Those tools can be useful later. For an analytics person shipping queries, charts, and answers to stakeholders, the first win is much simpler.

First, you freeze a small set of tasks that matter at your company. Next, you define what “good enough” looks like. Then you run the same tasks after a model upgrade, a prompt rewrite, or a tool change, and you record pass or fail. That way you never have to argue from a feeling.

Think of it as unit tests for judgment. Software teams do not redeploy without a quick smoke suite, and analysts who use AI for production-grade work need the same muscle. The suite can live in a shared document. The “runner” can be you. What software teams call continuous integration (CI), the automatic re-check after every change, can be a calendar reminder after each vendor changelog email.

Rule of thumb: If a change could alter SQL, definitions, or numbers that someone else will trust, you need at least one frozen question that would have failed the last time things broke.

Industry risk guidance treats measuring and testing as part of responsible use, and not as optional polish. Frameworks such as the NIST AI Risk Management Framework talk about measuring, evaluating, and managing AI systems across their whole life. The U.S. National Institute of Standards and Technology (NIST) publishes it. You do not need to adopt it as a program to borrow the useful idea: if you cannot measure the failure modes you care about, you are guessing.

Three layers: spot-check, golden questions, regression set

Spot-checks: cheap, frequent, incomplete

A spot-check is what good analysts already do. You skim the join, sanity-check the total, run LIMIT 20, and compare against last week’s known number. Spot-checks catch obvious nonsense. They fail when you are tired, when the error is subtle, or when the model invents a plausible column that almost exists.

Spot-checks are necessary but not enough. If “looks fine to me” is your only test, wrong regional revenue will ship on a Friday afternoon.

Golden questions: frozen tasks with a known good shape

A golden question is a prompt you keep stable, paired with the shape of a good answer. It is not always one exact string. Here is an example: “Write SQL for active customers in Q2 using the certified definition in the catalog card.” Pass criteria might say the answer uses the approved customer table, applies the active flag correctly, filters on the right date field, does not invent columns, and matches a known row count range on the sandbox.

Golden does not mean “one true answer forever,” because metrics and schemas change. It means “we agreed what good looked like on this date.” You update the card when the business definition changes, and never because the model felt creative.

Regression set: the same ten or so tests every time something moves

A regression set is a short suite you re-run after a change. Ten prompts is enough to start, and twenty is better once you have been burned a few times. The goal is not statistical glory. It is to catch a case like “this used to refuse hallucinated columns and now it invents three” before the whole team upgrades their default model.

The loop has five steps. Pick the suite, run it under the new setting, log pass or fail against the criteria, and investigate the fails. Then decide whether to adopt the change, and keep the log. It is boring on purpose.

Notice that the diagram does not start with “buy an eval platform.” It starts with questions and a log. Tools can help later, but the habit has to exist first.

What to score for analytics work

Generic chat rubrics such as “helpful, honest, harmless” are fine for customer support demos. Analytics needs sharper categories, so score the things that break trust. The table below lists six.

You can score pass or fail on each row, or use a simple 0 or 1 overall with notes. Fancy rubrics with five decimal places will not save you if nobody runs them. For the first month, a binary score plus a comment field wins.

Writing pass criteria people will actually use

A bad criterion reads “Answer should be accurate and professional,” because that invites an argument. Better criteria are checkable. Prefer bullets that a second person could apply without reading your mind.

• Must include: specific tables, filters, or phrase patterns, such as “uses order_date and not created_at.”
• Must not include: forbidden patterns, such as SELECT * on the customer table with personal data, or joins on free-text names.
• Numeric band: a sandbox row count or total inside a known range, when you have a reference query.
• Structure: SQL first and explanation second, or assumptions listed before the code.
• Refusal: for adversarial prompts, a pass means the model declines or asks for the missing schema instead of inventing it.

When the business definition changes, update the golden card and the reference query. Do not “fix” the model by lowering the bar so that the new answer passes.

Starter golden set: ten prompt types

Copy this into a spreadsheet and replace the toy names with your real certified objects. Keep the prompt text stable for a quarter unless the business rules change. The figure below shows the same idea as a wall card, with types instead of one-off chat logs.

Below is the same starter set in a form you can paste into a document. Each row is a type, and you write one concrete prompt per type for your own warehouse.

Types 9 and 10 are not “nice to have.” They test whether the tool behaves like a reckless intern. Types 3 and 1 catch the most common analytics failures, which are hallucinated structure and quiet redefinition of metrics. Type 2 is the join tax that every warehouse pays.

Worked example: one golden card in the wild

Imagine you own a weekly revenue card. The certified grain is one row per order. Net revenue is gross_amount - refund_amount, counting completed refunds only. The fiscal week starts on Sunday, and region means shipping country and not sales territory. You put these rules in the prompt as constraints, as the earlier post on prompting for analysts explained, and you also put them on the eval card so the human scorer does not improvise.

Frozen prompt (toy):

Using ONLY these tables and columns:
orders(order_id, order_date, shipping_country, gross_amount, status)
refunds(order_id, refund_amount, refund_status, refunded_at)

Write SQL for weekly net revenue by shipping_country for fiscal weeks
(Sunday start) in 2025. Exclude orders with status = 'cancelled'.
Only subtract refunds where refund_status = 'completed'.
Show SQL first, then a short explanation of grain and exclusions.
If anything is ambiguous, ask instead of inventing columns.

Reference sketch you keep private (or in the eval sheet, but not in the model prompt every time):

SELECT
  DATE_TRUNC('week', order_date) AS fiscal_week_start, -- confirm Sunday
  shipping_country,
  SUM(o.gross_amount) - COALESCE(SUM(r.refund_amount), 0) AS net_revenue
FROM orders o
LEFT JOIN refunds r
  ON o.order_id = r.order_id
 AND r.refund_status = 'completed'
WHERE o.status <> 'cancelled'
  AND o.order_date >= DATE '2025-01-01'
  AND o.order_date < DATE '2026-01-01'
GROUP BY 1, 2
ORDER BY 1, 2;

Your week-start function may differ by warehouse. The point of the reference is a known good shape, not copy-paste religion. When you score a model’s answer, check five things.

1. The join key is order_id, not country or name.
2. The refund filter counts completed refunds only, and it is applied so that cancelled orders do not still inflate the total.
3. The week truncation matches the Sunday fiscal rule, or the answer explicitly flags a different default.
4. No columns appear outside the provided schema.
5. The SQL appears before the story.

If the model uses sales_region from a table you never listed, that is an automatic fail on schema honesty, even if the prose is beautiful. That is the whole point of human evals for analytics, because a beautiful explanation is not a pass criterion.

After you score, log one line.

date | model_or_prompt_version | case_id | pass_fail | axes_failed | notes
2026-07-23 | vendor-x-default | G1-revenue | fail | schema, time | used sales_region; week Mon start

Do this for all ten cases. It takes about twenty minutes once the cards exist and an hour the first time you write them. That hour is cheaper than one wrong board chart.

How often to run, and what counts as a change

Re-run the suite when any of the following moves.

• The default model or major version in your chat tool or code editor, which developers call an integrated development environment (IDE).
• The system prompt, the custom instructions, or the team’s “analytics assistant” template.
• Connected tools, such as a SQL runner, a warehouse connection built on the Model Context Protocol (MCP), a standard way to plug tools into an AI assistant, or a schema browser.
• The certified definitions on the cards your prompts depend on.
• The arrival of a new teammate who will copy your workflow.

Between changes, keep spot-checks on every artifact headed for production. The golden suite is the seatbelt for upgrades, and spot-checks are the seatbelt for Tuesday tickets. This habit comes from data quality work, where fixed checks beat heroics. You can see the spirit of profiling and validation in the data quality series.

Common mistakes (eval theater)

• Scoring only friendly prompts. If every case is “summarize this nice CSV,” you never test joins, refusals, or personal data.
• Changing the prompt while “evaluating the model.” Then you cannot tell what moved.
• Exact string match on SQL. Formatting differs from run to run, so score the structure and meaning, or compare results on a sandbox.
• Letting the same person write and score without criteria. Write the pass rules first, so scoring is not mood-based.
• Never failing anything. A suite that always passes is a branding exercise.
• Ignoring documentation drift. When Finance changes how refunds work, update the golden cards that week and not next quarter.
• Confusing a vibe check with accountability. The first post in this series made the point that an assistant is fine, but owning the number is still on you. Evals keep that honest under speed pressure.

How this ties to AI-written SQL and the rest of the site

The AI SQL checklist post walks through joins, filters, and confident nonsense for a single query. This post covers the habit around many queries over time. Use the checklist on every high-stakes draft, and use the golden set when your tool chain changes. Use the skills from the SQL series so you can actually see when the model is wrong. Use the discipline from the metrics and KPIs series so that “net revenue” is not a mood. Evals do not replace literacy, but they let your literacy scale.

Practice: build your first suite this week

1. Pick one domain you touch weekly, such as orders, tickets, content, or finance.
2. Write five golden prompts from the type table, with at least one refusal, one join trap, and one metric definition.
3. Add pass criteria as bullets and not as adjectives.
4. Run them on your current default tool, and log pass or fail.
5. Grow the set to ten, and put the sheet where the team can find it.
6. Schedule a re-run the next time someone says “we switched models.”

The next post in this series explains RAG in plain English. Retrieval-augmented generation (RAG) is the trick of pulling company documents into a prompt. The post covers when that helps analytics work and when it only papers over a messy wiki. For more paths, browse Learn.

Quick recap

• Human evals for analysts come down to frozen questions, clear pass criteria, and a pass or fail log.
• Spot-checks catch obvious errors, golden questions freeze the expected shape, and a regression set catches silent slips after changes.
• Score schema honesty, definition fidelity, filters, grain, safety, and explainability.
• Start with about ten prompt types that match real work, including refusals and join traps.
• Beautiful prose is not a pass criterion, and wrong numbers with nice explanations still fail.

Series notes

This is Part 4 of Practical AI for analytics people. Next: RAG in plain English.

Sources

• NIST, AI Risk Management Framework (AI RMF 1.0): https://www.nist.gov/itl/ai-risk-management-framework
• Stanford CRFM, HELM (Holistic Evaluation of Language Models): https://crfm.stanford.edu/helm/
• OpenAI, Evals framework and evaluation concepts: https://github.com/openai/evals
• OWASP, Top 10 for Large Language Model Applications: https://owasp.org/www-project-top-10-for-large-language-model-applications/
• Analytics Made Simple, What are LLMs?: https://analyticsmadesimple.com/analytics/what-are-llms-chatgpt-generative-ai-and-more/
• Analytics Made Simple, How to Check AI-Written SQL: https://analyticsmadesimple.com/tutorials/how-to-check-ai-written-sql/
• Analytics Made Simple, SQL series: https://analyticsmadesimple.com/series/sql/
• Analytics Made Simple, Data quality series: https://analyticsmadesimple.com/series/data-quality/

More definitions in Key Terms →

Merch on Etsy

Open store →

Naive Bayes Coffee Mug View on Etsy → Standard Deviation White Ceramic Mug View on Etsy → Git Version Control Shirt View on Etsy → You're On Mute Coffee Mug – Data Science Gift View on Etsy →