Test design for LLM features: assertions, rubrics and regression

What makes these features different to test

Three things. The output is non-deterministic, so a single run is a sample, not a result. The specification is partly the prompt, so a one-word change to the instructions is a code change that can alter behaviour anywhere. And quality has several dimensions (correct, complete, grounded, safe, in the right tone) that a single assertion cannot capture. The response is not to give up on assertions but to layer them, cheapest and most reliable first, as the automation pyramid layers tests by cost.

The method

1. 1

   Write the behaviour specification

   List what the feature must always do, must never do, and should do well. Must-always and must-never items become assertions; should-do-well items become rubric criteria.

2. 2

   Build deterministic checks

   Parse the output. Assert on format, schema, length limits, required fields, language, and forbidden strings or patterns. These run on every commit and never flake.

3. 3

   Add property checks

   Check relations that need no expected value: every figure in a summary appears in the source, every citation points to a retrieved passage, the answer does not change when irrelevant context is reordered.

4. 4

   Define the rubric

   For quality, write a scoring rubric with three to five criteria and anchored levels, and have reviewers apply it to a sample. Calibrate any model grader against those human scores before trusting it.

5. 5

   Run for rates

   Run each case three to five times at production settings. Pass criteria are rates: at least 98 percent of runs pass all deterministic checks, the mean rubric score on the suite stays at or above the baseline.

6. 6

   Freeze a regression suite

   Version 200 to 500 cases covering every slice and every past defect, and run it on each prompt, model, retrieval or tool change, comparing against the last accepted run.

7. 7

   Test the attack surface

   Add adversarial cases for prompt injection through user input and retrieved documents, data leakage, excessive tool permissions and unsafe requests, using the OWASP list as a checklist.

Model graders need testing too

Using one model to grade another is fast and, with care, useful. Zheng and colleagues found that strong model graders agreed with human preferences about as often as humans agreed with each other, and also documented position bias (preferring the first answer shown), verbosity bias (preferring longer answers) and self-preference. So test the grader like any other instrument: score at least 100 cases with both humans and the grader, report agreement, swap answer order to measure position bias, and re-check agreement whenever the grader's prompt or model changes. The approach mirrors the detector benchmark protocol: an instrument is trusted to the extent its error rate has been measured.

Prompts are code

Keep prompts in the repository, reviewed and versioned like source, and run the regression suite on every change. A prompt edit that fixes one complaint routinely breaks three other cases, and the suite is the only way to see it. Record the model version and settings with every run, because a supplier's silent update is a change the team did not make but must still test. The continuous testing guide covers the gates; treat a drop in the regression pass rate like a failing build.

Exploratory sessions still find what the suite cannot

Scripted evaluation measures the behaviour you anticipated. Exploratory testing finds the rest: charter a session on a risk (for example, users pasting long documents in another language) and let a tester probe for 60 to 90 minutes, then turn every finding into a regression case. For the evaluation data behind the suite, follow the test set design guide, and for the wider approach see how to test AI systems.

This guide is part of the testing AI systems hub. It is best read alongside how to test ai systems and QA of AI-generated test cases and code, which cover the neighbouring questions.