Testing and evaluating LLM applications
Unit tests, eval suites, LLM-as-judge and regression testing for non-deterministic code.
7 min read
LLM output is non-deterministic and "correct" is often fuzzy, so normal unit tests aren't enough. You need a layered approach.
Layers
- Deterministic unit tests for everything around the model: prompt rendering, parsers, tool implementations, retry logic. Mock the model client.
- Contract tests that the model's output validates against your schema across a sample of inputs.
- Eval suites: a dataset of inputs with expected outputs or grading criteria, scored automatically.
- Online evaluation: A/B tests, user feedback, guardrail and error-rate dashboards.
Kinds of graders
| Grader | Good for |
|---|---|
| Exact match / regex | Classification, extraction, IDs |
| Programmatic checks | JSON validity, length, required fields, code that compiles and passes tests |
| Similarity to reference | Short factual answers |
| LLM-as-judge with rubric | Tone, helpfulness, faithfulness, open-ended answers |
| Human review | Calibrating judges, high-stakes or subjective tasks |
Practical tips
- Start with 20–50 hand-picked examples that reflect real usage, including failures you've seen. Grow from production logs.
- Track scores per category so an improvement in one area doesn't hide a regression elsewhere.
- Run evals in CI on every prompt or model change; block merges on significant regressions.
- For non-determinism, run important cases several times and look at pass rates rather than single runs.
- Keep a red-team set of adversarial prompts (jailbreaks, injections) and track the attack success rate.