A good demo proves that the system can work on the example you chose. It says much less about the cases that matter in production. Evaluation asks what happens across ordinary, difficult and adversarial cases, including the ones where the correct behaviour is to abstain, refuse or recover. Write the expected behaviour before looking at the output.
For Northstar, collect representative questions from policy owners and support staff. Include ambiguous dates, outdated documents, policy exceptions, different roles, unavailable tools and malicious content. Use synthetic personal records when real records are unnecessary, and preserve a held-out set that was not used to tune the prompt.
| Measure | What to inspect |
|---|---|
| Evidence retrieval | Whether required passages appear among permitted candidates |
| Grounding | Whether material claims follow from the supplied passages |
| Task correctness | Whether the response resolves the actual question with its exceptions |
| Structured output | Schema validity plus cross-field and business-rule checks |
| Authorization | Absence of prohibited data release and action execution |
| Tool completion | Correct operation, arguments, final status and handling of duplicates |
| User outcome | Verified task completion and total effort, including correction |
Retrieval recall tells you how much required evidence was found. Precision tells you how much of the retrieved material was relevant. Both matter, but neither rescues an answer when the one exception that changes the decision is missing. Score the evidence the model actually received, not the documents that happened to exist somewhere in the index.
Use deterministic checks for arithmetic, identifiers, schemas and access decisions. Use qualified human judgement where interpretation is required. A grading model can help at scale, but compare it with human decisions and inspect disagreements. An evaluator can share the same blind spots as the system it is grading. [1–2]
Record the model, prompt, retrieval configuration, data snapshot and tool versions. Repeat important cases when behaviour varies. Keep retrieval failures separate from generation failures so the next change addresses the cause rather than the symptom.
An evaluation result should tell the team what to change next. “Chunking loses exceptions in adjacent sections” and “timeouts can duplicate submissions” are actionable findings. “The model scored 87 percent” becomes useful only when the test population, rubric and consequences of the remaining failures are clear.