A better demo is not an evaluation dataset

Begin with the work people bring

A support assistant is demonstrated with five clear questions whose answers appear in short policy paragraphs. It responds well. In daily use, staff ask incomplete questions, refer to old product names and combine two issues in one request. The demonstration did not test those conditions.

Build the dataset around task types rather than around impressive answers. Identify common requests, consequential exceptions and cases the system should decline or escalate. Use realistic language, including the ambiguity people actually introduce.

A dataset does not need to be large before it is useful. A small reviewed collection covering distinct failure mechanisms can reveal more than hundreds of generated paraphrases of the same easy question.

Store an expectation, not just an ideal sentence

For each case, record the input, relevant source version or test state and the behaviour that counts as success. That may be a set of required facts, the correct tool target or a refusal to act without approval.

Avoid requiring one exact sentence when several answers would be correct. Conversely, do not use a vague label such as "good answer" for a task where an amount, exception or destination must be precise. Match the assertion to the property being tested.

Keep development and assessment distinct

Examples used repeatedly to tune a prompt become part of the development process. They remain useful regression cases, but their scores provide weaker evidence about unfamiliar requests. Maintain a held-out set that is not used to optimise each small change.

Split related examples together where appropriate. If a training example and test example are near-identical questions about the same source paragraph, changing a few words does not create much independence. Group by source, scenario or task family according to the generalisation you want to test.

Keep a separate challenge set for rare, serious failures. It need not match normal traffic proportions. Its purpose is to exercise boundaries that an ordinary sample might miss.

Make the release judgement explainable

Report results by task and consequence, with sample sizes and examples of regressions. A higher average can hide a new failure on an important approval rule. Include refusal and escalation rates so a candidate cannot improve apparent correctness simply by avoiding useful work.

Version the dataset, source snapshot and evaluator configuration with each run. When a result changes, the team should be able to determine whether the system improved, the test changed or the reviewer applied a different standard. That traceability turns a collection of prompts into evidence the team can use.