Skip to content

12. Guidance: unit tests, component evals and black-box evals

The guide has the practice with code, and migrating.md the move from other tools. Two rules shape the design. Evals import the production code, so the runner’s change selection finds them: a test-only route, or a debug field in a production response, to reach a step is wrong, so import the step. A task throws only for infrastructure problems: a wrong answer is a failed assertion (test form) or a return value that a scorer fails (definition form).