Skip to content

The levels

Use this table when you need to choose the smallest test that can answer the question.

Unit test Component eval Black-box eval
Calls Code around a model call One production step Product’s public interface
Model Fake Real Real
Other IO Fake Fake or absent Test environment
Answers Does code handle this reply? Does this step work for these inputs? Does the product work end to end?
Runs Every PR Changed import graph and on demand Schedule and on demand
Cost None One step’s calls Full pipeline

Required CI uses fakes and recordings, never a paid model call.