The levels
Use this table when you need to choose the smallest test that can answer the question.
| Unit test | Component eval | Black-box eval | |
|---|---|---|---|
| Calls | Code around a model call | One production step | Product’s public interface |
| Model | Fake | Real | Real |
| Other IO | Fake | Fake or absent | Test environment |
| Answers | Does code handle this reply? | Does this step work for these inputs? | Does the product work end to end? |
| Runs | Every PR | Changed import graph and on demand | Schedule and on demand |
| Cost | None | One step’s calls | Full pipeline |
Required CI uses fakes and recordings, never a paid model call.