Python
Use pytest when your app is Python and you want the same trial policies, JSON Lines records and CLI reports. The example at the top of this guide is the whole setup.
eval_suite(definition)is the shared-definition form. Python and TypeScript write the same JSON Lines events, hashes and run ordering; the default file is undereval-results/.skipandxfailare non-empty reasons orNone. Use--dist loadfilefor a module eval (--dist loadscopefor a class) so xdist does not split one run.- Do not call a blocking SDK or
time.sleepin anasync deftask. Use a plaindeffor unbounded synchronous work; core runs it in a daemon thread. A cancelled coroutine gets one loop turn, then core closes it.
See pytest-trialkit, trialkit-core, trialkit-scorers and trialkit-judge for options, defaults and fake-model tests.