Skip to content

Python

Use pytest when your app is Python and you want the same trial policies, JSON Lines records and CLI reports. The example at the top of this guide is the whole setup.

  • eval_suite(definition) is the shared-definition form. Python and TypeScript write the same JSON Lines events, hashes and run ordering; the default file is under eval-results/.
  • skip and xfail are non-empty reasons or None. Use --dist loadfile for a module eval (--dist loadscope for a class) so xdist does not split one run.
  • Do not call a blocking SDK or time.sleep in an async def task. Use a plain def for unbounded synchronous work; core runs it in a daemon thread. A cancelled coroutine gets one loop turn, then core closes it.

See pytest-trialkit, trialkit-core, trialkit-scorers and trialkit-judge for options, defaults and fake-model tests.