Time limits
Use these options when a model call is slow or can hang. You do not need to change the runner’s config.
import { describeEval } from '@trialkit/vitest';
describeEval('triage', { task, trials: 5, trialTimeoutMs: 30_000, runTimeoutMs: 600_000 }, (it) => { it('refund', async ({ run }) => { await run('How do I get a refund?'); });});- The adapter turns off the runner’s own timeout for the eval’s tests and closing hook only; other tests keep theirs. A
timeoutmark that you put on a pytest eval stays in force. Each adapter README gives its mechanism. trialTimeoutMslimits each trial; a trial that runs over ends inerror, notfail.runTimeoutMslimits the whole eval.errorRetriesandmaxErrorslimit retries and outage loops. Python usestrial_timeout_msandrun_timeout_ms. The defaults are in the run options.node --testhas its own--test-timeoutfor each whole file, and the adapter cannot change it. Leave the flag out, or run eval files in their own command (node:test adapter).