Skip to content

Time limits

Use these options when a model call is slow or can hang. You do not need to change the runner’s config.

import { describeEval } from '@trialkit/vitest';
describeEval('triage', { task, trials: 5, trialTimeoutMs: 30_000, runTimeoutMs: 600_000 }, (it) => {
it('refund', async ({ run }) => {
await run('How do I get a refund?');
});
});
  • The adapter turns off the runner’s own timeout for the eval’s tests and closing hook only; other tests keep theirs. A timeout mark that you put on a pytest eval stays in force. Each adapter README gives its mechanism.
  • trialTimeoutMs limits each trial; a trial that runs over ends in error, not fail. runTimeoutMs limits the whole eval. errorRetries and maxErrors limit retries and outage loops. Python uses trial_timeout_ms and run_timeout_ms. The defaults are in the run options.
  • node --test has its own --test-timeout for each whole file, and the adapter cannot change it. Leave the flag out, or run eval files in their own command (node:test adapter).