Skip to content

@trialkit/node-test

Run trialkit evals as node:test tests. The shared test form explains cases, marks, run, score, policy, and trial outcomes.

Terminal window
npm install --save-dev @trialkit/node-test @trialkit/core

Peer: @trialkit/core; no other dependencies (node:test is part of Node.js). Node.js 22.23.4+, TypeScript 5.4+; ESM and CommonJS builds.

import assert from 'node:assert/strict';
import { describeEval } from '@trialkit/node-test';
import { classify } from '../src/classify.ts';
describeEval('classify', { task: classify, trials: 3, verdict: 'majority' }, (it) => {
it('billing', async ({ run }) => assert.equal(await run('charged twice'), 'billing'));
});
  • Declare at the top level and run node --test. Node’s TypeScript stripping needs .ts relative imports; alternatively compile first. CommonJS can use require('@trialkit/node-test').
  • Exports: describeEval, evalSuite. it.for passes (row, ctx); node:test has no table form.
  • The default reporter writes eval-results/<eval name>.<run id>.jsonl; reporters: [] writes nothing.

describeEval(name, options, declare) and evalSuite(definition, options = {}) declare one test per case and a final set checks test. Shared validation, marks, and policy are in the Vitest reference; outcomes and closing rules are in runner adapters. The eval name and equal labels must be unique per file; invalid or unknown options, empty cases, and async declarations throw.

The shared option table applies to node:test. evalSuite takes its task, trial policy and checks from the definition.

Feature Rule
Filtering --test-name-pattern=<case id> matches a regex against case and suite names; matching the suite selects all. Excluded tests do not appear. it.only requires --test-only; it.skip uses the runner skip option.
Output Counts are t.diagnostic messages (2/3 trials passed); spec prints them under the test, TAP as comments. Failures start 0/2 trials passed, needs 2; <last failure>, followed by other non-passing trials and assertion details. Infrastructure errors show InfrastructureError: 0/0 trials passed, 1 error, needs 2; <reason>, the task stack and cause; misuse keeps its class, such as TypeError.
Skips and warnings t.skip(note) skips; xpass and partly skipped checks emit t.diagnostic('warning: ...').
Snapshots / fake timers t.assert.snapshot is unavailable in an eval body. Do not use mock.timers: it stops trial timeouts and is not detected. There are no soft assertions, assertion counts, or retries.
Lifecycle beforeEach/afterEach run once per test, so trials share state. Concurrent runner tests are unsupported; evalSuite concurrency runs cases, not tests.
Timeout Tests and the closing hook use the maximum timeout, 2147483647 ms; trialTimeoutMs limits each trial. Node 22.23.4 also applies --test-timeout to the whole file, which the adapter cannot override: use a large value or omit it.
Cancellation The runner’s t.signal reaches the case; cancellation errors the trial and aborts the task and body signal. The closing hook still finishes the run.
Order / sharding Tests use concurrency: false; do not use --test-randomize because set checks must run last. --test-shard shards by file; keep each check and its cases together.
Hooks / processes declare runs outside describe, so hooks there apply to the file. The after hook fails the suite, not individual tests. Files run in separate processes; use { dir } for custom file reporters.

Failed or errored cases show bounded trial logs with t.diagnostic before the error. Passing-case logs stay in reporters and the history report.

The closing hook fails for reporter errors, early stops, cases running without any verdict, runtime-skipped partial checks, unreported failed/errored checks, or a token budget without reported tokens. If set checks is filtered out, the hook reports failed or errored checks without repeating already reported failures. Definition-case and check failures use the declaration stack.

Definition-form look-ahead starts with the first case. CLI --test-name-pattern and --test-skip-pattern disable it, so excluded cases do not run. With name filters supplied only by programmatic run(), set concurrency: 1. Otherwise cases may run ahead; results stay in dataset order, events may interleave, a thrown callback or cancellation aborts in-flight work, and ahead-started tasks cannot rely on their test’s beforeEach. A token budget can overshoot by about one trial per concurrent case.

node:test has no built-in change selection.