@trialkit/node-test
Run trialkit evals as node:test tests. The shared test form explains cases, marks, run, score, policy, and trial outcomes.
Install
Section titled “Install”npm install --save-dev @trialkit/node-test @trialkit/corePeer: @trialkit/core; no other dependencies (node:test is part of Node.js). Node.js 22.23.4+, TypeScript 5.4+; ESM and CommonJS builds.
Example
Section titled “Example”import assert from 'node:assert/strict';import { describeEval } from '@trialkit/node-test';import { classify } from '../src/classify.ts';
describeEval('classify', { task: classify, trials: 3, verdict: 'majority' }, (it) => { it('billing', async ({ run }) => assert.equal(await run('charged twice'), 'billing'));});- Declare at the top level and run
node --test. Node’s TypeScript stripping needs.tsrelative imports; alternatively compile first. CommonJS can userequire('@trialkit/node-test'). - Exports:
describeEval,evalSuite.it.forpasses(row, ctx);node:testhas no table form. - The default reporter writes
eval-results/<eval name>.<run id>.jsonl;reporters: []writes nothing.
Reference
Section titled “Reference”describeEval(name, options, declare) and evalSuite(definition, options = {}) declare one test per case and a final set checks test. Shared validation, marks, and policy are in the Vitest reference; outcomes and closing rules are in runner adapters. The eval name and equal labels must be unique per file; invalid or unknown options, empty cases, and async declarations throw.
The shared option table applies to node:test. evalSuite takes its task, trial policy and checks from the definition.
node:test behavior
Section titled “node:test behavior”| Feature | Rule |
|---|---|
| Filtering | --test-name-pattern=<case id> matches a regex against case and suite names; matching the suite selects all. Excluded tests do not appear. it.only requires --test-only; it.skip uses the runner skip option. |
| Output | Counts are t.diagnostic messages (2/3 trials passed); spec prints them under the test, TAP as comments. Failures start 0/2 trials passed, needs 2; <last failure>, followed by other non-passing trials and assertion details. Infrastructure errors show InfrastructureError: 0/0 trials passed, 1 error, needs 2; <reason>, the task stack and cause; misuse keeps its class, such as TypeError. |
| Skips and warnings | t.skip(note) skips; xpass and partly skipped checks emit t.diagnostic('warning: ...'). |
| Snapshots / fake timers | t.assert.snapshot is unavailable in an eval body. Do not use mock.timers: it stops trial timeouts and is not detected. There are no soft assertions, assertion counts, or retries. |
| Lifecycle | beforeEach/afterEach run once per test, so trials share state. Concurrent runner tests are unsupported; evalSuite concurrency runs cases, not tests. |
| Timeout | Tests and the closing hook use the maximum timeout, 2147483647 ms; trialTimeoutMs limits each trial. Node 22.23.4 also applies --test-timeout to the whole file, which the adapter cannot override: use a large value or omit it. |
| Cancellation | The runner’s t.signal reaches the case; cancellation errors the trial and aborts the task and body signal. The closing hook still finishes the run. |
| Order / sharding | Tests use concurrency: false; do not use --test-randomize because set checks must run last. --test-shard shards by file; keep each check and its cases together. |
| Hooks / processes | declare runs outside describe, so hooks there apply to the file. The after hook fails the suite, not individual tests. Files run in separate processes; use { dir } for custom file reporters. |
Failed or errored cases show bounded trial logs with t.diagnostic before the error. Passing-case logs stay in reporters and the history report.
The closing hook fails for reporter errors, early stops, cases running without any verdict, runtime-skipped partial checks, unreported failed/errored checks, or a token budget without reported tokens. If set checks is filtered out, the hook reports failed or errored checks without repeating already reported failures. Definition-case and check failures use the declaration stack.
Definition-form look-ahead starts with the first case. CLI --test-name-pattern and --test-skip-pattern disable it, so excluded cases do not run. With name filters supplied only by programmatic run(), set concurrency: 1. Otherwise cases may run ahead; results stay in dataset order, events may interleave, a thrown callback or cancellation aborts in-flight work, and ahead-started tasks cannot rely on their test’s beforeEach. A token budget can overshoot by about one trial per concurrent case.
node:test has no built-in change selection.