Record and replay model replies
Use replay when required CI must exercise the real request builder, SDK and parser without a model call.
import { type Cassette, recordingFetch, replayFetch } from '@trialkit/replay';import { loadCassette, saveCassette } from '@trialkit/replay/node';
function recorder(): typeof fetch { const cassette: Cassette = []; // A fresh recording needs no existing file. return recordingFetch(fetch, cassette, { save: () => saveCassette('cassettes/triage.json', cassette), });}
function replayer(): typeof fetch { return replayFetch(loadCassette('cassettes/triage.json'), { hint: 'Record this request again.', });}- Give either wrapper to the app client’s
fetchoption. A replay matches the recorded request exactly; headers count only whenmatchHeadersnames them, and credentials must be redacted. - A changed prompt, model, rubric or case gets a replay 404 and therefore a trial
error. A person records that change. Usemode: 'auto'to replay hits and record misses. - Cassettes hold valid 2xx JSON replies and streams in call order. Replay tests wiring and scoring, not a new model sample; fix a bad recording and record again instead of editing it or hiding it with
xfail.
Use the same wrapper for task and judge calls. The replay reference covers cassette matching, redaction, streams and files, and examples/replay runs a whole eval offline.
Python has no replay package: record the HTTP replies with VCR.py and pytest-recording. See Record and replay for the recipe that makes a missing recording a trial error.