2. Prior art
trialkit keeps ideas from other tools and avoids their couplings. The comparison holds the facts about each tool’s current behavior. Some lessons here come from one private TypeScript monorepo (hoisting churn, a duplicate SDK identity, a judge cache, lenient reply parsing, a judge outage that failed cases): they motivate the design and do not describe any tool today. A measurement that nobody outside can repeat stays out of public copy.
Kept: scorers as plain functions, a dataset, task and scorer shape with a repeat count, evals as ordinary tests in the app’s runner, and results linked to traces.
Avoided: a second copy of a shared SDK, hoisting churn, a constrained test framework, a second telemetry SDK, native binaries, servers and bundlers in a test dependency, vendor SDKs and agent frameworks that the app does not use, a high Node.js floor (the dependency policy answers each); a judge cache on by default, lenient reply parsing, a 100% pass-rate threshold for a nondeterministic task, a fixed prompt for each named scorer, and reporting from hooks.