Example: history and PR gates
Copy this to generate real JSON Lines from a deterministic ticket router, then run trialkit report and trialkit compare. No model, API key, or network call.
Run these in this directory after the repository build. Node.js 22.23.4 or later.
pnpm testpnpm history # regression: exits 1pnpm history:pass # unchanged behavior: exits 0pnpm history fails for:
2 cases tagged regression changed from pass to fail or error (crash-and-charge, feature-and-login)1 set check changed from pass to fail or error (regression-cases)the trial pass rate changed by -33.3 pp (95% CI -56.3 to -8.8)pnpm history:pass keeps the priority rule and reports No regression..
prioritychecks categories in the order billing, account, bug, other.first-matchtakes the keyword that appears first. That candidate routescrash-and-chargeandfeature-and-loginto the wrong category.- The eval runs three trials and does not stop early. The command writes two baseline runs and one candidate run to a temporary directory, records the routing rule as
config.revision, and deletes the directory when it ends. These runs never enter the repository’seval-results/or the pull-request gate. - check.yml compares replayed evals from the pull request and its base. Failed baseline preparation fails the gate. Unmatched head series are
notCompared, and at least one series must compare. evals.yml runs manually, saves histories on theeval-historybranch, and its live model calls cost money. See History and reports and the regression rule.
| File | Role |
|---|---|
src/classify.ts |
The priority baseline and the first-match candidate. |
src/tickets.ts |
Tickets and tags. |
src/triage.eval.ts |
The eval: three trials, no early stop. |
src/history.ts |
Writes the runs and calls the built trialkit CLI. |
scripts/history.ts |
pnpm history and pnpm history:pass. |
history.test.ts |
Checks the failing comparison and the passing one. |