Skip to content

@trialkit/report-junit

Write one JUnit XML file per eval run so CI can display evals as test results. This file reporter runs in Node.js.

Terminal window
npm install @trialkit/report-junit @trialkit/core

Peer: @trialkit/core; no other dependencies. Node.js 22.23.4+.

import { runEval } from '@trialkit/core';
import { junitFile } from '@trialkit/report-junit';
await runEval(definition, { reporters: [junitFile({ dir: 'reports' })] });
  • In a runner suite, reporters replaces the default file; add jsonlFile({ dir: 'eval-results' }) if needed.
  • Reporter errors appear in RunResult.reporterErrors and do not change outcomes.
  • Publish the XML with a CI test-report action; on GitHub Actions use if: always() to publish after a failure.

junitFile(options) returns a Reporter and writes at run completion. It keeps state per run and can serve concurrent runs. Exactly one non-empty option is required; both, neither, or an empty value throws TypeError.

Option Default Meaning
dir None Create the directory and write <eval name>.<run id>.xml, with the same name rule as jsonlFile.
path None Create parent directories and write exactly this path; an existing file is a reporter error and remains unchanged.

A <testsuite> is named for the eval, has summary counts and start timestamp, and contains properties for dataset.hash, trialkit.run_id, trialkit.ok, optional trialkit.stopped_by, optional trialkit.config_hash, and each run label as label.<key>. Each case is a <testcase> with eval classname and case-id name; checks use set checks > <check name>. A stopped run adds a failing run testcase with classname <eval name> [run verdict], distinct from every case and check. A non-passing run with no failing or errored case or check also adds one.

Outcome Element
pass None.
fail / error <failure> / <error> with reason and non-passing trial, score, and error details.
skip <skipped> with reason.
xfail <skipped> with known failure: <reason>.
xpass Pass with the reason in <system-out>.
Check fail / error / skip Same elements, with the check value in the text.

Case time is the sum of trial durations, in seconds to three decimals; checks have no time. XML text and attributes are escaped, and XML 1.0-forbidden characters are removed.

Each case with recorded logs has bounded, escaped per-trial log text in <system-out>, including passing and skipped cases. Keep JSONL for the full evidence.

Not built: config contents, run usage and cost, and trial metrics. Keep JSONL for these facts. No XML file is created until the run completes.