5. Architecture
TypeScript and Python packages provide the same core contracts, with separate integrations for runners, model clients and backends. An application installs only the integrations that it uses, and files in spec/ keep the two implementations aligned.
Package layout
Section titled “Package layout”Each integration covers one technology and all of its supported majors. Every package takes core as a peer (Python: a bounded requirement).
| Package | Role |
|---|---|
| @trialkit/core | Engine and model port; /node, /util and /adapter subpaths |
| @trialkit/scorers | Deterministic scorers |
| @trialkit/judge | Judges, templates, calibration; optional peers: SDKs and cases |
| @trialkit/vitest | Vitest adapter |
| @trialkit/jest | Jest adapter |
| @trialkit/node-test | node:test adapter |
| @trialkit/replay | Record and replay HTTP |
| @trialkit/cases | YAML and TOML case files |
| @trialkit/report-junit | JUnit XML reporter |
| @trialkit/report-otel | OpenTelemetry reporter |
| @trialkit/report-mlflow | MLflow reporter and uploader |
| trialkit | CLI: report and compare |
| @trialkit/studio | The local page of §15 |
| trialkit-core | Python engine, readers and reporters |
| trialkit-scorers | Python scorers |
| trialkit-cases | Python case files |
| trialkit-judge | Python judges and an OpenAI-compatible client |
| pytest-trialkit | pytest plugin |
| trialkit-report-mlflow | Python MLflow reporter |
Not built: @trialkit/judge/anthropic (the AI SDK client reaches Anthropic); @trialkit/report-allure and @trialkit/report-datadog (JUnit and OpenTelemetry cover CI reports and Datadog traces); one trialkit-model-* or trialkit-report-* distribution for each other provider or backend.
Dependency policy
Section titled “Dependency policy”Packages declare integrations without bringing a second copy of an app’s technology.
- TypeScript packages have no runtime, optional or bundled dependencies: SDKs, test frameworks and other integrations are peers. Python has no peers: a requirement or an extra names a trialkit distribution or its integration, with bounds. Why: peer-only integrations avoid duplicate packages.
- A peer range lists every tested major, with no exact pin and no open range such as
>=4. It starts at the lowest version in its major with the APIs we use and no known security advisory. - An adapter imports no other adapter and re-exports no peer, except a core type that its API returns (scorers and judge export
Scorer). Why: pnpm links core only into the package that lists it, so an app emits declarations for a returned core type only through that package. - The one import of an adapter by another is
@trialkit/judge/yaml, which reads a template file or a prompt file throughreadYamlMappingof@trialkit/cases, an optional peer of judge (Python: the extratrialkit-judge[yaml]). An app that reads neither loads neither. Why: the YAML rules stay in one reader that the case files, the template files and the prompt files share, and core cannot hold the parser, because core has no peers. - Packages ship no native modules, binaries, install scripts, HTTP server, bundler or TypeScript loader. The one exception is
@trialkit/studio, an opt-in development tool that no other package imports.
Package format and resolution
Section titled “Package format and resolution”ESM is built for every package and CommonJS where tsconfig.cjs.json exists; @trialkit/vitest and the CLI are ESM only. Why: Jest has only experimental ESM support, so a CommonJS Jest project needs CommonJS dependencies. scripts/install-test.mjs checks each TypeScript resolution mode. Both builds can load in one process, so classification uses value signals and run state belongs to a run.
Runtimes and state
Section titled “Runtimes and state”Runtime-neutral packages (core, scorers, judge, replay, report-mlflow, report-otel) use only ECMAScript and shared web APIs (Node.js, Bun, Deno, browsers, edge workers); Node.js code lives in src/node/ subpaths. No package reads an environment variable, patches a global or registers a tracer provider: the app passes settings and supplies its own provider.
The Node.js floor (requirements) is the same in every package, in the root manifest and on the lowest CI leg. It does not come from a library need: the docs site’s Astro 7 requires it, and CI runs the whole workspace on one lowest Node.js. Vitest 5, the strictest runner peer, accepts 22.12, and a lower library floor is untested (§14). The package verifier checks that engines.node exists, not its value.
Versions and metadata
Section titled “Versions and metadata”The npm packages and Python distributions release together under one version, from 0.1.0. The core peer range accepts 0.1.x only, so a new core minor needs a release of every package. verify-packages checks the metadata.
CI and package verification
Section titled “CI and package verification”CONTRIBUTING.md lists the steps of the gate, and the workflow holds versions and limits. Tools have exact versions, so a release cannot change a run that nobody touched, and CI fails, not skips, when a tool is missing. Packed consumers install the adapters with strict peer checks, because --legacy-peer-deps proves declarations only. No test, gate script or unattended job calls a paid API.
Shared contracts and parity
Section titled “Shared contracts and parity”Both languages implement the contracts in spec/ instead of sharing code: change a contract in spec/ first, then both implementations in the same commit. The parity tool sends generated inputs to the built TypeScript core and a Python oracle, and a difference fails with the seed and the smallest case. Why: a behavior that only one core has is a bug, and fixtures cannot list every case.