9. Model port
ModelClient is the small port between judges and a model provider: an id (<provider>:<model>, recorded in every judge score), generate(request, signal?), and fromFunction for any other client or a test fake (reference). The port carries text only: a message content is a string, so a judge cannot inspect pixels or audio, and media support would need a media-aware port first. Only judges and model-based scorers use the port. The task calls the app’s own client, so the eval tests the app’s real call path.
openaiModelandaiSdkModelare subpaths of@trialkit/judge, not packages. Each takes the app’s SDK as an optional peer and adds no retries and no streaming, because the SDK owns transport retries. A judge wraps an SDK error, so the trial ends inerror.- Databricks Unity Gateway needs no package. It is OpenAI-compatible, so
openaiModelreaches it with anopenaiclient whosebaseURLis the gateway (setup). The app gives the token to the client, and the adapter reads noDATABRICKS_TOKEN. CI runs this path against a fakefetch; a person checks a live workspace before a release. - Python’s OpenAI-compatible client lives in
trialkit-judgeand uses the standard library, so it covers the gateway without a separate distribution. - Not built: the OpenAI Responses API in
@trialkit/judge/openai, and@trialkit/judge/anthropic.