Scorers: deterministic first
Use a deterministic scorer when code can decide the rule.
| Output | Scorer |
|---|---|
| Class, id or field | exact |
| Number with tolerance | numeric |
| Date, id or code | regex or exact |
| Required phrase or citation | includes |
| List | setMatch |
| Agent calls | toolCalls or toolPolicy |
| JSON shape | schema with the app’s Standard Schema validator |
import type { Scorer } from '@trialkit/core';
const maxWords: Scorer<string, { summary: string }> = { name: 'summary-words', version: '1', score: ({ output }) => { const count = output.summary.trim().split(/\s+/).length; return { value: count <= 12 ? 1 : 0, pass: count <= 12, reason: `${count} words` }; },};- A scorer has a unique
name,versionandscore. Change the version when its logic changes, so comparison does not mix measurements. - In a test form, use
await score(scorer, output, expected?). The optionaltoPassmatcher needsawaittoo: never discard its promise.
The scorers reference lists every scorer, option and default. For a whole-string pattern, an exact match without surrounding whitespace, or a JSON check, see the recipes for TypeScript and Python.