Skip to main content
An evaluation runs your system against a dataset of cases and scores each result. In TraceRoot every case runs as its own trace, so you inspect an eval with the same span timeline, token, and cost views you already use for tracing. Each scorer emits a metric, and you read a run per metric — there is no single headline score. The SDK is available in Python and TypeScript, with the same API in both.

​
What you write

  • A dataset — the cases to run, each with an input and an optional expected answer.
  • A task — your system under evaluation, mapping an input to an output.
  • One or more scorers — functions that turn a case’s result into a metric.
Then evaluate() runs the task over every case, scores each result, and reports the run.

​
Where to start

Get Started

Author a dataset, write a scorer, and run your first eval.

Datasets

Author, version, and publish the cases you run against.

Scorers

Code scorers and LLM-as-a-judge.

Running Evals

evaluate() — the task, the scorers, and where a run reports.

Reading Results

Per-metric summaries and per-case results.