Get Started

Detectors evaluate your traces and surface problems the moment they happen. This guide walks through creating your first detector, watching the LLM-as-a-judge flag a problem, and reading the AI agent’s root cause analysis.

​

Prerequisites

You need a project with at least one incoming trace. See Tracing if you don’t have that yet.

​

Create a detector

Open your project in TraceRoot Cloud, select Detectors from the left nav, then click + New Detector.

​

Pick a starting point

The create sheet offers six templates: Failure, Hallucination, Logic Error, Task Completion, Safety, and Blank. Pick whichever maps to a problem you’ve seen in production. The template fills in a prompt you can edit; save, and the detector starts considering new incoming traces.

​

Choose a sample rate

New detectors evaluate 25% of the traces that match their trigger conditions. Sampling is deterministic for each trace and detector, so sending the same trace again does not change the decision. Set the rate to 100% to evaluate every matching trace, or to 0% to pause the detector.

​

Choose how root cause analysis runs

The create sheet — and the edit panel for an existing detector — has a Root cause analysis setting in its Signals section. Automatic, the default, runs the AI agent on each new signal (a recurring problem the detector’s findings are grouped into), and on a signal reopened more than 24 hours after its last analysis. Manual runs it only when you click Run root cause analysis on a signal, so you pay agent-model cost only for the problems you look into. Detection and notifications are the same either way. RCA runs per signal, not per finding. See Root cause analysis for the exact rule.

​

How it works

The pipeline for each incoming trace:

1. Ingest. A trace lands in the project.
2. Match. TraceRoot checks each enabled detector’s trigger conditions.
3. Sample. Each matching detector applies its sample rate; new detectors default to 25%.
4. Evaluate. The LLM-as-a-judge (Claude Haiku by default) reads the sampled trace and returns identified plus a short summary.
5. Diagnose. When identified is true, the trace becomes a finding and joins a signal. When you run the analysis from the signal, or automatically when the finding starts a new signal (or reopens a resolved one) and its detector’s root cause analysis is Automatic, the TraceRoot AI agent investigates: it downloads the trace, reads your production source code at the failure point, checks recent commits and PRs touching that file, and produces a root cause analysis.
6. Notify. At the end of the configured window, TraceRoot sends one digest to each configured email and Slack channel announcing new and reopened signals. Further findings on an open signal send nothing. Each signal shows its title, a short summary, its detector and hit count, and a link; root cause analysis results stay on the signal page, so the digest never waits for them.

​

View findings

Open the detector to see its runs; from the detector list, its Findings count opens just the runs that identified a problem. Every run links back to the trace it checked, and a run that reached a verdict shows whether the detector identified a problem. Clicking the trace opens the trace viewer, with the AI agent’s RCA session in the chat panel when that trace was analysed. A finding also links to the signal it joined (Signal ID); one that grouping gave up on, after the model’s answers about it stayed unusable, says Not grouped. When the signal’s root cause analysis ran on that finding’s own trace, the row also links to the agent’s trace of it (Agent Run ID); a finding that only joined an already analysed signal shows a dash, and its signal holds the analysis. From the chat panel you can ask follow-up questions in the same session — you’re talking to the same agent that wrote the analysis. The trace viewer’s Detectors tab lists every detector that ran on the trace, with a link to the signal each finding joined.

​

Bring Your Own Key (BYOK)

Both stages of the pipeline — the LLM-as-a-judge and the AI agent — run on a model you choose. Either can use a TraceRoot-provided system model or your own API key, configured independently per stage.

​

Next steps