Optionaloptions: {Drops all fetch state for a thread. Called from the run() finally,
alongside judgeSpanCollector.clearSpansForThread, to prevent memory
growth in long-lived processes.
The judge's one extra wait: re-arms every trace that terminally failed
the first settle-wait (retracting its synthetic error span from the
collector) and settle-waits once more under timeoutMs. A trace that
fails again gets a fresh error span with the new reason. Returns whether
every given trace id is settled cleanly afterwards.
True when not one of the given trace ids ever settled cleanly for this thread. After a settle-wait this means every trace terminally failed, so the run's remote evidence cannot improve with more turns.
Records the "nothing to fetch" case: remote fetching is on, but not one message of the conversation carries a trace id, so there is no id to poll. Feeds the same synthetic error span the deadline path feeds, once per thread, so the judge reads why the traces section is empty instead of returning inconclusive criteria with no stated reason.
Verdict-time wait: polls every unsettled trace id until it settles (see
the class doc for the settle conditions), all ids in parallel, under one
shared deadline of timeoutMs total.
A trace whose spans are complete still waits out quietPeriodMs of an
unchanged span set, so this wait costs at least one quiet period when
there is anything to fetch. At the deadline a trace still inside its
quiet period settles cleanly, with no error span.
A failed poll retries until the deadline; only the deadline marks the id
failed and feeds one synthetic langwatch.span_collection.error span
carrying the reason to the collector. Never throws. Returns whether every
given trace id is settled cleanly after the wait.
Fetches remote traces from the LangWatch trace API and feeds them into the judge span collector, so the trace digest and the expand_trace/grep_trace tools work on remote spans exactly as they do on local ones.
Latency contract: conversation turns never fetch. The verdict settle-waits (settleWait): each unsettled id is polled every second, all ids in parallel under one shared deadline.
A trace settles cleanly when it holds at least one remote span (a fetched span that is not one of the scenario's own locally collected spans) AND every fetched agent span's parent resolves within the fetched and locally collected spans — the trace is complete, because ancestors always finish and export after their descendants — AND that span set has stayed unchanged for the quiet period. Count-stability is deliberately NOT a settle signal on its own: ingestion arrives in chunks that can be tens of seconds apart, and a stable early chunk would satisfy it while tool spans are still on the way. The quiet period only extends the parent-resolved condition, which a resolved parent chain already backs; it covers the one case that condition cannot see, a leaf tool span still in flight, whose absence leaves no unresolved parent behind. It costs one quiet period per verdict, DEFAULT_TRACE_QUIET_PERIOD_MS by default.
When the deadline expires with remote spans present but parents still unresolved, the trace settles best-effort: every span that arrived stays in the collector, plus one synthetic
langwatch.span_collection.errorspan marking the trace incomplete — so the judge can still pass criteria proven by the visible spans while treating the rest as inconclusive. When the deadline expires with no remote span at all (propagation broken, agent unreachable, agent not instrumented), the synthetic error span reports that nothing was collected. Fetch failures never propagate out of the judge.State is kept per thread id (merged span ids, settled flag) so a trace confirmed complete is never re-fetched;
run()clears it in the same finally that clears the judge span collector.