Optionaloptions: {Drops all fetch state for a thread. Called from the run() finally,
alongside judgeSpanCollector.clearSpansForThread, to prevent memory
growth in long-lived processes.
The judge's one extra wait: re-arms every trace that terminally failed
the first settle-wait (retracting its synthetic error span from the
collector) and settle-waits once more under timeoutMs. A trace that
fails again gets a fresh error span with the new reason. Returns whether
every given trace id is settled cleanly afterwards.
True when not one of the given trace ids ever settled cleanly for this thread. After a settle-wait this means every trace terminally failed, so the run's remote evidence cannot improve with more turns.
Records the "nothing to fetch" case: remote fetching is on, but not one message of the conversation carries a trace id, so there is no id to poll. Feeds the same synthetic error span the deadline path feeds, once per thread, so the judge reads why the traces section is empty instead of returning inconclusive criteria with no stated reason.
Verdict-time wait: polls every unsettled trace id until it settles (see
the class doc for the settle conditions), all ids in parallel, under one
shared deadline of timeoutMs total.
A failed poll retries until the deadline; only the deadline marks the id
failed and feeds one synthetic langwatch.span_collection.error span
carrying the reason to the collector. Never throws. Returns whether every
given trace id is settled cleanly after the wait.
Fetches remote traces from the LangWatch trace API and feeds them into the judge span collector, so the trace digest and the expand_trace/grep_trace tools work on remote spans exactly as they do on local ones.
Latency contract: conversation turns never fetch. The verdict settle-waits (settleWait): each unsettled id is polled every second, all ids in parallel under one shared deadline.
A trace settles cleanly when it holds at least one remote span (a fetched span that is not one of the scenario's own locally collected spans) AND every fetched agent span's parent resolves within the fetched and locally collected spans — the trace is complete, because ancestors always finish and export after their descendants. Count-stability is deliberately NOT a settle signal: ingestion arrives in chunks that can be tens of seconds apart, and a stable early chunk would satisfy it while tool spans are still on the way.
When the deadline expires with remote spans present but parents still unresolved, the trace settles best-effort: every span that arrived stays in the collector, plus one synthetic
langwatch.span_collection.errorspan marking the trace incomplete — so the judge can still pass criteria proven by the visible spans while treating the rest as inconclusive. When the deadline expires with no remote span at all (propagation broken, agent unreachable, agent not instrumented), the synthetic error span reports that nothing was collected. Fetch failures never propagate out of the judge.State is kept per thread id (merged span ids, settled flag) so a trace confirmed complete is never re-fetched;
run()clears it in the same finally that clears the judge span collector.