The criteria the judge will use to evaluate the conversation.
LLM invocation function. Can be overridden to customize LLM behavior.
Static ReadonlyAUDIO_Model substrings that indicate multimodal (audio-capable) support.
Mirrors python/scenario/judge_agent.py:_AUDIO_CAPABLE_MODEL_SUBSTRINGS.
Process the input and generate a response.
This is the main method that your agent implementation must provide. It receives structured information about the current conversation state and must return a response in one of the supported formats.
AgentInput containing conversation history, thread context, and scenario state.
The agent's response.
Resolves include_audio for this evaluation:
true/false wins.null (default): true only when conversation has audio AND the judge
model is known to be multimodal.Port of python/scenario/judge_agent.py:effective_include_audio.
Resolves include_timeline for this evaluation.
Defaults to true for voice conversations (auto-detect = conversation has audio).
Port of python/scenario/judge_agent.py:effective_include_timeline.
Resolves include_traces for this evaluation.
Defaults to true when OTel / LangWatch is configured.
Port of python/scenario/judge_agent.py:effective_include_traces.
Whether the configured judge model can ingest raw audio. Determined by checking model name substrings.
StaticconversationWhether any message in messages contains an audio content part.
Recognizes the canonical AI-SDK file audio part
({ type: "file", mediaType: "audio/*" }, EDR §4.2 — the single
in-message format the voice subsystem now produces) and, for
adapter-edge tolerance, the legacy OpenAI input_audio / audio
conventions.
Port of python/scenario/judge_agent.py:_conversation_has_audio.
Agent that evaluates conversations against success criteria.
This is the default judge agent that is used if no judge agent is provided. It is a simple agent that uses function calling to make structured decisions and provides detailed reasoning for its verdicts.
Param: cfg
{JudgeAgentConfig} Configuration for the judge agent.