@langwatch/scenario
    Preparing search index...

    Class JudgeAgent

    Agent that evaluates conversations against success criteria.

    This is the default judge agent that is used if no judge agent is provided. It is a simple agent that uses function calling to make structured decisions and provides detailed reasoning for its verdicts.

    {JudgeAgentConfig} Configuration for the judge agent.

    Hierarchy (View Summary)

    Index
    constructor
    criteria invokeLLM name role AUDIO_CAPABLE_MODEL_SUBSTRINGS
    call effectiveIncludeAudio effectiveIncludeTimeline effectiveIncludeTraces modelSupportsAudio conversationHasAudio
    criteria: string[]

    The criteria the judge will use to evaluate the conversation.

    invokeLLM: (
        params: CallSettings & (
            Prompt & { model: LanguageModel; tools?: ToolSet | undefined; toolChoice?: ToolChoice<NoInfer<ToolSet>> | undefined; ... 19 more ...; _internal?: { ...; } | undefined; }
        ),
    ) => Promise<InvokeLLMResult> = ...

    LLM invocation function. Can be overridden to customize LLM behavior.

    name: string = "JudgeAgent"

    The name that LangWatch shows as the target of the run. When you do not set it, the framework uses the class name of the adapter.

    role: AgentRole = AgentRole.JUDGE
    AUDIO_CAPABLE_MODEL_SUBSTRINGS: readonly string[] = ...

    Model substrings that indicate multimodal (audio-capable) support. Mirrors python/scenario/judge_agent.py:_AUDIO_CAPABLE_MODEL_SUBSTRINGS.

    • Resolves include_audio for this evaluation:

      • Explicit true/false wins.
      • null (default): true only when conversation has audio AND the judge model is known to be multimodal.

      Port of python/scenario/judge_agent.py:effective_include_audio.

      Parameters

      • conversationHasAudio: boolean

      Returns boolean

    • Resolves include_timeline for this evaluation. Defaults to true for voice conversations (auto-detect = conversation has audio).

      Port of python/scenario/judge_agent.py:effective_include_timeline.

      Parameters

      • conversationHasAudio: boolean

      Returns boolean

    • Resolves include_traces for this evaluation. Defaults to true when OTel / LangWatch is configured.

      Port of python/scenario/judge_agent.py:effective_include_traces.

      Parameters

      • otelConfigured: boolean

      Returns boolean

    • Whether any message in messages contains an audio content part.

      Recognizes the canonical AI-SDK file audio part ({ type: "file", mediaType: "audio/*" }, EDR §4.2 — the single in-message format the voice subsystem now produces) and, for adapter-edge tolerance, the legacy OpenAI input_audio / audio conventions.

      Port of python/scenario/judge_agent.py:_conversation_has_audio.

      Parameters

      • messages: readonly unknown[]

      Returns boolean

    Settings
    Member Visibility

    On This Page
    Constructors
    Properties
    Methods