@langwatch/scenario
    Preparing search index...

    Class ElevenLabsVoiceAgent

    Composable voice agent with ElevenLabs-opinionated defaults.

    Not to be confused with ElevenLabsAgentAdapter (above) which talks to ElevenLabs' hosted ConvAI endpoint. This class is local: you compose ElevenLabsSTTProvider + any LLM + ElevenLabs TTS yourself.

    Default stack:

    • STT: ElevenLabsSTTProvider with the same API key.
    • LLM: openai("gpt-5.4-mini") — text-only chat completion.
    • TTS: elevenlabs/EXAVITQu4vr4xnSDxMaL (Sarah — free-tier premade). Override via the ELEVENLABS_VOICE_ID env var or the voice arg.
    // Defaults — all ElevenLabs STT, gpt-5.4-mini, EL TTS
    const apiKey = process.env.ELEVENLABS_API_KEY;
    if (!apiKey) throw new Error("ELEVENLABS_API_KEY is required");
    const agent = new ElevenLabsVoiceAgent({ apiKey });

    // Override just the LLM
    import { anthropic } from "@ai-sdk/anthropic";
    const agent = new ElevenLabsVoiceAgent({ apiKey, llm: anthropic("claude-sonnet-4-6") });

    // Bring your own STT
    const agent = new ElevenLabsVoiceAgent({ apiKey, stt: new MyCustomSTT() });

    Hierarchy (View Summary)

    Index
    _voiceTurnContext?: Context

    Live OTel context of the CURRENT voice.turn, published by defaultVoiceCall for background-receive-loop adapters (Pipecat/Twilio) to parent their detached-callback recv spans under the turn (#774 — the reusable pattern Twilio PR5 inherits). undefined between turns, so a callback firing outside a turn skips its span rather than parenting under a closed turn. Internal (underscore) — not a public API.

    agentHungUp: boolean = false

    SET when the AGENT deliberately ended the call (e.g. an ElevenLabs hosted agent invoking the end_call system tool), as opposed to the transport dropping. A scripted turn arriving after this concludes the conversation instead of failing the run — the agent behaved as designed. Assertions and judges can read it to reason about WHO ended the call.

    agentSpeakingEvent?: AgentSpeakingEvent

    Set when the adapter has emitted its first agent audio chunk for the current turn — gates timing-based barge-in. Concrete adapters expose this so scenario.interrupt can wait for real speech before firing the interruption. Optional: adapters without server-VAD-style interrupt sequencing can leave it undefined.

    capabilities: AdapterCapabilities = ...

    Declaration of what this adapter can and cannot do. Concrete subclasses MUST publish a non-default value; the base instance defaults to "nothing supported" so capability-gated steps fail safely when an adapter forgets to declare.

    history: ModelMessage[]
    lastLlmResponse: string | null = null
    lastUserTranscript: string | null = null
    llm: LanguageModel
    name?: string
    responseMaxDuration: number = 30.0

    Hard cap on a single agent turn's audio. Prevents runaway loops if a transport never signals end-of-stream. 30s = a long sentence.

    responseTailSilence: number = 0.6

    Tail silence: once the first agent chunk arrives, keep draining receiveAudio until no chunk shows up within this many seconds — that's how we detect the agent finished talking.

    responseTimeout: number = 60.0

    Seconds to wait for agent audio after sending user audio: the STT + LLM + TTS budget for one agent turn. Kept identical to Python's VoiceAgentAdapter.response_timeout so the same scenario passes or fails the same way in both SDKs.

    Raise it for an agent that runs a tool call or a retrieval step before it speaks:

    const agent = elevenLabsAgent({ agentId, apiKey });
    agent.responseTimeout = 180; // wait up to 3 minutes
    role: AgentRole = AgentRole.AGENT
    streamingTranscript?: string

    Incremental transcript text emitted while the agent speaks. Populated by adapters that advertise capabilities.streamingTranscripts. Read by scenario.interrupt when afterWords: N is set.

    transcriptGraceWait: number = 2.0

    Bounded grace-wait (seconds) for the agent turn's transcript AFTER audio drains (#734). Audio silence closes the turn (responseTailSilence), but a live voice agent (hosted ElevenLabs) delivers the turn's text on a SEPARATE socket event (agent_responselastAgentTranscript). When that event lands after the audio-silence boundary, snapshotting lastAgentTranscript at drain-close reads null and the turn reaches the text-only simulator as a bare [audio message] — the simulator then fabricates.

    The default call() flow (defaultVoiceCall) polls this field up to this ceiling for a pending transcript before reading it. It short-circuits the INSTANT lastAgentTranscript is already set (zero added latency on the happy path — the common case where the transcript won the race) and only elapses when the transcript genuinely never arrives, so a real ElevenLabs drop still terminates the turn. Set to 0 to disable the wait.

    tts: string
    ttsOptions: SynthesizeOptions
    turnOutputEmitted: boolean = false

    Turn-output guard. The default call() drains receiveAudio until tail-silence; on this adapter that would kick a second LLM call. Reset by sendAudio (new user turn → new LLM call allowed), set by the end of receiveAudio.

    voice: string
    DEFAULT_SYSTEM_PROMPT: string = ...
    • Send a first-class interrupt signal to the agent under test.

      Adapters that advertise capabilities.interruption === true override this to send the transport-native interrupt (e.g. Twilio clear, OpenAI Realtime response.cancel). The default raises UnsupportedCapabilityError; callers (scenario.interrupt()) check capabilities.interruption and fall back to timing-based barge-in when this returns false.

      Returns Promise<void>

    • Whether the transport is currently open and ready to exchange audio (Gap #11). The default call flow (defaultVoiceCall) consults this BEFORE sending audio and raises PendingTransportError uniformly when it returns false — so a call() issued before the executor's connect() fails with one clear error across every transport instead of a transport-specific null-dereference or silent hang.

      Base default is true: adapters with no meaningful "not connected" state (in-process composable, test doubles) never trip the gate. Network transport leaves override this to report their real socket/session state.

      Returns boolean