`ElevenLabsAgentAdapter`

Connects to an ElevenLabs Conversational AI ("Convai") agent over WebSocket. Use this when you've built an agent in the ElevenLabs dashboard and want to test it end-to-end.

Constructor

python

import os
import scenario
 
adapter = scenario.ElevenLabsAgentAdapter(
    agent_id=os.environ["ELEVENLABS_AGENT_ID"],
    api_key=os.environ["ELEVENLABS_API_KEY"],
    # Personalise this call without touching the deployed agent:
    dynamic_variables={"tenant_id": "acme", "seat_tier": 2, "is_vip": True},
    overrides={"agent": {"language": "es"}},
)

All per-session options are transmitted in the WebSocket conversation_initiation_client_data handshake at connect time. ElevenLabs applies each one only if the agent allowlists it server-side (Agent → Security → Overrides in the dashboard); anything not opted in is ignored.

Python	TypeScript	Description
`dynamic_variables`	`dynamicVariables`	Per-call values that fill the deployed prompt's `{{template}}` placeholders. Text, numeric, and boolean values pass through with their JSON type intact. This is how production integrations personalise an agent per call. Omitted entirely when unset.
`overrides`	`overrides`	Any `conversation_config_override` the narrow knobs below do not cover, e.g. `{"agent": {"language": "es"}}` or `{"tts": {"stability": 0.3}}`. Deep-merged, so your `agent.language` and the adapter's `agent.prompt` both survive.
`system_prompt_override`	`systemPromptOverride`	Replaces the agent's system prompt for this session only. See the warning below before using this on a hosted agent.
`first_message_override`	`firstMessageOverride`	Overrides the agent's opening message for this session.

Turn commit modes

How the adapter tells ElevenLabs a user turn is over. The default streams the user's real voice, which is what makes the run evidence about your agent's own STT and turn-taking.

Mode	Behaviour
`"audio"` (default)	Streams the turn's real PCM as 20 ms frames at microphone cadence, then unbounded closing silence. EL's server VAD closes the turn on that audio→silence transition, and its STT transcribes what you sent.
`"text"`	Sends a `user_message` text commit and no audio. Deterministic, but your agent's STT and VAD never run, so a green run says nothing about either. Opt in only when scripted audio cannot drive your agent's turn-taking.
`"silence"`	Streams the PCM followed by a fixed silence tail. Superseded by `"audio"`; a bounded tail does not reliably close a scripted turn.

adapter = scenario.ElevenLabsAgentAdapter(
    agent_id=..., api_key=..., turn_commit_mode="text",  # opt out of real voice-in
)

Turn boundaries

ElevenLabs delivers agent audio in bursts. A delivery gap longer than response_tail_silence (0.6s by default) closes the turn while the agent is still speaking, which would otherwise leave the rest of the utterance to surface as the next turn's opening audio — making reply N+1 look like an answer to question N.

At each user-turn boundary the adapter sweeps up any agent audio still in flight and attributes it to the turn that produced it. When that happens you'll see:

ElevenLabsAgentAdapter: recovered 41280 bytes of agent audio stranded by an
early turn close and attributed them to the preceding agent turn. Raise
response_tail_silence if this recurs.

That warning is informational, not a failure. If it fires on most turns your agent's delivery is gappier than the default tolerates, so raise response_tail_silence:

adapter = scenario.ElevenLabsAgentAdapter(agent_id=..., api_key=...)
adapter.response_tail_silence = 1.2

Agent-initiated hangup

Hosted agents commonly end a call themselves by invoking the end_call system tool (or a transfer_to_* tool) right after their farewell, which closes the WebSocket. When that happens the adapter records it and any remaining scripted turn concludes the conversation and falls through to the judge, rather than failing a run in which the agent behaved exactly as designed.

Read adapter.agent_hung_up (Python) / adapter.agentHungUp (TypeScript) to assert on who ended the call.

Capabilities

streaming_transcripts	native_vad	dtmf	interruption	input_formats	output_formats
✅	✅	❌	❌	`pcm16/24000`	`pcm16/24000`

Worked examples

Python: elevenlabs_hosted.py — hosted Convai scenario with a judge step; elevenlabs_branded.py — branded voice variant; elevenlabs_interruption.py — barge-in handling.
TypeScript: elevenlabs-hosted.test.ts — hosted ConvAI path and the branded ElevenLabsVoiceAgent composable variant.

Common failures

HTTP 401 quota_exceeded — your character quota is exhausted. See Troubleshooting → ElevenLabs HTTP 401 quota_exceeded.

Composable variant — `ElevenLabsVoiceAgent`

If you'd rather pair ElevenLabs STT/TTS with a different LLM, use ElevenLabsVoiceAgent: