Optional_Live OTel context of the CURRENT voice.turn, published by
defaultVoiceCall for background-receive-loop adapters
(Pipecat/Twilio) to parent their detached-callback recv spans under the
turn (#774 — the reusable pattern Twilio PR5 inherits). undefined between
turns, so a callback firing outside a turn skips its span rather than
parenting under a closed turn. Internal (underscore) — not a public API.
SET when the AGENT deliberately ended the call (e.g. an ElevenLabs hosted
agent invoking the end_call system tool), as opposed to the transport
dropping. A scripted turn arriving after this concludes the conversation
instead of failing the run — the agent behaved as designed. Assertions and
judges can read it to reason about WHO ended the call.
OptionalagentSet when the adapter has emitted its first agent audio chunk for the
current turn — gates timing-based barge-in. Concrete adapters expose
this so scenario.interrupt can wait for real speech before
firing the interruption. Optional: adapters without server-VAD-style
interrupt sequencing can leave it undefined.
ReadonlycapabilitiesDeclaration of what this adapter can and cannot do. Concrete subclasses MUST publish a non-default value; the base instance defaults to "nothing supported" so capability-gated steps fail safely when an adapter forgets to declare.
Most-recent output transcript received from the server, for observability.
ReadonlymodelOptionalnameHard cap on a single agent turn's audio. Prevents runaway loops if a transport never signals end-of-stream. 30s = a long sentence.
Tail silence: once the first agent chunk arrives, keep draining receiveAudio until no chunk shows up within this many seconds — that's how we detect the agent finished talking.
Seconds to wait for agent audio after sending user audio: the STT + LLM +
TTS budget for one agent turn. Kept identical to Python's
VoiceAgentAdapter.response_timeout so the same scenario passes or fails
the same way in both SDKs.
Raise it for an agent that runs a tool call or a retrieval step before it speaks:
const agent = elevenLabsAgent({ agentId, apiKey });
agent.responseTimeout = 180; // wait up to 3 minutes
OptionalstreamingIncremental transcript text emitted while the agent speaks. Populated
by adapters that advertise capabilities.streamingTranscripts. Read
by scenario.interrupt when afterWords: N is set.
ReadonlysystemBounded grace-wait (seconds) for the agent turn's transcript AFTER audio
drains (#734). Audio silence closes the turn (responseTailSilence), but a
live voice agent (hosted ElevenLabs) delivers the turn's text on a SEPARATE
socket event (agent_response → lastAgentTranscript). When that event
lands after the audio-silence boundary, snapshotting lastAgentTranscript
at drain-close reads null and the turn reaches the text-only simulator as
a bare [audio message] — the simulator then fabricates.
The default call() flow (defaultVoiceCall) polls this field up to
this ceiling for a pending transcript before reading it. It short-circuits
the INSTANT lastAgentTranscript is already set (zero added latency on the
happy path — the common case where the transcript won the race) and only
elapses when the transcript genuinely never arrives, so a real ElevenLabs
drop still terminates the turn. Set to 0 to disable the wait.
ReadonlyvoiceDefault call() body, ported from Python VoiceAgentAdapter.call.
Threads the latest user-message audio through sendAudio, drains the agent response on tail silence, records one user and one agent segment into the executor state, and returns the merged assistant audio message. Subclasses may override for specialised flows but will usually inherit it.
Open a Gemini Live session.
Lazy-imports @google/genai so the SDK only loads when this adapter
is actually used. Registers an onmessage callback that pushes
LiveServerMessage instances onto an internal queue, which
receiveAudio drains.
Close the Gemini Live session and release the WebSocket.
Signal an in-flight receiveAudio() to return the cut-off sentinel
immediately, so the interrupted turn doesn't replay stale agent audio
into the next turn's drain loop.
Abort-sentinel pattern (fixes the single-consumer concurrency race):
The original implementation called dequeue() concurrently with an
in-flight receiveAudio(). Since dequeue() has a single resolveNext
slot, the second caller (interrupt) overwrote the first caller's
(receiveAudio's) resolver. When a message arrived it resolved the
interrupt's dequeue, leaving receiveAudio's resolver orphaned — its
timer eventually fired with a TimeoutError, causing drainAgentResponse
to catch and break prematurely.
Fix: interrupt() no longer calls dequeue(). Instead it:
_interruptPending = true.dequeue() by calling the current resolveNext
directly with an abort sentinel ({ interrupted: true }).receiveAudio()'s loop checks _interruptPending (and item.interrupted)
and returns the cut-off sentinel immediately on seeing it.Best-effort: if nothing is in-flight (resolveNext is null), the flag
stays set and receiveAudio() catches it at the top of its next iteration.
Whether the Gemini Live session is open (Gap #11).
Send a canonical 24kHz AudioChunk to Gemini Live as a complete turn.
Resamples 24kHz → 16kHz at the wire boundary and wraps the audio in
explicit activityStart / activityEnd markers. With Automatic Activity
Detection disabled (see connect), each sendAudio call is a
complete user turn from Gemini's perspective: the model replies the moment
we close the turn.
Transmit DTMF tones to the telephony peer. Adapters that advertise
capabilities.dtmf MUST implement this; the default raises
UnsupportedCapabilityError so an adapter that forgot to ship
sendDtmf while claiming the capability fails loudly instead of
silently routing through a PCM fallback.
Gemini Live native-audio adapter.
Connects directly to the Gemini Live API via the
@google/genaiSDK. Audio flows bidirectionally as raw PCM16; canonical 24kHz internally, resampled to/from 16kHz at the wire boundary.Remarks
The
@google/genaipackage is declared as an optional peer dependency so the SDK ships without a hard Gemini coupling. Users who import this adapter must install@google/genaithemselves.