NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI, #3638 most downloaded on PyPI
An open source framework for voice (and multimodal) assistants
Latest stable
1.16.0
Last release 6 days ago
12 Nov 2026
Ships on a steady schedule
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
2 versions withdrawn
withdrawn after publishing
3 years old
119 releases · first in 2024
Drag across the ruler to compare, or tap a release to open it.
LiveAvatarNewSessionRequest now defaults to H264 video encoding, matching the LiveAvatar API's own default. LiveAvatar has deprecated VP8; pass video_…
Added LatencyBreakdown.contributions, a timeline of the user-to-bot interval whose durations sum to the measured latency. It names the time no service reports — VAD silence, turn detection, turn-completion markers and holds, sentence aggregation, and function handlers — and attributes time a setting governs to that setting rather than to the service it runs inside.
The first thing the bot says is measured too, from where StartupTimingObserver stops — the pipeline ready with a client on it — so the two reports meet rather than overlap. Waiting to be asked is usually most of that wait, and no service reports it.
Each part carries a stable key and an owner_kind — service, setting, bot or pipeline — so data keyed on them survives a label being reworded, and the breakdown says what it was measured_from alongside the total_secs it measured. Print it with LatencyBreakdown.turn_contribution_lines(); UserBotLatencyObserver(min_contribution_secs=...) sets how brief a part must be before it rolls into a single pipeline entry.
```
0.200s endpointing wait [config: VAD stop_secs]
0.125s transcription [DeepgramSTTService#0]
0.336s LLM inference [OpenAILLMService#0]
0.024s turn completion [config: filter_incomplete_user_turns]
0.359s speech synthesis [CartesiaTTSService#0]
1.044s TOTAL
```
(PR #5445)
Eval scenario turns can play an audio file as the user instead of synthesizing text. A turn's audio: names a recording, resolved relative to the scenario file, in any format soundfile reads (WAV, MP3, FLAC, OGG, ...); multi-channel audio is downmixed to mono and the file keeps its own sample rate. user: is required alongside it and is what the recording says, so the judge and text_contains still have the turn's input. A scenario whose audio turns all name a file needs no user.speech: block.
(PR #5473)
Added a cancellable_by_llm argument to LLMSwitcher.register_function(), which forwards it to every LLM the switcher fronts. Tools registered through a switcher can now opt into LLM-side cancellation, matching what LLMService.register_function() accepts.
(PR #5474)
Added AssemblyAISyncSTTService, a segmented speech-to-text service backed by AssemblyAI's Sync API: pipeline VAD segments the audio and each segment (up to 120 seconds) is transcribed in one HTTP request, with no streaming session to manage. It takes an aiohttp_session, base_url selects a data-residency endpoint, and Settings supports model, language, prompt, keyterms_prompt, and conversation_context.
Recent user and agent turns are sent automatically as conversation_context on each request, bounded by max_context_turns (0 disables) and max_context_chars; setting conversation_context yourself replaces that buffer. The connection is pre-warmed when the user starts speaking so the transcription request skips the handshake (enable_prewarming).
(PR #5488)
AzureSTTService.Settings gained segmentation_silence_timeout_ms, which sets how much silence (100–5000 ms) Azure allows inside a phrase before it emits a final transcript. Azure's default of 500 ms applies when it's unset.
(PR #5511)
Added on_progress and on_update event handlers to the eval framework. EvalSession and EvalSuite are now BaseObjects, so eval progress is observed the same way as every other Pipecat event. EvalSuite's on_update handlers run synchronously and should return promptly, since an EvalRun is mutated in place as it executes; EvalSession's on_progress handlers run as tasks, and EvalSession.run() waits for them before it returns.
@session.event_handler("on_progress")
async def on_progress(session, progress):
print(progress.event_name, progress.status)
(PR #5520)
Added a voice_parameters setting to AzureTTSService and AzureHttpTTSService, which passes SSML's parameters attribute on <voice> through to Azure so HD voices can be tuned (temperature, top_p, top_k, cfg_scale, enhancePronunciation).
(PR #5522)
Added profanity_filter and redact to DeepgramFluxSTTService.Settings and DeepgramFluxSageMakerSTTService.Settings, exposing Flux's profanity masking and number redaction.
(PR #5523)
Added a version field to DeepgramSTTService.Settings (and DeepgramSageMakerSTTService.Settings), pinning transcription to a specific Deepgram model version instead of whichever one latest currently resolves to. Previously reachable only as an untyped extra key, which continues to work.
(PR #5527)
Added thinking to DeepSeekLLMService.Settings, which controls DeepSeek's thinking mode: settings=DeepSeekLLMService.Settings(thinking={"type": "disabled"}) turns off the reasoning pass that V4 models run by default.
(PR #5528)
CartesiaTurnsSTTSettings gained turn_start_threshold, turn_eager_end_threshold, turn_end_threshold and turn_end_timeout_ms, Cartesia's turn detection tuning parameters. Unset by default, so the server's own defaults apply.
(PR #5533)
Added a denoiser_config setting to GoogleSTTService, which passes Google's Chirp 3 background-noise removal (denoise_audio, snr_threshold) through to the recognition request.
(PR #5536)
Added a temperature field to HumeTTSService.Settings, passing Hume's sampling temperature through to the synthesis request. Higher values increase variation, lower values increase consistency; when unset, Hume applies its own per-model default.
(PR #5540)
Added a speed setting to DeepgramTTSService and DeepgramHttpTTSService, passing Deepgram's Aura speech-rate multiplier (0.7 to 1.5) through to /v1/speak.
(PR #5552)
Added a no_verbatim setting to ElevenLabsSTTService and ElevenLabsRealtimeSTTService, which asks Scribe to drop filler words, false starts and non-speech sounds from the transcript.
(PR #5555)
Added a hotwords field to WhisperSTTService.Settings, which biases Faster Whisper's transcription towards the words or phrases it names, such as product names or domain jargon.
(PR #5558)
Added a reduce_silence setting to SonioxTTSService, which shortens the pauses between words on models that support silence reduction.
(PR #5567)
Added an include_results setting to SpeechmaticsSTTService, which asks Speechmatics to include word-level results in transcript messages.
(PR #5568)
WebsocketService now accepts reconnect_backoff_min_wait and reconnect_backoff_max_wait to configure the wait between reconnection attempts. Defaults (4s / 10s) preserve the previous hardcoded behavior.
(PR #5586)
Added MetaSTTService, a streaming speech-to-text service using Meta's muse-voice-transcribe-1.0 model over the Muse Voice realtime API. Install it with pipecat-ai[meta], or pick Meta when scaffolding with pipecat init. MetaSTTService.Settings takes language or language_bias to bias recognition across the 25 supported languages, keywords for domain-specific terms, and mode to select the model's own endpointing, client-delimited turns, or diarization.
(PR #5587)
Added universal-3-6-pro as a supported AssemblyAISTTService model. It is universal-3-5-pro upgraded — the same model with the same feature set — and is recognized as part of the Universal-3 Pro family, so every u3-rt-pro feature (built-in turn detection, prompting, continuous partials, interruption_delay, context carryover, and voice focus) applies to it as well. universal-3-5-pro remains supported and stays the default.
(PR #5598)
StartupTimingReport.warmup reports what warming Pipecat's deferred imports cost startup, which no processor accounts for and which is often the largest single contributor to a cascaded bot's start. warmup.duration_secs is how long warming took and warmup.blocking_duration_secs is what it added beyond the slowest processor — zero for a pipeline whose services take longer to connect than warming takes to load. Observers can also handle the new BaseObserver.on_startup_warmup() event directly.
(PR #5600)
StartupTimingReport now accounts for the whole of a pipeline's startup, so a slow start can be read off the report rather than inferred. setup_phase_secs and start_phase_secs split the span into the two phases that compose differently: processors are set up concurrently, so the longest single piece of work decides that phase, while the StartFrame reaches them one at a time, so what each spends on it adds up. ProcessorStartupTiming.start_duration_secs reports the latter per processor alongside the existing setup_duration_secs.
(PR #5600)
Added language_codes and timestamps settings to AssemblyAISyncSTTService, aligning it with the Sync API's config surface. language_codes (e.g. [Language.EN, Language.ES]) declares the audio's languages for multilingual or code-switching audio, and is bound in preference to the single language when both are set; regional variants resolve to their base code and duplicates are dropped, preserving declaration order. timestamps requests per-word start/end timings on the transcription result, and is unset by default so the API default of False applies.
(PR #5603)
Added ServiceMetricsObserver, which reports each metric a service publishes as a record rather than a log line. on_service_latency carries time to first byte, first audio and first answer token, with the parts each decomposes into; on_service_usage carries speech-to-text audio seconds, text-to-speech characters, and the token counts an LLM reports. One record per metric, never summed, so a consumer groups them by turn, session or model.
(PR #5607)
Added SpeakingObserver, which reports a conversation's speaking lifecycle through on_speech_event: user_speech_started / user_speech_stopped as the detector heard them, user_turn_started / user_turn_stopped as the turn strategy ruled on them, bot_speech_started / bot_speech_stopped, and interruption. Speech is timed to where it began and ended rather than to where the detector confirmed it, and a moment that closes a stretch of speech names its started_at, so an interval reads whole from one record.
(PR #5612)
Added ErrorObserver, which reports every error a pipeline raises through on_error as an ErrorEvent: the message, the category it was attributed to, the exception_type behind it, the processor that raised it, and whether that processor can still do its job (processor_usable). Errors are read where they are raised, so ones a processor answers for itself — a ServiceSwitcher failing over, or a service it holds in reserve — are reported too; PipelineWorker's on_pipeline_error sees only the errors that reach the top of the pipeline.
(PR #5618)
Added FunctionCallObserver, which reports each function call through on_function_call_event as a FunctionCallEvent: function_call_started when the LLM asks for it, function_call_in_progress when it begins running, and one of function_call_completed, function_call_failed, function_call_timed_out or function_call_cancelled when it settles. Each moment names the one before it, so the wait to run and the time spent running each read from one record. Arguments travel by default and results do not; include_arguments and include_results change either.
(PR #5622)
DeepgramFluxSTTService, DeepgramFluxSageMakerSTTService and CartesiaTurnsSTTService push an EagerTranscriptionFrame when they predict an end of turn and an EagerEndOfTurnCancelFrame when they withdraw it, which EagerUserTurnStrategies acts on. Both frames are emitted only while enable_eager_end_of_turn is on.
(PR #5625)
Added support for answering an eager end of turn: a response is generated from a service's predicted end of turn and held until the turn is confirmed, so the gap before the committed end of turn is spent generating rather than waiting. Turn it on with enable_eager_end_of_turn=True on DeepgramFluxSTTService, DeepgramFluxSageMakerSTTService or CartesiaTurnsSTTService, which then recommend EagerUserTurnStrategies in place of ExternalUserTurnStrategies. The LLM service holds the response, so no extra processor is needed. The response is discarded if the user resumes speaking or if the committed transcript differs from the predicted one, per the strategy's match_policy (NormalizedMatch by default, which ignores the capitalization and punctuation services commonly add when committing a transcript, or ExactMatch to require the two to be identical). An unconfirmed turn never reaches the user or the LLM context, and tool calls are never executed for one.
(PR #5625)
Added FlowConfig, which describes a Pipecat Flows conversation as data so a bot can load its flow at runtime. A config names the initial node and, for each node, its messages, the tools it offers, and where each tool leads. Load it with FlowConfig.from_file(), from_yaml(), or from_json(), construct a Flow from it and the module holding your direct functions and action handlers, or a list of modules, and hand that to FlowManager.
Tools stay in Python and return (result, TRANSITION_IN_YAML); a function
that only moves the conversation is written in the config alone, as a
transition_only entry with a description. The config picks the next node,
by name or with a branch table keyed on a field of the result, where case
keys may be strings, booleans, or numbers. Constructing the Flow reports
every tool, handler, or variable the config names but cannot resolve,
together, as a FlowReferenceError. Actions use the built-in tts_say,
end_conversation, and function types, or a custom type registered in code
or given a handler name in the config. Prompts may use {{ key }}
placeholders, which FlowManager fills from its state on entering the
node, and a YAML config can keep long prompts in their own files with
!include path, resolved relative to the config. The format's JSON Schema is
kept in the repository at src/pipecat/flows/flow_config.schema.json for
editors and other tools to vendor.
initial_node: greet
nodes:
greet:
role_message: You take food orders for {{ restaurant_name }}.
task_messages:
- role: developer
content: Greet the caller and ask: pizza or sushi?
functions:
- name: choose_pizza
transition_only: true
description: The caller wants pizza.
transition_to: pizza
pizza:
task_messages:
- role: developer
content: Take the pizza order.
functions:
- name: select_pizza_order
transition_to:
field: status
cases:
ok: confirm
unavailable: pizza
import handlers # direct functions: select_pizza_order, ...; action
handlers
config = FlowConfig.from_file("flow.yaml")
flow = Flow(config, handlers=handlers)
flow_manager = FlowManager(
worker=worker,
llm=llm,
context_aggregator=context_aggregator,
global_functions=flow.global_functions,
)
flow_manager.state["restaurant_name"] = "Luigi's" # fills {{
restaurant_name }}
await flow_manager.initialize(flow.initial_node)
The flows examples are grouped by form: examples/flows/yaml/ holds a
folder per YAML flow with bot.py, flow.yaml, and handlers.py
(hello_world, food_ordering, restaurant_reservation, patient_intake,
insurance_quote, and podcast_interview), and examples/flows/python/ holds
the flows that show what needs code. The README says when to write a flow in
each form.
(PR #5628)
Added pipecat.utils.yaml.include_loader(), which returns a yaml.SafeLoader subclass that resolves !include <relative-path> tags against a base directory. Flow configs and eval scenarios use it; any YAML-based configuration can.
(PR #5628)
A suite manifest lists scripted and simulated scenarios side by side under scenarios:, and pipecat eval suite -k simulation (or -k script) runs only one kind.
(PR #5646)
Simulations for pipecat eval: give a scenario a persona: and a goal:
instead of scripted turns, and an LLM plays the caller, holding the whole
conversation with your bot, in text or over synthesized speech, and hanging
up once it has what it came for. A judge then reads the transcript, the bot's
tool calls in place, and decides whether the bot did its job (success:) and
how every reply scored on your metrics:, judged per turn (criterion, with
min_score as the share of turns that must pass) or measured from the run
(measure: turns, duration, words, or latency against a range, or
function_calls against the list of calls the bot should make, [] for
none). pipecat eval run plays one; a suite runs it runs: times and every
run must pass.
persona: "Jamie, booking dinner for two tonight."
goal: "Book a table for two at 6 PM, then end the call."
success: "the bot confirmed a reservation for two at 6 PM"
metrics:
- name: politeness
criterion: "the reply is courteous, never curt or dismissive"
min_score: 1
- measure: latency
max_value: 5
runs: 3
(PR #5646)
RTVIClientTransport and RTVIClientSerializer let a Pipecat pipeline join a bot as an RTVI client. The eval harness now runs on them: its STT, TTS, persona, and recording are stages of a pipeline of its own, and each suite run gets its own process.
(PR #5646)
Added require_given() to pipecat.utils.types. It narrows a settings value past NOT_GIVEN and None like assert_given(), and also rejects the empty string, raising ValueError("<what> must be specified") for settings a service cannot run without.
(PR #5670)
A simulation's results tell judge trouble from bot trouble: a turn the judge left unanswered carries verdict: none, a failed metric carries a failure_kind (judge_no, judge_no_verdict, out_of_range, function_calls), and a judge that gives no verdict on the goal makes the run an error with kind judge_no_verdict, kept out of the suite's pass rate. A failed latency measure's reason says what was timed, the first token in text mode or the first spoken sentence in audio mode.
(PR #5678)
A simulation ends as soon as it is going nowhere: silence when neither side does anything for max_silence_s (30 s by default), and an error the moment the harness's own pipeline fails, the persona LLM first among them, instead of waiting out max_duration_s.
(PR #5678)
A simulation's simulator: block is optional: without one the persona runs on the same local Ollama model as the default judge, gemma4:12b, so a simulation needs no API key. The release-eval simulations use it.
(PR #5678)
Added BackendLLMWorker and BackendOutput (pipecat.workers.llm): a worker that runs any LLM service, with its own context and multi-step tool calling, as the backend OpenAILiveLLMService client delegation hands work to. Everything the backend produces comes back as a BackendOutput saying what it is and whether the user may hear it, and transform_output decides that per output, or rewrites the text on its way out — see examples/realtime/realtime-openai-live-client-delegation-spoken-updates.py.
(PR #5688)
Added OpenAILiveLLMService, a speech-to-speech service for the OpenAI Live API (gpt-live-1). The live model is full-duplex — it listens and speaks at the same time and handles being interrupted itself — and delegates search, reasoning and tool use to a backend model while the conversation continues. Both delegation modes are supported: OpenAILiveLLMService.ResponsesDelegation lets OpenAI host the backend (Responses API) model, with its function calls executed by the pipeline's registered tool handlers; OpenAILiveLLMService.ClientDelegation runs any Pipecat LLM service as the backend through a BackendLLMWorker. See examples/realtime/realtime-openai-live-responses-delegation.py and examples/realtime/realtime-openai-live-client-delegation.py, plus examples/realtime/realtime-openai-live-client-delegation-spoken-updates.py for a backend that chooses what the user hears.
(PR #5688)
⚠️ Behavior change: DeepgramSTTService and DeepgramSageMakerSTTService no longer send profanity_filter, so Deepgram's own default of off applies. Deepgram's filter rewrites the words it matches rather than tagging them, so a false positive silently corrupts a transcript that may be stored, analyzed, or sent downstream. To keep the previous behavior, pass settings=DeepgramSTTService.Settings(profanity_filter=True).
(PR #5462)
CartesiaTTSService and CartesiaHttpTTSService now default to sonic-3.6, Cartesia's current Sonic model. Pass model="sonic-3.5" to stay on the previous default.
(PR #5472)
soundfile is now a required dependency instead of an optional extra, so SoundfileMixer and the eval harness can load audio files with no extra install step. The pipecat-ai[soundfile] extra still resolves and is now a no-op, so existing installs keep working unchanged.
(PR #5473)
text_contains on a user_transcription eval expectation now accumulates the STT's final transcription segments within the turn before checking, so an STT that finalizes an utterance in pieces can still satisfy a phrase. Substring checks also ignore whitespace differences between the expected text and the event's text.
(PR #5473)
Raised the pipecat-ai[cli] Context Hub floor to pipecat-ai-context-hub 0.6.0, which registers its MCP server with Claude Code for every directory rather than only the one install ran in. A project scaffolded after the hub was set up now has the server without re-running install.
(PR #5480)
Bots scaffolded by pipecat init build their WorkerRunner before wiring up event handlers, so handlers can end the session with await runner.cancel(). The runner takes handle_sigint=runner_args.handle_sigint, letting a bot run directly install its own signal handler while a hosted one leaves signals to its host.
(PR #5482)
The AssemblyAI STT examples and docs now name universal-3-5-pro, AssemblyAI's current Universal-3 Pro streaming model. The u3-rt-pro names are legacy aliases AssemblyAI redirects to it and are still accepted by AssemblyAISTTService.
(PR #5507)
AWSTranscribeSTTService now maps 46 more AWS Transcribe streaming languages, including Turkish, Hungarian, Tamil, Telugu, Swahili, Mexican Spanish and Welsh. Passing one of these as a Language enum previously sent a bare base code such as tr, which AWS Transcribe rejects.
(PR #5509)
AWSPollyTTSService now maps Language.EN_IE and Language.EN_SG to Polly's en-IE and en-SG locales, so Irish and Singaporean English no longer log an unverified-language warning.
(PR #5510)
Raised the pipecat-ai[cli] Context Hub floor to pipecat-ai-context-hub 0.7.0, whose refresh indexes the framework at its newest release tag rather than tracking its default branch. Hub answers stop mixing in unreleased APIs stamped with the previous release's number, which version_compatibility reported as compatible to anyone running that release.
Pass --framework-version head to index the default branch — the right choice when developing Pipecat itself or building against unreleased code. A --framework-version tag that does not exist now fails the refresh instead of logging a warning and exiting 0.
(PR #5513)
SarvamTTSService and SarvamHttpTTSService now default to bulbul:v3, Sarvam's current TTS model. Sarvam's API no longer serves bulbul:v2 and rejects requests for it, so the previous default could not synthesize. The v3 defaults come with it: shubh as the speaker and a 24000 Hz sample rate.
(PR #5517)
SmallestTTSService now maps all 31 languages the Waves Lightning API accepts, and no longer maps Language.HE, which the API rejects.
(PR #5519)
FireworksLLMService now defaults to accounts/fireworks/models/nemotron-3-ultra-nvfp4. Fireworks no longer serves the previous default, accounts/fireworks/models/firefunction-v2, on its serverless endpoint, so requests using it returned a 404.
(PR #5524)
CartesiaTTSService and CartesiaHttpTTSService now map Language.OR (Odia) and Language.UR (Urdu), the two languages Sonic 3.6 added.
(PR #5532)
Added a turn_coverage setting to GeminiLiveLLMService.Settings, which selects how much of the realtime input stream a user turn covers (for example TURN_INCLUDES_ONLY_ACTIVITY to keep video frames outside detected audio activity out of the turn).
(PR #5535)
GroqTTSService now supports a variety of sample rates, defaulting to 24khz.
(PR #5538)
LiveAvatarNewSessionRequest now defaults to H264 video encoding, matching the LiveAvatar API's own default. LiveAvatar has deprecated VP8; pass video_settings=VideoSettings(encoding=VideoEncoding.VP8) to keep the previous encoding while it lasts.
(PR #5539)
KokoroTTSService.Settings has a speed field, kokoro-onnx's speech rate multiplier (0.5 to 2.0, default 1.0), settable at construction and at runtime.
(PR #5543)
OpenAIResponsesLLMService's service_tier documentation now lists OpenAI's full set of tiers, including fast — the low-latency tier that replaced the priority name.
(PR #5548)
The OpenAI server-side turn detection example now uses gpt-transcribe, replacing gpt-4o-transcribe, which shuts down on 2027-02-26.
(PR #5549)
OpenAISTTService now defaults to gpt-transcribe, OpenAI's replacement for gpt-4o-transcribe, which shuts down on 2027-02-26. With include_prob_metrics=True, GPT transcription models now request logprobs alongside a json response; only Whisper models use verbose_json.
(PR #5549)
PocketTTSService now loads pocket-tts's distilled models for German, Spanish, Italian and Portuguese instead of the 24-layer variants, which are too slow to synthesize in real time on a typical CPU.
(PR #5553)
FalImageGenService.Settings now accepts negative_prompt and guidance_scale, passed through to Fal's image generation request when set.
(PR #5556)
MoonshineSTTService now maps Language.DE (German) and Language.TL / Language.FIL (Tagalog/Filipino), the languages Moonshine added to its model catalogue.
(PR #5557)
TogetherLLMService now defaults to zai-org/GLM-5.2. The previous default, zai-org/GLM-5.1, has been retired from Together's serverless inference and returns model_not_available. Pass settings=TogetherLLMService.Settings(model=...) to choose a different model.
(PR #5570)
NeuphonicTTSService and NeuphonicHttpTTSService now accept temperature in their settings, passing Neuphonic's synthesis randomness control (0.0–1.0) through to the API. Left unset, Neuphonic's own default applies.
(PR #5578)
NvidiaLLMService now defaults to nvidia/nemotron-3-super-120b-a12b. The previous default, nvidia/nemotron-3-nano-30b-a3b, reached end of life on NVIDIA's NIM cloud endpoint and now returns HTTP 410.a
(PR #5579)
⚠️ Behavior change: A word-timestamp event that matches nothing left to speak is now dropped instead of being emitted as a TTSTextFrame. Passing it through put text the LLM never wrote into the conversation context. Nothing is lost: the text that word should have covered is carried by the word that puts the sentence back in step, or emitted when the audio context ends.
(PR #5585)
⚠️ Mem0MemoryService now adds retrieved memories with the "developer" role instead of "system" when add_as_system_message is set. Memories are extra guidance for the model, not a system prompt. Services without a "developer" role receive them as "user" — the same treatment non-OpenAI services already gave the "system" message.
(PR #5596)
SegmentedSTTService now appends 0.5 s of silence to each speech segment before transcribing it, so the model hears the end of speech and finishes the last word instead of dropping or garbling it. This applies to every service built on it, such as WhisperSTTService, OpenAISTTService, GroqSTTService and MoonshineSTTService. The padding is submitted to the provider, so it counts toward STT usage metrics. Pass trailing_silence_secs=0 to the service to disable it, or another value to tune it.
(PR #5621)
FunctionCallResultFrame now carries an error field, set when a call ends because its handler raised, so a failed call can be told apart from a successful one. The result is unchanged: it remains the stand-in message the LLM reads.
(PR #5622)
⚠️ Behavior change: the Deepgram Flux services no longer push their eager end-of-turn transcript as an InterimTranscriptionFrame. That transcript is a prediction the service may withdraw, and it now travels as an EagerTranscriptionFrame, emitted only under enable_eager_end_of_turn, so it is no longer surfaced to clients as partial user speech. Flux emits no interim transcriptions as a result; subscribe to on_update for incremental transcript text.
(PR #5625)
The on_user_turn_inference_triggered event emitted by UserTurnController and by BaseUserTurnStopStrategy now carries a UserTurnSpeculation | None as its last argument, set when the inference answers a turn that hasn't ended yet and carrying the turn text it was run against. A custom stop strategy requests one by passing it to trigger_user_turn_inference_triggered(). The events of the same name on LLMUserAggregator and UserTurnProcessor are unchanged, so handlers registered on those keep their signature.
```python
@controller.event_handler("on_user_turn_inference_triggered")
async def on_user_turn_inference_triggered(controller, strategy,
speculation): ... ``` (PR #5625)
LLMContextFrame carries a speculation flag marking an inference as speculative, so the response it produces is held until the turn it answers is confirmed.
(PR #5625)
The AGENTS.md that pipecat init writes now describes Pipecat Flows as part of Pipecat (pipecat.flows) and directs coding agents to build a new flow as a FlowConfig YAML file plus a Python tools module, reserving Python-built flows for routing or prompts that need code.
(PR #5628)
Eval suite results.jsonl records carry scenario and kind (script or simulation), and a repeated sweep (--repeat) reports pass rates and exits 0.
(PR #5646)
pipecat eval loads the nearest .env (walking up from the working directory, shell variables winning) before running, so a simulation's persona LLM and a hosted judge find their credentials the way the bots do. python-dotenv is now a core dependency.
(PR #5646)
CrusoeLLMService now defaults to openai/gpt-oss-120b. The previous default was zai/GLM-5.2. Pass settings=CrusoeLLMService.Settings(model=...) to choose a different model.
(PR #5656)
The Speechify service now sends the standard Speechify-Caller-Version attribution header (valued from pipecat_version()), replacing the non-standard X-Pipecat-Version header.
(PR #5668)
The runner extra now requires pipecat-ai-prebuilt>=1.1.0. The prebuilt client UI served by the development runner gains a LiveKit option in its transport selector, backed by @pipecat-ai/livekit-transport 1.0.0, and moves to @pipecat-ai/voice-ui-kit 0.14.0.
(PR #5675)
EvalSession.from_scenario() builds the session for a loaded scenario of either kind, an EvalScriptSession for a scripted scenario or an EvalSimulationSession for a simulation, and EvalSession is the base class of both. Constructing EvalSession(...) directly is no longer supported; use the kind's own class.
(PR #5676)
EvalClientParams holds only what the scenario asks of the bot, and EvalClient takes it together with the run's EvalSessionParams and the services. EvalClient.for_scenario() and EvalClient.for_simulation() are gone; the sessions construct the client.
(PR #5676)
The eval sessions and EvalSuite.run() take how a run behaves as one EvalSessionParams (connect_timeout_s, default_timeout_ms, record_path, cache_dir, use_cache, stop_bot, trigger_disconnect), and the services a run uses (judge, user_tts, bot_stt, and a simulation's persona_llm) as keyword arguments on both from_scenario() and the constructors.
(PR #5676)
Cartesia word timestamps for Chinese and Japanese now keep one token per reported entry, each with its own start time, instead of joining a whole word_timestamps message into a single token.
(PR #5679)
PiperTTSService downloads voice models to ~/.cache/pipecat/piper by default, the same cache location KokoroTTSService uses, instead of the current working directory. Pass download_dir to keep models elsewhere; a model already downloaded into a working directory is fetched again into the cache on first use.
(PR #5687)
Deprecated LatencyBreakdown.chronological_events(). Use LatencyBreakdown.turn_contribution_lines() instead, which names every part of the interval rather than the services that happened to report a metric, and whose lines sum to the measured latency. Will be removed in 2.0.0.
(PR #5445)
Deprecated SarvamTTSModel.BULBUL_V2. Sarvam's API rejects bulbul:v2 on both the REST and the WebSocket endpoint, so it cannot synthesize; use SarvamTTSModel.BULBUL_V3. Constructing SarvamTTSService or SarvamHttpTTSService with it still resolves a model config and now emits a DeprecationWarning. It will be removed in 2.0.0.
(PR #5517)
Deprecated the on_progress parameter of EvalSession() and EvalSession.from_scenario(), and the on_update parameter of EvalSuite.run(). Use the event handlers of the same name instead. Passing a callback still works, keeps its existing signature, and emits a DeprecationWarning. They will be removed in 2.0.0.
(PR #5520)
Setting the system prompt as a "system" message at the start of LLMContext is deprecated and will stop working in 2.0.0. Set it on the LLM service instead:
```python
# Before
context = LLMContext([{"role": "system", "content": "Be helpful."}])
# After
llm = OpenAILLMService(api_key=..., system_instruction="Be helpful.")
context = LLMContext()
```
To change the prompt at runtime, push an LLMUpdateSettingsFrame carrying LLMSettings(system_instruction=...) instead of rewriting the first context message.
(PR #5596)
EvalScenario, EvalResult, EvalTurn, EvalTurnResult, and EvalTurnProgress are now EvalScriptScenario, EvalScriptResult, EvalScriptTurn, EvalScriptTurnResult, and EvalScriptTurnProgress, the scripted kind of scenario. The old names remain as aliases until 2.0.0.
(PR #5646)
pipecat.evals.harness moved to pipecat.evals.script_session. The old module path remains until 2.0.0.
(PR #5646)
RTVIEvalSerializer is now EvalSerializer, the serializer of the bot's EvalTransport; its client-side counterpart is EvalClientSerializer. The old name remains as an alias until 2.0.0.
(PR #5646)
Deprecated the connect_timeout_s, default_timeout_ms, record_path, cache_dir, use_cache, stop_bot, and trigger_disconnect keyword arguments of EvalSession.from_scenario() and EvalScriptSession.from_scenario(), and the use_cache and default_timeout_ms keyword arguments of EvalSuite.run(). Pass params=EvalSessionParams(...) instead. The old arguments still work, override the params field of the same name, and emit a DeprecationWarning. They will be removed in 2.0.0. EvalSimulationSession.from_scenario() is new and takes params only.
(PR #5676)
Deprecated service: openai in a scenario's judge.eval: and simulator: blocks and service: cartesia in its user.speech: block, along with the openai_service() and cartesia_service() builders behind them. The built-in names are the local, keyless services (Ollama, Kokoro, Moonshine, Whisper); any other provider is a factory:, a dotted path to a callable that takes the block and returns the service, documented with examples in pipecat.evals.services. The old names still work and emit a DeprecationWarning; they will be removed in 2.0.0.
(PR #5678)
SEND_CHUNK_MS from pipecat.evals.harness. The harness now sends the user's audio through its own output transport, which paces it like any Pipecat transport does.\n
(PR #5646)Fixed AsyncAITTSService ignoring runtime model, voice and language updates. Those three are sent only in the websocket init message and never repeated per utterance, so a change to any of them now opens a new session instead of being stored and warned about.
(PR #5464)
Deepgram Flux STT now applies a model or numerals update by reconnecting, since Flux reads those only from the connection URL. The reconnect waits until the user stops speaking, and settings Flux accepts on a live connection (keyterm, eot_threshold, eager_eot_threshold, eot_timeout_ms, language_hints) are still sent without dropping it.
(PR #5477)
A Deepgram Flux FatalError now reports the code and description Flux sends, instead of a generic Unknown error.
(PR #5477)
Deepgram Flux STT now fails instead of hanging when a connection cannot be established. A connection setting the endpoint rejects leaves the SageMaker session opening forever and produces no confirmation on either transport, so both the session handshake and the wait for Flux to confirm it are now bounded and reported through on_connection_error.
(PR #5498)
Deepgram Flux STT now marks itself unusable once it can no longer transcribe — after a fatal error from Flux, or a connection that never establishes — so the failure reaches the pipeline instead of the service staying connected and silent. (PR #5498)
Fixed GladiaSTTService dropping Gladia's stop_recording message on a graceful end. The websocket closed before the message was sent, so sessions ended by dropping the connection rather than closing cleanly, and the service's on_disconnected event fired twice.
(PR #5499)
Fixed the pipeline idle timeout cancelling a live session while the user is still talking, on pipelines that get their turns from a provider instead of local VAD. idle_timeout_frames now counts UserStartedSpeakingFrame, TranscriptionFrame and InterimTranscriptionFrame as user activity, alongside BotSpeakingFrame and the VAD-driven UserSpeakingFrame.
(PR #5502)
Corrected the Nova Sonic region lists in AWSNovaSonicLLMService's docstring and example: eu-north-1 is a supported region for both Nova 2 Sonic and Nova Sonic.
(PR #5508)
Fixed the Vonage-Google package conflict so it no longer prevents GeminiSTTService usage in environments where the Vonage connector can't be installed anyway, such as development environments on macOS. The repo lockfile resolved google-genai to 2.8.0 — below the 2.9.0 the service needs — for every Python 3.13 environment, to satisfy vonage-video-connector's pydantic<2.12 pin, though the connector is installable only on Python 3.13 Linux. That environment is now resolved apart from every other. Installs from PyPI were never affected.
(PR #5512)
Fixed SarvamHttpTTSService ignoring its sample_rate. It named the field sample_rate in the request body where Sarvam's REST API reads speech_sample_rate, so synthesis always came back at the model's default rate while the audio frames were tagged with the requested one — audio played back at the wrong speed for any other rate.
(PR #5517)
Fixed the Fireworks function-calling example, which named accounts/fireworks/models/gpt-oss-20b. Fireworks retired that model from its serverless endpoint, so the example now uses accounts/fireworks/models/nemotron-3-ultra-nvfp4.
(PR #5524)
Fixed the examples/flows/llm_switching.py Bedrock service, which pinned a Claude model Bedrock has retired and a latency mode Bedrock does not offer for its replacement. It now uses us.anthropic.claude-haiku-4-5-20251001-v1:0.
(PR #5525)
Corrected GeminiTTSService's documented model names: they differ between the Cloud Text-to-Speech and Gemini API backends, and the docstrings named only Cloud Text-to-Speech's gemini-2.5-flash-tts / gemini-2.5-pro-tts, which the Gemini API rejects with a 404.
(PR #5526)
Fixed Deepgram STT settings passed through extra being discarded when the key matched a declared settings field (for example extra={"diarize": True}). Such keys are now promoted to their field and sent to Deepgram.
(PR #5527)
Fixed GeminiTTSService ignoring its language setting on the Gemini API (GenAI) backend. The configured language is now sent as the speech config's language_code, as it already is on the Cloud Text-to-Speech backend.
(PR #5537)
LmntTTSService now maps all 31 languages LMNT's Blizzard model speaks, adding Assamese, Bengali, Czech, Danish, Finnish, Malayalam, Marathi, Slovak, Tamil and Telugu. These languages already reached the API through the base-code fallback, which logged a "not verified" warning for each of them.
(PR #5544)
Fixed an OpenAISTTService error where include_prob_metrics=True sent a request that diarization models reject. Those models report no per-token or per-segment probabilities, so the service now transcribes without requesting them and warns once.
(PR #5549)
Fixed the examples/voice/voice-pockettts.py import of PocketTTSService, which named a module that does not exist.
(PR #5553)
Fixed an issue where ElevenLabsRealtimeSTTService ignored the realtime API's quota_exceeded, unaccepted_terms and invalid_request messages instead of reporting them as errors.
(PR #5555)
Fixed the PCM sample rates GrokRealtimeLLMService accepts: the Grok Voice Agent API's 22050 Hz option is now allowed, and 21050 Hz — which the API rejects — is not.
(PR #5560)
Fixed the Rime TTS services' language mapping: Arabic, Italian, Japanese and Portuguese — all served by Rime's coda model — now resolve to Rime's language codes, and a region-qualified language such as Language.EN_US falls back to a base code Rime accepts instead of a BCP-47 tag that made Rime close the connection without synthesizing.
(PR #5564)
SambaNovaLLMService now sends the frequency_penalty, presence_penalty and seed settings, which SambaNova's chat completions API accepts. They were previously dropped from the request.
(PR #5565)
Fixed a SpeechmaticsSTTService error where setting split_sentences raised ValueError: "VoiceAgentConfig" object has no field "split_sentences" and the service failed to construct.
(PR #5568)
Fixed SpeechmaticsTTSService reporting the speechmatics-rt SDK version, rather than Pipecat's, in the sm-app tag it sends to Speechmatics. The tag now carries the Pipecat version, matching SpeechmaticsSTTService.
(PR #5569)
Fixed the NVIDIA examples update-settings/llm/llm-nvidia.py and voice/voice-nvidia-sagemaker.py, which requested Llama models that NVIDIA's NIM cloud endpoint no longer serves.
(PR #5579)
Corrected the model docstring in OneShotInputParams for UltravoxRealtimeLLMService: the field defaults to None, which lets Ultravox pick its current default model, rather than to the legacy fixie-ai/ultravox alias.
(PR #5582)
Fixed word-level tracking stalling for the rest of a sentence when a TTS service reports a word-timestamp event that doesn't match what is being spoken. The next correctly reported word is now matched past the gap and carries the text that was never reported, so TTSTextFrames and RTVI progress keep pace with the audio. Previously the sentence stayed stuck at the first unmatched word, every word after it went unmatched too, and the remainder arrived as a single frame once the audio context ended.
(PR #5585)
Fixed NvidiaLLMService emitting reasoning frames before TTFB metrics, so observers receive TTFB before reasoning output for structured reasoning fields and inline <think> blocks.
(PR #5589)
The LLM usage debug log now includes cache creation tokens alongside cache reads. (PR #5602)
OpenAI-compatible LLM services now report prompt-cache write tokens as LLMTokenUsage.cache_creation_input_tokens, so usage-based cost tracking accounts for them. Covers BaseOpenAILLMService and the services inheriting it, OpenAIResponsesLLMService, and SambaNovaLLMService. Providers that do not report the count leave the field unset.
(PR #5602)
OutputAudioRawFrames pushed from upstream of an STT service (for example a sound effect or a recording played into the pipeline before the STT) now reach the output transport untouched. STT services only transcribe InputAudioRawFrames. Previously any audio frame was treated as user speech, so the audio was sent to the transcription provider and, with audio_passthrough=False, never reached the transport.
(PR #5609)
VoicemailDetector no longer swallows EndWorkerFrame travelling upstream through a closed gate, so a handler that ends the call reaches PipelineWorker. Closed classifier and conversation gates already admitted CancelWorkerFrame (a system frame) but dropped other worker lifecycle frames.
(PR #5614)
SegmentedSTTService (Whisper, Moonshine, and the other segmented STT services) now transcribes each speech segment in a background task. Previously the transcription blocked the service's input audio frames, so a VAD downstream could not analyze them until it finished. Transcripts are still pushed in segment order.
(PR #5630)
A punctuation mark reported as its own word-timestamp entry no longer shifts the text around it or records the mark twice. Trailing punctuation is attributed to the word before it, so the mark's own event arrived with nothing left to account for and spent a character of the following word instead — no text was lost and the cursors resynced a word later, but the words in between were recorded with the break misplaced ("Yeah", ",", "I" gave "Yeah, I c an help"), and a turn ending inside that window kept the misplaced split. Where a service forwards raw tokens (pre_merge_tokens=False) the mark was also written to the context a second time. Most visible for languages reported one character at a time, where every mark is affected, though not specific to them.
(PR #5633)
LLMService now logs the composed system instruction only when it changes. Previously every tool sync recomposed it, so the full prompt was logged once per turn.
(PR #5635)
Fixed LLMContextSummarizer's message-count trigger (max_unsummarized_messages), which never reset after a summary was applied and counted the injected summary message as a new conversation turn.
(PR #5642)
Bots scaffolded by pipecat init with Sarvam, Whisper, or Moonshine STT, Sarvam TTS, or Google STT no longer crash at startup. Sarvam and Google STT now use each service's default model or location, and Whisper and Moonshine read an optional WHISPER_MODEL or MOONSHINE_MODEL variable that falls back to the default model. Previously the generated code read a variable the generated .env.example never listed, and the unset variable arrived as an explicit None that overrode the working default and failed validation.
(PR #5661)
The generated .env.example now lists every variable the scaffolded bot reads. This adds entries for ElevenLabs realtime STT, Gemini STT and TTS, Mistral STT, xAI STT, Soniox TTS, Deepgram Flux TTS, Deepgram Flux SageMaker TTS, and the NVIDIA TTS voice ID, names the Fal key FAL_API_KEY to match the generated code, and reads the AWS Transcribe session token from AWS_SESSION_TOKEN instead of the misspelled AWS_AWS_SESSION_TOKEN.
(PR #5661)
Fixed WhisperSTTService raising ValueError at construction when Settings(language=None) is paired with a multilingual model. None asks Whisper to detect the language itself, which every model supports; the language check previously looked None up in the model's supported-language list.
(PR #5669)
Fixed the Model.DISTIL_LARGE_V2 and MLXModel.DISTIL_LARGE_V3 Whisper models being documented as multilingual. Every Distil-Whisper release is English-only.
(PR #5669)
Local STT and TTS services now reject a missing or blank model or voice at startup with a "must be specified" error. Piper, Pocket TTS, Kokoro, and XTTS check the voice in the constructor (Pocket TTS before loading its model, and Kokoro also against the loaded voices file), and Whisper, Whisper MLX, and Moonshine check the model. Previously a blank value, which is what a copied .env.example produces, reached the underlying library and failed with an unrelated message, sometimes only on the first utterance. XTTS also reports an unknown studio speaker as an error frame instead of raising a KeyError.
(PR #5670)
Fixed XAISTTService emitting each utterance's text twice. xAI sends a chunk final for every locked segment and then an utterance final that restates the whole utterance, and both were pushed as TranscriptionFrame, so LLM context aggregators and other consumers that accumulate finals saw the text duplicated. Chunk finals are now pushed as InterimTranscriptionFrame and only the utterance final becomes a TranscriptionFrame, always with finalized=True.
(PR #5673)
Fixed SonioxSTTService dropping Soniox's end-of-audio frame on a graceful end. The websocket closed before it was sent, so sessions ended by dropping the connection rather than closing cleanly, and the service's on_disconnected event fired twice.
(PR #5682)
Fixed SarvamSTTService mislabeling every transcript as Hindi when no language was detected, the default behavior in transcribe mode. Unresolved languages now stay None, and ur-IN, mai-IN, sd-IN, and kok-IN are now correctly mapped.
(PR #5692)
The pipecat eval suite dashboard fits the terminal: rows are windowed to the active runs, including in a repeated sweep, and never wrap, so the tally at the bottom is always on screen. A running row's spinner animates and each row carries a clock, from its first attempt's start to its last attempt's end; the failures list is set apart from its header by a blank line.
(PR #5694)
CartesiaSTTService, DeepgramFluxSTTService, and DeepgramFluxSageMakerSTTService now log the provider's request ID when the connection is established, so a session can be traced in provider-side diagnostics.
(PR #5475)
Documented SonioxSTTService's context setting as accepting either a SonioxContextObject or a plain string of background text on any current Soniox model, along with the 8k-token cap Soniox applies to the context object.
(PR #5566)
Your coding agent can read these notes before it upgrades. Set up the MCP server →
A directory passed to pipecat eval run now reads .yml files as well as its .yaml ones, matching the scenario names a manifest resolves. A directory ho
A directory passed to pipecat eval run now reads .yml files as well as its .yaml ones, matching the scenario names a manifest resolves. A directory holding neither is still an error rather than an empty run.
(PR #5463)
Fixed pipecat init repeatedly offering to build a Context Hub index that already exists, and the stale-index warning never appearing, on any index refreshed by Context Hub v0.5.3 or later. A future hub release can no longer silence these checks.
(PR #5468)
SpeechTimeoutUserTurnStopStrategy no longer overrides the deprecated reset() hook. Turn detection through UserTurnController is unaffected; code that…
Added image_url support for Gemini adapter, enabling external URLs via Part.from_uri().
(PR #3573)
Added Google Speech-to-Text v2 adaptation support to GoogleSTTService, so recognition can be biased toward domain terms using inline or referenced phrase sets. Configurable at construction and updatable at runtime.
(PR #4413)
Added MCPClient(tools_arguments=...), which injects extra arguments into every call of a tool. Use it for arguments the model shouldn't choose — a fixed search mode, an account id, a caller-supplied filter. The pinned arguments override anything the model supplies, and are hidden from the schema it sees:
mcp = MCPClient(
server_params=...,
tools_arguments={"search": {"mode": "realtime"}},
)
In this example, the model only ever sees search(query=...), while every call reaches the server as search(query=..., mode="realtime").
(PR #4939)
Added MCPClient.tools(): LLMContext(tools=await mcp.tools()) is now all you need to use MCP tools — connecting, tool registration, and closing the connection at pipeline end are automatic.
(PR #4939)
Added KeenableWebSearch (pipecat.services.keenable.search), an optional service that gives voice agents live web search and page reading via a hosted MCP server powered by Keenable AI. It exposes the server's search_web_pages (with optional site and date-range filters) and fetch_page_content tools — pass await search.tools() to your LLMContext and the tools register automatically (the connection is released automatically when the pipeline ends, too). Install with the keenable extra. Works keyless (pro mode); pass api_key= for higher rate limits and access to the lower-latency realtime mode (requires an account with realtime mode enabled), selected with mode="realtime".
(PR #4942)
Added the Pipecat Context Hub to the cli extra, so uv tool install "pipecat-ai[cli]" provides pipecat context-hub (alias pipecat ch) with no separate install — the guides pipecat init writes tell coding agents to query the hub, so the CLI ships it. Costs about 195 MB on top of the extra; bot runtimes are unaffected, since cli stays optional so base images remain lean. The scaffolded agent guides teach pipecat context-hub as the primary way to query the hub, with uvx pipecat-ai-context-hub as the no-install fallback.
(PR #5122)
Added Context Hub setup to pipecat init. On the coding-agent path it registers the hub's MCP server with each coding agent CLI it finds, and says what came of it: Cursor, VS Code, and Zed are configured by hand, so it points at pipecat context-hub install to print the config block to paste, and a client that rejects the registration reports why. It then offers to build the local index when there isn't one — a few minutes and roughly 900 MB, so it asks rather than assumes. The question only appears while no index exists, so it doesn't return once you have one. pipecat init quickstart skips setup entirely to stay a short path to a running bot, and --no-context-hub opts out anywhere.
(PR #5122)
Added a Pipecat Context Hub freshness notice to the CLI. When a local hub index exists and has gone stale, or was built for a different pipecat-ai minor than the project in the working directory, the CLI prints a one-line hint on stderr suggesting pipecat context-hub refresh — so a coding agent citing an API that has since changed is caught before the generated code is. The check reads the hub's published index metadata directly with the standard library, adding no dependency and no meaningful startup cost. It is silent when no index exists, compares only major.minor (ignoring patch and dev segments), stays quiet for editable pipecat checkouts, and can be switched off with PIPECAT_HUB_CHECK=0; the staleness threshold shares the hub's own PIPECAT_HUB_STALE_AFTER_DAYS.
(PR #5122)
Added ProposedUserStartedSpeakingFrame and ProposedUserStoppedSpeakingFrame, the way a service with its own turn detection tells the pipeline where it thinks a turn boundary falls. ExternalUserTurnStrategies resolve those proposals into UserStartedSpeakingFrame / UserStoppedSpeakingFrame, so the strategies are a single place that decides turns and can be subclassed to adjust the timing — previously a service that emitted turn frames took the reins entirely and left nothing to extend. See examples/turn-management/turn-management-custom-external-turn-strategy.py for a stop strategy that holds the turn open past the service's proposal so a trailing afterthought can reopen it.
Every in-repo service with built-in turn detection now emits proposed turn frames rather than real turn frames: the AssemblyAI, Cartesia Ink-2, Deepgram Flux, Gladia, Sarvam, Soniox, Speechmatics, and OpenAI Realtime STT services, and the OpenAI, xAI, and Inworld realtime LLM services. Third-party services that emit UserStartedSpeakingFrame / UserStoppedSpeakingFrame directly keep working unchanged; switching them to the proposal frames hands interruption handling back to the pipeline.
OpenAIRealtimeSTTService (the transcription-only service, not the speech-to-speech OpenAIRealtimeLLMService) and SarvamSTTService now also recommend ExternalUserTurnStrategies when their server-side VAD is enabled, matching the other turn-detecting services. Their turn frames were previously informational and nothing in the pipeline acted on them.
(PR #5156)
Enabled MoQ client mode, where the bot and the browser both dial a relay and rendezvous there instead of the bot serving its own socket. Since neither side needs a reachable address, this works when the bot is behind NAT.
python bot.py -t moq --moq-connect https://cdn.moq.dev/anon. Without --moq-connect the bot serves its own socket, as before.--moq-namespace to pin a well-known room instead.MOQParams.response_path and MOQParams.request_path, which set the bot's broadcast paths directly (the bot publishes its response_path, subscribes to the peer's request_path) instead of deriving them from namespace + participant_id / peer_id.<namespace>/response and subscribes to the peer at <namespace>/request (previously bot0 / client0). --moq-bot-id / --moq-client-id still override them.
(PR #5158)Added an optional language key to the eval harness's built-in user.speech: and judge.transcription: blocks. Each built-in speech service builder (kokoro, cartesia, whisper, moonshine) now forwards language (a code like zh or a Language) into the service settings, so non-English audio evals can synthesize user turns and transcribe bot audio in the right language without the factory: escape hatch. Omitting language is unchanged; the TTS audio cache key now includes the language so English and non-English renders of the same text don't collide.
(PR #5171)
Added JobParams and JobGroupParams, which carry everything a job dispatch needs in one object: name, payload, timeout, plus cancel_on_error for groups and label / cancellable for how the work presents to a client UI. Pass one to job(...), job_group(...), request_job(...), or request_job_group(...):
job_id = await ui_jobs.request_job_group(
"wikipedia",
"news",
params=JobGroupParams(payload={"query": query}, label=f"Research:
{query}"),
)
(PR #5221)
Added BaseUIWorker, a worker that surfaces its jobs and job groups on the client UI without involving an LLM. Every group it dispatches streams its lifecycle to the client as the standard ui-job-group envelopes, with the client's reserved __cancel_job_group event honored for groups dispatched as cancellable. Dispatch from a plain BaseWorker when the work should stay invisible. It is instantiable directly, so an app can register one on the runner as a dispatcher and call it from a tool, and UIWorker now inherits from it, keeping the same capability for a page-driving LLM worker. BaseWorker itself is unchanged. The async-tasks example fans out research through a BaseUIWorker dispatcher driven by the main pipeline's own LLM tool, one LLM instead of two, while document-review keeps its UIWorker, which reads and drives the page content its review depends on.
(PR #5221)
Added WorkerRunner.get_worker(name), which returns a worker added to that runner, along with BaseWorker.worker_runner, FrameProcessor.worker_runner, and FunctionCallParams.worker_runner to reach the runner from inside a worker, a processor, or a tool handler. A tool that needs a peer worker can now find it by name rather than having the application pass the object in through app_resources:
async def research(params: FunctionCallParams, query: str):
ui_jobs = params.worker_runner.get_worker("ui-jobs")
Only workers on the same runner have a local instance to return; a worker on another runner is addressable over the bus but has no object to hand back. (PR #5221)
Added BaseWorker.request_cancel_job_group(job_id, reason=...), the door for cancellation asked for from outside the worker. It honors the request only for a group dispatched with JobGroupParams(cancellable=True) and returns whether it did, so a client UI, an operator endpoint, or anything else reaching in gets the same rule. Cancellation the worker decides on itself, on shutdown, on a timeout, or through cancel_on_error, still calls cancel_job_group() and is never refused.
(PR #5221)
Added an extra_headers argument to CartesiaSTTService, CartesiaTTSService and CartesiaHttpTTSService, matching CartesiaTurnsSTTService. The headers are sent with the websocket handshake (or with each synthesis request, for CartesiaHttpTTSService), so deployments can supply their own authentication or routing headers.
(PR #5223)
Added AudioVolumeTracker (pipecat.audio.volume), which measures the volume of an audio stream over a rolling 400ms window. Audio is fed in chunks of any size with update(audio, sample_rate) and read back from the volume property, which reads 0 until the window holds enough audio to be measurable. Measuring happens on read and is cached until more audio arrives, so callers that report volume less often than they receive audio pay only for the reads. VADAnalyzer and RTVIObserver both track volume through it.
(PR #5232)
AICFilter and AICQuailVADAnalyzer now close their ai-coustics session when the pipeline stops, instead of waiting for garbage collection.
(PR #5239)
Added PipelineWorker(processor_unusable_policy=...), deciding what the pipeline does when a processor reports an error that leaves it unable to do its job (becomes is_usable=False), such as a service whose API key was rejected.
ProcessorUnusablePolicy.CONTINUE (the default) keeps the pipeline running and leaves the decision to the application, while END and CANCEL stop it gracefully or immediately. It is applied once per processor, not once per failed request:
```python
worker = PipelineWorker(
pipeline,
processor_unusable_policy=ProcessorUnusablePolicy.END,
)
```
(PR #5242)
Added FrameProcessor.is_usable, reporting whether a processor can still do its job, so applications can tell one that's briefly struggling from one that will never work again until something changes.
A processor stays usable through failures it might recover from, and becomes unusable once its work can no longer succeed: a provider has rejected its API key, model or voice, or it has failed enough times to stop trying. Services stop accepting work and stop reconnecting once that happens, instead of retrying something that will keep failing.
Errors set it as they are reported, so an error handler reading frame.processor.is_usable always sees the verdict that came with the error it is handling. This works in a worker's on_pipeline_error handler, which sees every error in the pipeline:
```python
@worker.event_handler("on_pipeline_error")
async def on_pipeline_error(worker, frame):
if frame.processor and not frame.processor.is_usable:
logger.error(f"{frame.processor} can no longer do its job:
{frame.error}") ```
and equally in a single processor's own on_error handler, when only one service is of interest:
```python
@tts.event_handler("on_error")
async def on_error(processor, frame):
if not processor.is_usable:
logger.error(f"TTS can no longer do its job: {frame.error}")
```
Changes are also reported through on_usable_changed, a new event handler on the processor, which fires on the transition rather than on every error.
Bring a processor back with set_usable(True) once whatever stopped it working has been dealt with. Services do this for themselves whenever their settings change, since a new model or voice may be exactly the fix. Credentials aren't runtime settings, so a rejected API key needs either a new service or an explicit set_usable(True).
(PR #5242)
Added ErrorCategory, recording what kind of failure an error was — a rejected API key (AUTHENTICATION) versus a provider outage (SERVER), for example.
ErrorFrame carries it in a new category field, and FrameProcessor.push_error() accepts it as an argument:
```python
await self.push_error("rejected API key",
category=ErrorCategory.AUTHENTICATION) ```
The category says what went wrong, not what became of the processor. To decide whether a processor is worth using again, read processor.is_usable.
Every error reaching a handler carries a category; ErrorCategory.UNKNOWN means the cause couldn't be determined, which handlers can treat the way they treated every error before.
FrameProcessor.push_error() and push_error_frame() also take a force_treat_as_permanent argument, for an error that will keep recurring and so leaves the processor unable to do any more work. It's only needed for failures the category doesn't already convey, such as a websocket service exhausting its reconnection attempts; leaving it unset doesn't keep the processor usable, since a permanent category costs it its is_usable on its own.
A category is worked out from the exception only when the reporter left it unset, so an error is never mistaken for a verdict on a processor it didn't come from:
ErrorCategory.APPLICATION. A tool handler or TTS text transformer whose own API call returns 401 leaves the service usable, since its credentials were never in question.except, which may not have come from the processor at all, are reported as ErrorCategory.UNKNOWN.Classification falls back to the HTTP status code the exception carries. Processors whose provider signals failures through SDK-specific exceptions, or whose credentials can be rejected for reasons a reconnection would clear, refine it by overriding _classify_error():
```python
class MyService(TTSService):
def _classify_error(self, exception: Exception) -> ErrorCategory |
None: if isinstance(exception, MyProviderAuthError): return ErrorCategory.AUTHENTICATION return None ``` (PR #5242)
A scenario's judge: block accepts an extra: mapping, forwarded to the judge model as top-level request parameters. This is how provider-specific options reach the judge; the default judge uses reasoning_effort: none so that a thinking-capable model does not spend latency, or the token budget its verdict needs, on reasoning that is never read.
(PR #5243)
GoogleLLMService now logs a warning naming Gemini's finish_reason when a response ends for a notable reason — withheld for safety or recitation, a rejected tool call, or truncated at the output token limit. Previously these ended the turn with little or no text and no indication why. Whatever text did arrive is still passed downstream, and responses ending normally are unaffected.
(PR #5248)
GoogleLLMService now bounds how long it waits for a streamed response, via a new stream_idle_timeout_secs argument that defaults to 20 seconds. Previously a stream that stopped producing without closing left the turn open indefinitely, since the API client applies no timeout of its own. Reaching the timeout fires on_completion_timeout, pushes an ErrorFrame, and closes the response, so the pipeline continues with whatever text arrived. The timeout covers the gap between chunks rather than the response as a whole, leaving a slow but healthy stream free to take as long as it needs. Raise it for models configured to think at length, since thinking emits no chunks, or pass None to wait indefinitely.
(PR #5249)
Added FrameProcessor.pause_processing_all_frames_until(ready, timeout=...), which holds frames arriving at a processor until a condition resolves and then delivers them in order. Useful for a processor that establishes a connection in the background and cannot act on frames the moment it starts.
ready is anything awaitable, typically an asyncio.Event.wait the processor already owns, so each service decides what "ready" means. The pause takes hold from the frame after the one being processed, so a StartFrame that triggers it still travels on downstream and pipeline startup is not delayed. Both frame queues are held, so timeout bounds the wait and the pause is always lifted, at the latest during cleanup.
(PR #5254)
Added full client/server coverage for xAI Voice Agent item truncate/delete, force_message, idle-timeout / DTMF / MCP event hooks, and session fields (reasoning, resumption, replace, transcription, VAD idle timeout) on GrokRealtimeLLMService.
(PR #5255)
Added Speechify to the text-to-speech services offered by pipecat create, which scaffolds a bot wired to SpeechifyHttpTTSService and adds SPEECHIFY_API_KEY and SPEECHIFY_VOICE_ID to the generated project's env.example.
(PR #5259)
Added SpeechifyHttpTTSService, a Speechify text-to-speech service backed by the /v1/audio/stream/with-timestamps endpoint. Audio and word-level speech marks arrive together over Server-Sent Events, so bot speech is attributed to the conversation context word by word and an interruption commits only the portion actually spoken. Speech marks require a streaming-native model: the service defaults to simba-3.2 (English), and simba-3.0 covers the other supported languages.
```python
from pipecat.services.speechify.tts import SpeechifyHttpTTSService
tts = SpeechifyHttpTTSService(
api_key=os.environ["SPEECHIFY_API_KEY"],
aiohttp_session=session,
settings=SpeechifyHttpTTSService.Settings(voice="geffen_32"),
)
```
(PR #5259)
Every eval suite run writes a results.jsonl next to its logs, one line per run with its outcome, its failures, and paths to its artifacts, appended as each run finishes so an interrupted sweep keeps everything already done. Runs that didn't pass also carry events_seen, the record of what the bot actually did. Each failure carries a machine-readable kind (timeout, judge_no, missing_function_call, ...; see FAILURE_KINDS in pipecat.evals.harness), which groups failures across many runs in a way the judge's free-text reasons cannot.
(PR #5260)
An eval suite can run each (bot, scenario) pair several times, via repeat: in the manifest or --repeat N on pipecat eval suite, and reports a pass rate per pair instead of a single verdict. This is how a behavior with a race in it — interruptions, async function results, turn detection — gets measured rather than sampled, since a bot that passes half the time looks identical to a reliable one in a single pass. Attempts interleave across bots (A#1, B#1, C#1, A#2, ...) so every bot meets the same machine conditions in the same stretch of the sweep, and each attempt's number joins its artifact filenames so nothing is overwritten. A repeated sweep always exits 0: it reports a rate, and what rate is acceptable is the caller's policy.
(PR #5260)
Added retry_on_timeout and retry_timeout_secs to GoogleLLMService, matching the OpenAI, Anthropic, and AWS services. With retry_on_timeout set, a request whose first chunk doesn't arrive within retry_timeout_secs is issued once more, so a request the API accepts and then never answers costs a few seconds instead of the whole idle timeout. Only the first chunk is retried, since re-issuing after that would duplicate the response. Gemini's client sends the request lazily, when the first chunk is pulled, so the window spans the whole round trip including any thinking the model does before it emits anything — leave it off for models that think at length.
(PR #5262)
Added gemma4, glm5.2, and sarvam-105b-conversations model support to SarvamLLMService. gemma4 adds vision (inline data-URI image input), glm5.2 adds reasoning support, and sarvam-105b-conversations targets multi-turn conversation on the /v1 endpoint. The base URL is resolved automatically from the model (/v1 for sarvam-105b-conversations, /v2 for all others), and switching models at runtime recreates the client when the API version changes. Model-specific capabilities — vision, reasoning_effort, and wiki_grounding — are gated to the models that support them.
(PR #5288)
Added Icelandic, Sundanese, and Uzbek to the languages SonioxTTSService can speak.
(PR #5295)
DeepgramFluxTTSService now supports Flux's speed and expressivity voice controls, set via DeepgramFluxTTSService.Settings and updatable at runtime with a TTSUpdateSettingsFrame. A speed change is applied to the open connection with Flux's Configure message, so the cross-turn acoustic state survives it; expressivity is fixed when the connection opens, so a change reconnects. A settings update Deepgram rejects is reported as a non-fatal ErrorFrame.
(PR #5296)
Added LiveKit as a transport option in the development runner: python bot.py -t livekit, and POST /start support ("transport": "livekit"). Requires LIVEKIT_URL, LIVEKIT_API_KEY, and LIVEKIT_API_SECRET to be configured on the server.
(PR #5297)
Added SarvamRealtimeSTTService for low-latency streaming speech-to-text with Sarvam's saaras:v3-realtime model. Supports server-side endpointing (endpointing="vad") and pipeline-driven endpointing (endpointing="manual"), interim and final transcripts, timestamps, and in-band configuration updates via config.update.
(PR #5301)
Added cancellable_by_llm to @tool_options and register_function(), which lets the LLM stop a running async tool call whose result the user no longer wants.
A tool that opts in is advertised alongside its own cancel_<name>, which stops the one call of it that's running, and takes a tool_call_id only when several calls of that tool are running at once. A tool that doesn't opt in has no cancel tool and can't be stopped.
Only applies when the cancel_on_interruption=False @tool_options is set. Consider using for long-running tool calls that a user might want to cancel, such as a long report or a background job that keeps producing results. The work has to outlast the LLM's route to cancelling it.
```python
@tool_options(cancel_on_interruption=False, cancellable_by_llm=True)
async def write_report(params: FunctionCallParams, topic: str):
"""Write a long research report on a topic.
Args:
topic: What the report should cover.
"""
...
```
(PR #5304)
Added OpenClaw Gateway support in pipecat.services.openclaw, for driving an OpenClaw coding agent from a pipeline.
OpenClawGatewayService starts a run on an OpenClawSendFrame, redirects the one in flight on an OpenClawSteerFrame, and stops it on an OpenClawAbortFrame. A run answers with an OpenClawStartedFrame, any number of OpenClawTextFrames, and one OpenClawEndFrame saying whether it completed, was cancelled, or failed. OpenClawGatewayClient speaks the same protocol without a pipeline.
examples/multi-worker/openclaw-agent is a voice front end built on it.
(PR #5308)
Added the function_call_stopped scenario event to pipecat.evals, which reports a function call ending with the tool_call_id and a cancelled flag in its args. It takes the same calls: shape as function_call, so a scenario can assert how a call ended — telling work that was stopped from work that finished on its own, which a check on what the bot said about it cannot.
```yaml
- event: function_call_stopped
calls:
- name: write_report
args: { cancelled: true }
```
(PR #5314)
Added setup_timeout_secs and start_timeout_secs to PipelineWorker, both defaulting to 20 seconds. A processor that blocks while connecting, or while handling the StartFrame, would leave run() waiting on it forever; the pipeline is now torn down once the timeout elapses.
(PR #5316)
Added acquires and releases (pipecat.utils.shared) for a resource shared by several processors, such as the client an input and an output transport share. The first owner to acquire runs the decorated method while the rest wait for it, and only the last owner to release runs the undo. A method that raises is not attempted again: the exception reaches every owner, so the two halves of a transport either both come up or both fail.
```python
class MyTransportClient:
@acquires("client")
async def setup(self, setup: FrameProcessorSetup): ...
@releases("client")
async def cleanup(self): ...
```
(PR #5316)
Added BaseObserver.on_processor_setup, called with a ProcessorSetUp once each processor has been set up. Services connect during setup, so this is where that cost can be measured; processors are set up concurrently, so these arrive in the order they finish rather than in pipeline order.
(PR #5316)
Added on_setup_timeout and on_pipeline_timeout events to PipelineWorker, so a pipeline that gives up waiting says so. on_setup_timeout fires when the processors never finish setting up, and takes no frame, since none has been pushed yet. on_pipeline_timeout fires when a frame the worker was waiting on never reaches the end of the pipeline: a StartFrame that never starts it, or a CancelFrame that never drains it, so inspect the frame to tell the two apart.
@worker.event_handler("on_pipeline_timeout")
async def on_pipeline_timeout(worker, frame):
if isinstance(frame, StartFrame):
...
(PR #5316)
Added a stop_on_failure field to eval scenarios. It defaults to true, the existing behavior, where the first turn with a failed assertion ends the scenario. Set it to false for a scenario whose turns are scored independently — a benchmark reporting a per-turn pass rate needs every turn driven, not just the ones before the first miss.
(PR #5317)
Added TTFATMetricsData, reporting time to first answer token for LLM services that answer in text. It runs from the request to the first token the caller sees, excluding any reasoning streamed first, and carries ttfat, ttfb, and thinking_time (ttfat minus ttfb) so the cost of a model thinking is readable from one metric. It reaches RTVI clients as ttfat and is logged by MetricsLogObserver. Speech-to-speech services report nothing, having no answer token to measure to.
(PR #5320)
Added pcm_to_wav() to pipecat.audio.utils, which wraps raw 16-bit PCM in a WAV container and returns the file as bytes. It takes PCM in the forms Pipecat pipelines carry it — bytes, bytearray, or memoryview — so audio from an AudioBufferProcessor event handler can be written or uploaded directly.
(PR #5326)
Added AudioBufferProcessor events on_user_turn_audio and on_bot_turn_audio, which fire once when a turn ends with a TurnAudioData holding that speaker's audio for the whole turn and the turn number. Turn tracking, which the pipeline worker enables by default, supplies the boundary and the number.
(PR #5329)
Added per-turn results to pipecat.evals. EvalResult.turns holds an EvalTurnResult for each turn in the scenario — its status (passed, failed, or not_run), the failures it produced, and its duration — so scoring a run turn by turn no longer means grouping EvalResult.failures by turn_index and opening the scenario file for a denominator. A turn the run stopped before reaching reports not_run instead of looking like a pass. pipecat eval suite writes the same statuses to each run's results.jsonl line, and pipecat eval prints a 2/4 turns tally for a run that drove every turn and failed some.
(PR #5332)
Added ElevenLabsDialogueTTSService, a WebSocket TTS service for ElevenLabs Eleven v3 models (eleven_v3 and eleven_v3_conversational), which ElevenLabsTTSService can't reach. It needs workspace access to ElevenLabs' Text-to-Dialogue API. stability is the only voice setting Text-to-Dialogue reads, and text is always aggregated into sentences (TextAggregationMode.SENTENCES):
from pipecat.services.elevenlabs.dialogue.tts import
ElevenLabsDialogueTTSService
tts = ElevenLabsDialogueTTSService(
api_key=os.getenv("ELEVENLABS_API_KEY"),
settings=ElevenLabsDialogueTTSService.Settings(
voice=os.getenv("ELEVENLABS_VOICE_ID"),
model="eleven_v3_conversational",
),
)
Keep using ElevenLabsTTSService for Flash, Turbo, and Multilingual models, which have lower latency and a fuller set of voice controls.
(PR #5353)
Added a warning when a thinking_budget is set on a Gemini 3 model. Gemini 3 takes thinking_level instead. Passing thinking_budget results in ill-defined behavior (it may be honored, silently ignored, or rejected, depending on the model and the backend).
(PR #5356)
Added DeepgramFluxSageMakerTTSService, running Deepgram Flux TTS on a SageMaker endpoint. It takes endpoint_name and region instead of an API key and accepts the same settings as DeepgramFluxTTSService, including voice, speed and expressivity. Requires pipecat-ai[deepgram,sagemaker] and AWS credentials.
(PR #5360)
Added support for Azure's v1 API surface, which Microsoft Foundry displays as the "Azure OpenAI endpoint". AzureLLMService selects it whenever endpoint ends in /openai/v1, and AzureRealtimeLLMService reaches it from a base_url with no query string, appending the deployment named by Settings.model:
llm = AzureLLMService(
api_key=os.getenv("AZURE_CHATGPT_API_KEY"),
endpoint="https://my-resource.openai.azure.com/openai/v1",
settings=AzureLLMService.Settings(model="my-deployment"),
)
realtime = AzureRealtimeLLMService(
api_key=os.getenv("AZURE_REALTIME_API_KEY"),
base_url="wss://my-resource.openai.azure.com/openai/v1/realtime",
settings=AzureRealtimeLLMService.Settings(model="my-deployment"),
)
AzureLLMService still serves dated endpoints, routing them through api_version. For AzureRealtimeLLMService, v1 is the supported surface: endpoints carrying a dated api-version serve the superseded preview protocol, which rejects the session configuration and names its events differently, so they aren't usable here.
(PR #5363)
Added token_provider to AzureLLMService and AzureRealtimeLLMService for Microsoft Entra ID authentication, so Azure services can run without an API key. api_key is now optional, and passing neither credential raises ValueError:
from azure.identity.aio import DefaultAzureCredential,
get_bearer_token_provider
llm = AzureLLMService(
token_provider=get_bearer_token_provider(
DefaultAzureCredential(), "https://ai.azure.com/.default"
),
endpoint="https://my-resource.openai.azure.com/openai/v1",
)
(PR #5363)
Added --ice-servers to the development runner, along with the matching PIPECAT_ICE_SERVERS environment variable, so a bot started through pipecat.runner.run.main() can gather candidates from custom STUN and TURN servers:
python bot.py -t webrtc --ice-servers stun:stun.l.google.com:19302
An entry is a bare URL, or a JSON object with urls, username, and credential when a TURN server needs authentication. The environment variable takes the same entries comma-separated or as a JSON array. Configured servers also reach WebRTC clients in the iceConfig of the /start response, so both peers negotiate against the same servers.
(PR #5376)
Added saaras:v4 to the models supported by SarvamSTTService. It uses the same WebSocket contract as saaras:v3 — the same modes and fine-grained VAD tuning parameters — and adds Global English alongside Indian English and the 22 Indic languages.
(PR #5382)
Added client-side TLS options to the MoQ transport: client_tls_cert/client_tls_key present a certificate to a relay that authenticates its peers with mTLS, and client_tls_roots/client_tls_fingerprints verify a relay behind a private CA or a self-signed one. The latter two are alternatives to switching verify_ssl off, which was previously the only way to reach such a relay; a bot in serve mode already publishes its own fingerprints as MOQTransport.cert_fingerprints for a peer to pin.
(PR #5387)
Added BlandTTSService, realtime WebSocket text-to-speech using Bland, and BlandHttpTTSService for complete-text HTTP requests. Install with uv add "pipecat-ai[bland]".
(PR #5388)
Added max_consecutive_zero_audio_contexts to TTSService. A provider can accept every request and answer with silence — an unknown voice ID, say — without ever reporting an error, leaving the bot mute with nothing in the logs to explain it. Every TTS context that completes without producing audio reports an error the service can carry on from, so application code hears about a turn that produced no speech as it happens. After this many silent contexts in a row, the service reports a permanent error instead, stops being given work, and the pipeline worker applies its ProcessorUnusablePolicy (a ServiceSwitcher fails over to another provider). Defaults to 3; set it to 0 to report silent contexts without ever writing the service off.
(PR #5393)
Added PipelineWorker(handle_flush_frame=...), which says whether a worker answers a flush probe. It defaults to whether the pipeline is unbridged, so a bridged worker takes part in the trip but leaves the answering to the pipeline that owns the bridge, which is what makes flush_pipeline() on a bridged worker wait for what it produced to reach the end of that pipeline rather than only for its own queues to empty. A bridged worker with no such peer never completes a flush.
(PR #5399)
Added BusSubscriber.accepts_bus_message(message), which the bus consults before every delivery to decide whether to hand the message to that subscriber. Returning False drops it for that subscriber alone; others still receive it. It accepts everything by default.
(PR #5399)
pipecat eval run now accepts a directory of .yaml scenario files and executes them in deterministic filename order.
(PR #5414)
Added "max" to the reasoning effort levels OpenAIResponsesLLMService.ReasoningConfig accepts, matching OpenAI's current set for the Responses API.
(PR #5432)
Added complete_marker, incomplete_short_marker and incomplete_long_marker to UserTurnCompletionConfig, so a bot can choose markers that are a single token in its own model's tokenizer. The turn completion instructions and both incomplete-turn re-prompts are rendered from whichever markers are configured.
(PR #5437)
Added GeminiSTTService, a streaming speech-to-text service using Google's gemini-3.5-transcribe-live model over the Gemini Live API. Language is auto-detected by default; GeminiSTTService.Settings supports languages hints and adaptation_phrases to bias recognition toward domain-specific terms. Requires google-genai >= 2.9.0.
The model detects utterance boundaries itself, and when the pipeline's VAD signals end of speech the service flushes the utterance so the final transcript arrives promptly instead of when the model decides the utterance ended. (PR #5449)
⚠️ ExternalUserTurnStrategies, when driven by the new proposed turn frames, now pushes UserStartedSpeakingFrame / UserStoppedSpeakingFrame and broadcasts the interruption itself rather than leaving both to the service. Fed real turn frames it still emits nothing, so pipelines built around a shared UserTurnProcessor or a third-party service that emits turn frames directly are unaffected.
This matters if you pass should_interrupt=False to a turn-detecting STT and pin user_turn_strategies=ExternalUserTurnStrategies() by hand: the service carries should_interrupt on the strategies it recommends, but a user-supplied user_turn_strategies discards that recommendation, so interruptions come back on. Drop the manual user_turn_strategies — the service recommends the right strategies on its own now — or pass ExternalUserTurnStrategies(enable_interruptions=False). The aggregator logs a warning naming both fixes when it detects this.
Relatedly, a turn-detecting service paired with pinned non-external strategies (e.g. VAD or turn analyzer strategies) no longer drives turns at all; the pinned strategies own them. (PR #5156)
Renamed MOQParams.serve_bind to MOQParams.bind, which now also sets the local source address a client-mode bot dials from. MOQRunnerArguments.serve_bind is renamed to match. The old name still works and warns; it will be removed in 2.0.0.
(PR #5158)
MoonshineSTTService now resolves languages the way the other STT services do: a Language maps to one of Moonshine's eight languages (Arabic, Chinese, English, Japanese, Korean, Spanish, Ukrainian, Vietnamese) through language_to_moonshine_language(), with regional variants such as Language.ES_MX resolving to their base code. A language Moonshine publishes no model for raises with the list of supported languages, instead of failing inside the model download.
(PR #5182)
MoonshineSTTService reloads its model when language or model changes at runtime (including via set_language()). Previously the new value was stored but the loaded model kept transcribing in the old language. A failed reload keeps the loaded model and pushes an ErrorFrame.
(PR #5182)
OpenAI-compatible LLM services now report token usage once per completion. Providers that repeat a cumulative usage snapshot on every streamed chunk previously produced a token-usage MetricsFrame for each one, over-counting a single turn for anything aggregating those frames. SambaNovaLLMService also now reports the cache-read and reasoning token counts its provider sends.
(PR #5190)
GrokRealtimeLLMService's voice setting is typed str rather than a fixed list of five names, and accepts any built-in Grok voice ID (xAI documents the catalogue at https://docs.x.ai/docs/guides/voice/agent) or a custom ID from the Custom Voices API. Voice IDs are case-insensitive. GrokVoice is an alias of str.
(PR #5200)
Behavior change: the default Grok Realtime voice is now eve, the voice xAI documents as its default, instead of Ara. Anyone relying on the previous out-of-the-box voice should set voice="ara" explicitly on SessionProperties.
(PR #5200)
⚠️ Streaming STT services no longer report processing metrics — ProcessingMetricsData in MetricsFrame, surfaced as the processing field of RTVI's metrics message. Nothing changes for SegmentedSTTService subclasses.
Processing metrics time a discrete unit of work, and a streaming STT doesn't really perform one — audio arrives continuously. The 22 affected services' measurement methodologies were inconsistent and either not meaningful or duplicative of TTFB.
TTFB — speech end to final transcript — is the STT latency measure, and it is unaffected. (PR #5209)
⚠️ WebsocketTTSService subclasses and DeepgramSageMakerTTSService no longer report processing metrics, which were meaninglessly reporting zero on every turn. The metric is ProcessingMetricsData in MetricsFrame, surfaced as the processing field of RTVI's metrics message. TTS services whose processing time was a real number are unaffected.
Processing time is measured around run_tts. For a service that requests
audio and waits for it in that call, that covers the real work.
WebsocketTTSService subclasses instead push the text onto the socket and
return, leaving the audio to arrive on a separate receive task, so the
measurement only ever covered the send. DeepgramSageMakerTTSService does
the same over bidirectional HTTP/2. TTFB and TTFA measure the latency that
matters for all of them, and are unaffected.
There's a new TTSService.supports_processing_metrics property, which
defaults to True. Set it to False on a custom service whose run_tts
returns before synthesis finishes, or back to True on a
WebsocketTTSService subclass that waits for the server to signal the end.
(PR #5220)
Changed JobGroup.worker_names from a set to a list, preserving the order the workers were dispatched in so anything rendering them, such as a client UI job-group card, stays stable across a group's lifetime. JobGroup also now carries the group's label and cancellable settings and the set of workers that have reached a terminal state.
(PR #5221)
CartesiaSTTService's base_url now also accepts a URL carrying a scheme (ws://localhost:8000) rather than only a bare host, so the connection can be made over plain ws against a local or proxied endpoint instead of always wss.
(PR #5223)
CartesiaTTSService now authenticates with the X-API-Key and Cartesia-Version headers on the websocket handshake instead of api_key and cartesia_version query parameters, matching the Cartesia STT services.
(PR #5223)
⚠️ calculate_audio_volume() now requires at least 400ms of audio, the length of an ITU-R BS.1770 gating block, and raises ValueError for anything shorter. Code passing individual audio frames should use AudioVolumeTracker instead, which accumulates them into a rolling window. Volume is still reported on the same 0 to 1 scale, so VADParams.min_volume thresholds carry over unchanged.
VAD continues to run on 32ms frames; only the volume measurement spans a wider window. Because loudness is now integrated over 400ms rather than a single frame, brief dips between phonemes no longer drop the measured volume below min_volume mid-word.
(PR #5232)
GoogleLLMService now defaults to gemini-3.6-flash, up from gemini-2.5-flash. 2.5 Flash follows the async-tool result-reporting instruction unreliably, and its failure mode is announcing a fabricated result rather than staying silent. Set model in GoogleLLMService.Settings to pin the previous default.
(PR #5236)
AICQuailVADAnalyzer now uses vad-2.1-xxs-16khz by default. The old default, quail-vad-2.0-xxs-16khz, does not work with aic-sdk 3.0. If you set model_id yourself, pick a model listed at https://artifacts.ai-coustics.io/.
(PR #5239)
The aic extra now requires aic-sdk~=3.0, which reworked its audio and VAD APIs. Upgrade the SDK when you upgrade pipecat; aic-sdk 2.5.x no longer works.
(PR #5239)
ServiceSwitcher now fails over only on errors that leave a service unable to do its job (is_usable=False), and reports its services' failures as its own.
ServiceSwitcherStrategyFailover switches only once the active service reports an error that leaves it unable to do its job, rather than on any error, so a provider hiccup no longer costs a failover. It switches to the next service that is still usable, and a successful switch consumes the error: the switcher went on doing its job, so nothing upstream needs to act on it.
The rest of the pipeline deals with the switcher rather than with the services inside it, so what it does with an error depends on which service reported it:
on_usable_changed to hear about it.The switcher's is_usable is a reading of its services: it reports itself unusable only once none of them can work, so one service's rejected API key never writes off the switcher along with it. Bringing any service back with set_usable(True) brings the switcher back with it; calling that on the switcher itself does nothing, since it has no usability of its own to set. The switcher raises on_usable_changed for itself whenever that reading moves, so watching the switcher is enough to hear about the services inside it.
(PR #5242)
Websocket services now stop reconnecting once the service can no longer do its job, instead of retrying credentials the provider has already rejected.
Running out of reconnection attempts now leaves the service unusable too (is_usable=False), so a connection that can't be re-established is reported as such rather than being retried on every subsequent request. Errors reported during reconnection carry the exception that caused them, so they can be classified.
Giving up is reported through the report_error callback, which takes an optional force_treat_as_permanent argument alongside the error frame. A service that overrides _report_error should accept and forward it.
(PR #5242)
STT and TTS services now stop working once they can no longer do their job — a bad API key, an unknown model or voice, a connection that won't come back — instead of retrying for every chunk of audio or piece of text.
Previously a rejected API key on a service that connects on demand produced a connection attempt and an ErrorFrame several times a second for as long as the pipeline ran. STTService and TTSService now skip transcription and synthesis while the service is unusable.
Services whose credentials are signed or resolved per connection — AWSTranscribeSTTService and NvidiaSageMakerTTSService — treat a rejected credential as recoverable, since reconnecting is what refreshes it.
Pair this with PipelineWorker(processor_unusable_policy=...) or an on_pipeline_error handler to decide what the bot should do about it.
(PR #5242)
The eval judge now defaults to gemma4:12b with reasoning_effort: none, replacing gemma2:9b. Run ollama pull gemma4:12b before running scenarios that use the default judge. The previous default mistook a short interim reply for a complete answer — a bot that had so far said only "Let me check on that." would satisfy the criterion, passing a turn in which the bot said nothing. To keep the old judge, set it explicitly in a scenario's judge block: judge: {eval: {service: ollama, model: gemma2:9b}}.
(PR #5243)
Updated the default model for DeepSeekLLMService from deepseek-chat to deepseek-v4-flash. Set model in DeepSeekLLMService.Settings to pin the previous default.
(PR #5246)
Updated the default model for MiniMaxHttpTTSService from speech-02-turbo to speech-2.8-turbo. Set model in MiniMaxHttpTTSService.Settings to pin the previous default.
(PR #5246)
Updated the default model for FishAudioTTSService from s2-pro to s2.1-pro. Set model in FishAudioTTSService.Settings to pin the previous default.
(PR #5246)
Updated the default model for LmntTTSService from aurora to blizzard. Set model in LmntTTSService.Settings to pin the previous default.
(PR #5246)
Updated the default model for AsyncAITTSService and AsyncAIHttpTTSService from async_flash_v1.0 to async_flash_v1.5. Set model in AsyncAITTSService.Settings or AsyncAIHttpTTSService.Settings to pin the previous default.
(PR #5246)
SpeechTimeoutUserTurnStopStrategy no longer overrides the deprecated reset() hook. Turn detection through UserTurnController is unaffected; code that calls .reset() directly on this strategy now reaches the inherited no-op instead — call handle_user_turn_started() (turn start) or handle_user_turn_stopped() (turn stop) instead.
(PR #5252)
Changed the default model for GrokRealtimeLLMService to grok-voice-latest, xAI's recommended Voice Agent alias. Pin a versioned model explicitly (e.g. settings=GrokRealtimeLLMService.Settings(model="grok-voice-think-fast-1.0")) for stability.
(PR #5255)
AWS Nova Sonic's AudioConfig now requires an int for each of its sample-rate, sample-size, and channel-count fields, rejecting an explicit None at construction. Every field already defaults to a real value, and a None that reached session continuation — which sizes its audio buffer from them — raised a TypeError there instead.
(PR #5273)
⚠️ LLMSetToolsFrame.tools no longer lists a bare list of provider-specific tool dicts among the forms it accepts. That form last worked through OpenAILLMContext, which stored it verbatim, and stopped when that deprecated context was removed in 1.0.0. Provider-native tools travel in a ToolsSchema's custom_tools, keyed by adapter type — a form the frame already carries end to end, into a realtime service's session update included. Nothing changes at runtime: frames are dataclasses and don't validate.
(PR #5273)
The async function-calling examples no longer set enable_async_tool_cancellation=True, so they demonstrate async tools on their own. Cancellation asks the model to judge whether a pending result is still wanted, and a model that judges too readily cancels a result the user was waiting on and never mentions it — which is worth knowing before turning it on, and is now noted on the parameter itself.
(PR #5278)
⚠️ Removed Arcana model support from Rime TTS services before Rime's cloud cutoff on August 15, 2026 at 12:00 UTC. Set model="coda" when you upgrade. Rime examples use Luna for cross-model voice continuity.
(PR #5279)
⚠️ Changed SarvamLLMService default base_url from https://api.sarvam.ai/v1 to be resolved automatically from the selected model: https://api.sarvam.ai/v1 for sarvam-105b-conversations, https://api.sarvam.ai/v2 for all other models. Existing users who relied on the /v1 default with sarvam-105b must pass base_url="https://api.sarvam.ai/v1" explicitly or update to /v2.
(PR #5288)
FunctionCallCancelFrame carries a run_llm field, defaulting to False. LLMService sets it only when a call is cancelled by its own timeout — an interruption must not trigger inference, and a cancellation the LLM requested already runs inference through the result of the tool that requested it. LLMAssistantAggregator pushes the context upstream when the flag is set, holding off while sibling calls from the same LLM response are still in flight so the group still triggers inference exactly once.
(PR #5291)
⚠️ A function call that exceeds function_call_timeout_secs (or a per-tool timeout_secs) is now cancelled rather than left to run: its handler is thrown an asyncio.CancelledError so it can clean up, and the call settles through the path interruptions and LLM-requested cancellation already use — a FunctionCallCancelFrame and the on_function_calls_cancelled event — then runs inference so the bot reports that the call didn't complete. Previously the deadline reported an empty result while the handler kept running, so its side effects still landed and its real result was discarded. The deadline covers the handler's own execution; work it spawns into a task of its own is not cancelled with it.
(PR #5291)
⚠️ SonioxTTSService now defaults to Soniox's tts-rt-v2 model, with Bryce as the default voice. tts-rt-v2 speaks the same WebSocket API as tts-rt-v1 but offers a different roster of voices, so a voice set explicitly must be one tts-rt-v2 offers. Soniox removes tts-rt-v1 on August 31, 2026, after which requests naming it route to tts-rt-v2 regardless.
(PR #5295)
DeepgramFluxTTSService now cancels the active turn with Flux's Interrupt message instead of reconnecting the websocket, so the cross-turn acoustic state that keeps a voice consistent survives a barge-in.
Because an interruption no longer closes the connection, on_connected and on_disconnected stop firing on every barge-in.
(PR #5296)
Changed how a function_call expectation matches arguments in pipecat.evals. A turn expecting args: now passes if any call of that name matches them, where before it checked only the first call sharing the name and failed there. An LLM that gets a call wrong and immediately repeats it correctly now satisfies the turn, and when nothing matches, the failure names the arguments that did arrive.
(PR #5314)
A processor holds every frame it receives until its StartFrame arrives. A service that connects during setup can push frames before the pipeline starts; those frames now wait and are delivered after the StartFrame, in arrival order, so a processor never acts on a frame before it has started.
(PR #5316)
StartupTimingObserver measures a startup that now happens mostly before the StartFrame, so its report covers setting up as well as starting.
ProcessorStartupTiming.duration_secs is what a processor cost to get ready, its setup() and start() together, so it keeps reporting the same magnitude now that connecting has moved into setup(). The new setup_duration_secs breaks out the connecting part.StartupTimingReport.total_duration_secs is the span from the pipeline starting to set up until it had started, rather than the sum of what each processor cost. Processors are set up concurrently, so a sum would report a pipeline as slower the more of its work overlapped.TransportTimingReport.bot_connected_secs and client_connected_secs run from the pipeline starting to set up, so they measure the real time to a connected bot. A transport that connected before the StartFrame was pushed previously went unreported.
(PR #5316)The pipeline clock now runs from the moment the pipeline starts setting up rather than from the StartFrame, so frames pushed while processors connect are no longer timestamped zero. Presentation timestamps therefore start at roughly what setting up cost; everything comparing them does so relatively, so pacing and playback are unaffected.
(PR #5316)
Pipeline configuration reaches processors through FrameProcessorSetup in setup() rather than through StartFrame. setup.audio_in_sample_rate, setup.audio_out_sample_rate, setup.enable_metrics, setup.enable_tracing, setup.enable_usage_metrics, setup.report_only_initial_ttfb and setup.tracing_context are available from setup() onwards, which is what lets a custom processor connect or resolve sample rates there.
(PR #5316)
GroqLLMService now defaults to openai/gpt-oss-120b. The previous default, llama-3.3-70b-versatile, is being retired by Groq. Pass settings=GroqLLMService.Settings(model=...) to choose a different model.
(PR #5338)
⚠️ UltravoxRealtimeLLMService and VonageVideoConnectorTransport no longer cancel the pipeline when their connection fails. They now report the failure as one that leaves the service unusable, so the pipeline follows the processor_unusable_policy its PipelineWorker was given — by default it keeps running and the application decides what to do. Pass processor_unusable_policy=ProcessorUnusablePolicy.CANCEL to keep the previous behavior.
(PR #5348)
⚠️ GoogleVertexLLMService now defaults to gemini-3.6-flash, and its default location changed from us-east4 to global, because Vertex serves the Gemini 3 series only from the global endpoint. Pass settings=GoogleVertexLLMService.Settings(model="gemini-2.5-flash") and location="us-east4" to keep the previous configuration.
(PR #5356)
DeepgramFluxSTTBase moved to pipecat.services.deepgram.flux.stt_base.
(PR #5360)
OpenAI Realtime sessions now use gpt-realtime-2.1 as the default model.
(PR #5362)
AzureLLMService now routes endpoints outside the v1 API surface through 2025-04-01-preview, the last dated version Azure issued, so recent Azure features are available without naming a version.
(PR #5363)
The cli extra now requires Pipecat Context Hub 0.5.3 or newer, so a plain pipecat-ai[cli] install can run pipecat context-hub refresh --framework-version latest — the refresh the agent guides written by pipecat init prescribe. It pins the index to the newest released pipecat-ai tag instead of main, and re-resolves on every run, so a later incremental refresh picks up a new release without --force.
(PR #5367)
Updated the MoQ transport to moq-rs 0.4. Broadcasts are now created on an origin rather than constructed standalone, the track subscriptions (subscribe_catalog, subscribe_audio, subscribe_json_stream) are awaited, and MoqError is Error. The publish broadcast and transcript track are still created synchronously in __init__, so the bot loses no startup audio.
Fixed the MoQ transport reporting a normal peer hangup as an error. A peer that vanishes mid-call drops its producer without finishing, which moq-rs raises with the reason as the message tail rather than as a reset code, so the hangup classifier missed it and the disconnect surfaced through on_error with a traceback.
(PR #5378)
SarvamSTTService now uses saaras:v4 as its default model instead of saaras:v3. Applications that relied on the previous default should set settings=SarvamSTTService.Settings(model="saaras:v3") explicitly.
(PR #5382)
A TTS context that completes without producing any audio now resumes frame processing as soon as that is known, instead of leaving it paused until the pause watchdog fires a few seconds later. The non-fatal ErrorFrame the watchdog reports no longer accompanies these silent turns.
(PR #5393)
TTSService with pause_frame_processing=True now pauses only while there is audio to wait for: the bot speaking, or an audio context still open that may yet produce audio. Previously a turn that produced no audio could stall the pipeline for a few seconds until a watchdog force-resumed it and reported a non-fatal error.
(PR #5394)
The example bots set processor_unusable_policy=ProcessorUnusablePolicy.END, so an example ends once one of its processors can no longer do its job — a rejected API key or an unknown model, say — instead of running on with a service that will keep failing.
(PR #5397)
Changed PipelineWorker.end() and PipelineWorker.activate_worker() to wait for in-flight frames before they go through, so a closing line is heard rather than cut off and a worker handing over stops talking before the one taking over starts. Previously only LLMWorker arranged this. A pipeline that never started, or one that has already finished, is left alone. Cancelling still takes effect immediately.
(PR #5399)
Changed WorkerRunner to send each worker one shutdown message instead of two. cancel() now signals shutdown and the messages go out as the runner exits, carrying the reason the caller gave rather than a generic one, and addressed only to workers that have not already finished.
(PR #5399)
Changed BaseWorker(active=...) so that it governs whether a worker accepts bus messages at all. It previously gated only the frames a bridged worker received, leaving job requests, UI events and every other kind of bus traffic to arrive whatever the worker's state. An inactive worker is now handed only activation, deactivation, end or cancel messages. Nothing else reaches it, so no on_bus_message override or on_bus_message event handler runs for it either, which includes a BaseUIWorker no longer honouring the client's __cancel_job_group while inactive. @worker_ready handlers are unaffected, since they fire from the WorkerRegistry rather than over the bus.
(PR #5399)
Changed PipelineWorker.flush_pipeline() to wait for as long as the pipeline keeps working. Its timeout now counts seconds without progress rather than seconds in total, so a long turn keeps the wait alive while a stuck pipeline still gives up promptly. Progress is a frame reaching the sink, or a report from the pipeline answering the probe when it crossed into another worker. Heartbeats are not counted. A caller that gives up now says what it did: settled a function call before its output was delivered, or handed over without draining.
(PR #5399)
KrispVivaSDKManager now keeps the Krisp VIVA SDK initialized for the life of the process: release() no longer calls krisp_audio.globalDestroy(), and is_initialized() stays True after the last reference is released. Native sessions are still released per component, so per-call memory is unchanged. One consequence is that api_key is read only by the call that initializes the SDK, so a process serving sessions under different Krisp licenses uses the first one for all of them.
(PR #5411)
The moonshine extra now requires moonshine-voice>=0.1.5, up from >=0.0.62. Existing installs need uv sync (or pip install -U "pipecat-ai[moonshine]") to pick the new version up.
(PR #5422)
Updated the runner extra to require pipecat-ai-prebuilt>=1.0.6, refreshing the prebuilt client UI served by the development runner with @pipecat-ai/client-react 1.8.2, @pipecat-ai/moq-transport 0.1.1, and @pipecat-ai/voice-ui-kit 0.13.1.
(PR #5427)
AnthropicLLMService.ThinkingConfig now covers Anthropic's current thinking API: type="adaptive", the mode Claude 4.7 and later models require, and display, which asks for summarized thinking text on models that omit it by default.
(PR #5429)
A session now tears its controllers and its input audio filter down once, instead of once from the EndFrame or CancelFrame handler and again from cleanup().
(PR #5434)
VADController and UserTurnController gained a start(), called by their owner, and they and UserIdleController gained a stop(). BaseAudioFilter.start() is now called from the input transport's setup() rather than on StartFrame.
(PR #5434)
⚠️ Changed the user turn completion markers to a fill gradient: ● marks a complete turn (previously ✓), ◐ a turn cut off mid-thought (previously ○), and ○ a user who needs more time (previously ◐). The two incomplete markers have swapped meaning, so a custom UserTurnCompletionConfig.instructions string or a model fine-tuned on the old markers now maps short and long waits the wrong way round; set complete_marker, incomplete_short_marker and incomplete_long_marker on UserTurnCompletionConfig to keep the previous characters. Every marker is now a single token in every major tokenizer, and since the complete marker is generated before any speakable text, this removes up to two decode steps from the bot's first spoken word.
(PR #5437)
Changed PipelineWorker.flush_pipeline() to also wait for work the pipeline starts by pushing upstream, such as the LLM run a function call result triggers. The probe used to turn around at the source and settle there, returning before that response had been generated, let alone rendered; it now travels down, up, and down again, settling on the second arrival at the sink.
(PR #5438)
Changed PipelineWorker.activate_worker() to drain the pipeline only when deactivate_self is set. A worker that stays active is handing nothing over, so there is nothing in flight to wait for, and waiting meant the first activation of a session blocked on the very worker it was about to wake.
(PR #5438)
pipecat init now scaffolds Cartesia TTS with a voice recommended for sonic-3.5, the service's default model. Examples use the same voice.
(PR #5441)
GradiumSTTService now defaults language to Language.EN instead of leaving it unset. Grounding the model to a language improves transcription accuracy. Set settings=GradiumSTTService.Settings(language="any") to have Gradium detect the language instead.
(PR #5444)
AnthropicLLMService now disables thinking by default on Sonnet 5 and later, where adaptive thinking is otherwise on and the model decides per request whether to think, to keep latency low for real-time voice — mirroring how the Gemini service disables thinking by default on Flash models. Opus and Fable are left at Anthropic's default. Set Settings.thinking to configure thinking explicitly.
(PR #5446)
CerebrasLLMService now sends "developer"-role messages unchanged instead of converting them to "user" messages. Cerebras maps the role to its developer instruction layer, which sits above user instructions in the prompt hierarchy.
(PR #5448)
MoondreamService now defaults revision to 2025-06-21 instead of 2025-01-09, picking up the newer Moondream build. Pass revision="2025-01-09" to stay on the previous one.
(PR #5458)
Deprecated MCPClient methods register_tools(), register_tools_schema(), and get_tools_schema(). Use MCPClient.tools() instead.
(PR #4939)
Deprecated the enable_user_speaking_frames constructor parameter on BaseUserTurnStartStrategy and BaseUserTurnStopStrategy, which will be removed in 2.0.0. Whether a turn is announced is a per-turn decision rather than a per-strategy setting: pass enable_user_speaking_frames to trigger_user_turn_started() / trigger_user_turn_stopped() where the strategy decides the turn. Passing it to a constructor still applies and now emits a DeprecationWarning.
ExternalUserTurnStartStrategy and ExternalUserTurnStopStrategy suppress emission on their own whenever the turn was already announced elsewhere — by a shared UserTurnProcessor, or by a service that emits turn frames rather than proposing them — so a pipeline built on those strategies doesn't need to set the flag anywhere.
(PR #5156)
Deprecated UIWorker.ui_job_group(), UIWorker.start_ui_job_group(), and UIJobGroupContext (all removed in 2.0.0): use job_group(...) / request_job_group(...) / JobGroupContext instead, since every group a BaseUIWorker dispatches is client-visible. The deprecated wrappers keep their historical signatures and behavior in the meantime.
(PR #5221)
Deprecated passing name, payload, timeout, and cancel_on_error directly to BaseWorker.job(), job_group(), request_job(), request_job_group(), and create_job_group_and_request_job() (removed in 2.0.0). Pass params=JobParams(...) or params=JobGroupParams(...) instead. The individual arguments keep working in the meantime, and passing both raises TypeError.
(PR #5221)
Deprecated the cartesia_version parameter of CartesiaTTSService and CartesiaHttpTTSService. Both services send the Cartesia-Version header they are written against, since their request payloads and response handling are tied to that version. Passing cartesia_version warns and still overrides the header until it is removed in 2.0.0.
(PR #5231)
Deprecated enable_async_tool_cancellation on LLM services; it will be removed in 2.0.0. Set cancellable_by_llm=True on the tools that should be cancellable instead. It still works meanwhile, treating every async tool as cancellable — which is worth moving off, because a model that wrongly decides a pending result is unwanted destroys work the user asked for, and a tool that never opted in can't have that happen to it.
The flag's shape has changed with it: where it used to advertise a single generic cancel tool, it now advertises a cancel_<name> for every async tool, so the tool set a model sees grows with the number of async tools registered.
```python
# Before
llm = OpenAILLMService(api_key=..., enable_async_tool_cancellation=True)
@tool_options(cancel_on_interruption=False)
async def write_report(params: FunctionCallParams, topic: str): ...
# After
llm = OpenAILLMService(api_key=...)
@tool_options(cancel_on_interruption=False, cancellable_by_llm=True)
async def write_report(params: FunctionCallParams, topic: str): ...
```
(PR #5304)
Deprecated StartFrame.audio_in_sample_rate, StartFrame.audio_out_sample_rate, StartFrame.enable_metrics, StartFrame.enable_tracing, StartFrame.enable_usage_metrics, StartFrame.report_only_initial_ttfb and StartFrame.tracing_context, which will be removed in 2.0.0. Read the same values from FrameProcessorSetup in setup() instead. The fields still carry the pipeline's configuration, so a processor that reads one keeps working and emits a DeprecationWarning, once per call site.
(PR #5316)
AudioBufferProcessor's on_user_turn_audio_data and on_bot_turn_audio_data are deprecated and will be removed in 2.0.0. They report a run of speech at a time, so one turn produces several and none carries a turn number. Use on_user_turn_audio and on_bot_turn_audio instead.
(PR #5329)
Deprecated "fatal" errors. Concretely, deprecated 3 things: ErrorFrame.fatal, the fatal argument of FrameProcessor.push_error(), and FatalErrorFrame, all of which will be removed in 2.0.0. A fatal error would cancel the pipeline outright; that's now an application decision. Passing fatal=True still cancels the pipeline, but now also emits a DeprecationWarning. There are two alternatives to fatal errors, depending on what your error means:
The error leaves its originating processor unable to do any more work: report it with push_error(..., force_treat_as_permanent=True). That marks the processor unusable, and PipelineWorker applies its processor_unusable_policy to specify how to handle the resulting error.
The error isn't about any processor's state, but the pipeline should stop anyway: push a regular ErrorFrame (without fatal) and follow it with an EndWorkerFrame, which ends the pipeline after queued frames drain. Use CancelWorkerFrame instead to abandon the queued frames, as fatal=True did.
(PR #5348)
Deprecated the api_version constructor parameter on AzureLLMService, which will be removed in 2.0.0. Point endpoint at Azure's v1 API surface instead, by ending it in /openai/v1. Azure issued no dated version after 2025-04-01-preview, and new features reach only the v1 surface. Endpoints outside that surface still route through 2025-04-01-preview; passing api_version explicitly still applies and now emits a DeprecationWarning.
(PR #5363)
Deprecated the pause_watchdog_timeout_s parameter of TTSService, which will be removed in 2.0.0. Passing it warns and does nothing: a pause is now only taken while audio is playing or still on its way, so it is always lifted by the BotStoppedSpeakingFrame that follows playback or by the audio context completing in silence — no timer is needed to break it.
Note truncated.
Deprecated XTTSService. Constructing one now emits a DeprecationWarning; it will be removed in 2.0.0. The Coqui XTTS streaming server it connects to h…
Added token usage metrics to AWSNovaSonicLLMService, which now emits LLMUsageMetricsData from Nova Sonic's usageEvent. Each event reports its token delta, with speech and text tokens combined into prompt_tokens and completion_tokens, so metrics summed over a session match the session total.
(PR #4783)
Added an http_client parameter to OpenAITTSService and Whisper-based STT services (BaseWhisperSTTService, OpenAISTTService, GroqSTTService), so a custom httpx.AsyncClient — e.g. one with a raised request timeout for high-latency endpoints — can be used for API requests. Prefer openai.DefaultAsyncHttpxClient, which retains the OpenAI SDK's connection limits and redirect handling.
(PR #4941)
Added the ability for users to provide a local image to the LemonSlice transport to be used as the avatar image. (PR #4977)
Added a filter_background_audio setting to ElevenLabsRealtimeSTTService.Settings so callers can have ElevenLabs suppress background and far-end audio before transcription, which reduces spurious partial transcripts and empty commits on noisy telephony audio. When set, it is forwarded as a connection query parameter regardless of commit strategy. Defaults to unset, which preserves ElevenLabs' default of no filtering.
(PR #5003)
Added STT usage metrics: every STT service reports usage as STTUsageMetricsData, carrying the client-measured seconds of audio submitted to the service (audio_seconds). Continuous services emit incrementally per final transcript with a flush on stop/cancel; segmented services emit per transcribed segment. Enabled with enable_usage_metrics=True; usage is forwarded to RTVI clients (stt_usage), logged by MetricsLogObserver, and attached to OpenTelemetry stt spans as metrics.audio_seconds.
(PR #5055)
Added inbound SIP DTMF support on the LiveKit transport via on_dtmf_event and InputDTMFFrame, so DTMFAggregator works with LiveKit SIP/PSTN calls.
(PR #5097)
Added numerals setting to DeepgramFluxSTTSettings to convert spoken numbers to numeral form in transcripts (e.g. "twenty three" → "23"). Enable with numerals=True in the settings; configured at connection time per Deepgram's Flux API.
(PR #5099)
Added safety_settings to GoogleLLMService.Settings (and GoogleVertexLLMService.Settings), exposing Gemini's content safety filters. Previously these could only be set by smuggling them through extra.
```python
from google.genai.types import HarmBlockThreshold, HarmCategory,
SafetySetting
llm = GoogleLLMService(
api_key=os.getenv("GOOGLE_API_KEY"),
settings=GoogleLLMService.Settings(
safety_settings=[
SafetySetting(
category=HarmCategory.HARM_CATEGORY_HATE_SPEECH,
threshold=HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
),
],
),
)
```
Categories left unspecified keep the Gemini API defaults. The setting is
runtime-updatable via LLMUpdateSettingsFrame, which also accepts plain
dicts.
(PR #5109)
Added language_codes setting to AssemblyAISTTSettings, exposing the list-valued name of AssemblyAI's declared-language parameter and the one to prefer over the singular language_code, which stays supported and is ignored when both are set. It takes Language enums: a single language (e.g. language_codes=[Language.ES]) pins transcription to that language, while several (e.g. language_codes=[Language.EN, Language.ES]) steer toward that subset while keeping code-switching among them. Steering is prompt-based, so it applies to U3 Pro models only and isn't sent forother models; on U3 Pro a change applies to the live connection instead of reconnecting, and an empty list clears steering back to the model default. At most 10 distinct languages can be declared — regional variants resolve to their base code, so they collapse rather than counting twice. An over-long list raises at construction; an over-long runtime update is dropped with a warning, leaving the current steering in place.
(PR #5129)
Added endpointing, keywords, and format to SmallestSTTService.Settings. endpointing finalizes transcripts promptly on trailing silence, keywords boosts recognition of domain-specific words/phrases (e.g. "Blackwell:2,NVIDIA:1"), and format controls whether transcripts getpunctuation and capitalization applied.
(PR #5137)
Added an opt-in force_locale setting to AzureTTSService and AzureHttpTTSService (AzureTTSSettings) that wraps synthesized text in SSML's <lang xml:lang> element, so multilingual voices (e.g. en-US-EmmaMultilingualNeural) speak in the configured locale/accent instead of auto-detecting one per segment. Defaults to False, leaving SSML output untouched unless enabled.
(PR #5154)
Added reach_inactive_services to ServiceUpdateSettingsFrame, and so to LLMUpdateSettingsFrame, TTSUpdateSettingsFrame and STTUpdateSettingsFrame. Set it when a settings update is provider-neutral and needs to survive a service switch, and every service a ServiceSwitcher manages applies it rather than the active one alone.
```python
update = LLMUpdateSettingsFrame(
delta=LLMSettings(temperature=0.2),
reach_inactive_services=True,
)
await worker.queue_frames([update])
```
It defaults to `False`, which suits values only one provider understands:
a Cartesia voice id applied to a Deepgram TTS service would leave it unusable
once it takes over. To configure one specific service, address the update to
it with service= instead.
(PR #5155)
Added keyterm to CartesiaSTTService.Settings and CartesiaTurnsSTTService.Settings, biasing transcription toward domain-specific words and phrases such as product names and jargon.
```python
stt = CartesiaTurnsSTTService(
api_key=os.environ["CARTESIA_API_KEY"],
settings=CartesiaTurnsSTTService.Settings(keyterm=["Pipecat", "Ink
2"]), ) ```
Cartesia binds keyterms to a connection, so updating them with an
STTUpdateSettingsFrame reconnects to apply them. Lists longer than
Cartesia's limit of 100 keyterms or 1200 total characters are truncated with
a warning. CartesiaSTTService sends keyterms only for ink-2 models, the
only family Cartesia supports them on.
(PR #5168)
Added PocketTTSService, a local CPU-only text-to-speech service built on kyutai-labs' pocket-tts streaming model. Supports English, French, German, Italian, Portuguese, and Spanish, predefined voices, and voice cloning from a wav file or hf:// voice prompt. Install with pip install "pipecat-ai[pocket-tts]".
```python
from pipecat.services.pocket_tts.tts import PocketTTSService
tts = PocketTTSService(settings=PocketTTSService.Settings(voice="alba"))
```
(PR #5170)
SmallestSTTService now uses the Waves v4 STT endpoint (/waves/v1/stt/live) and sends finalize per utterance to keep the WebSocket session alive across turns.
(PR #4747)
The TeXML served by the development runner for Twilio and Telnyx no longer includes a trailing <Pause length="40"/>. <Connect><Stream> holds the call for the duration of the WebSocket session, so the call now hangs up as soon as the stream ends instead of lingering for another 40 seconds.
(PR #5135)
SageMakerBidiClient now relies on the SageMaker Runtime SDK's default auth configuration (SigV4 for the sagemaker service) instead of passing an equivalent explicit configuration, so future SDK auth changes are picked up automatically.
(PR #5144)
⚠️ TavusParams.audio_out_faster_than_realtime now defaults to True. Bot audio is accumulated into 100ms chunks and sent to Tavus as fast as it is produced, giving the avatar a larger rendering buffer, instead of being paced to real playback time. Pipelines that need bot audio to arrive at roughly real time — for example when an AudioBufferProcessor downstream is recording the conversation — should now set audio_out_faster_than_realtime=False explicitly.
(PR #5162)
Websocket-based services now bound the websocket closing handshake at 2 seconds instead of the websockets default of 10. A service disconnects while handling the EndFrame, before the frame continues downstream, so a peer that never acknowledges the close delayed pipeline shutdown bythat much per service. The bound is configurable per service via the new ws_close_timeout argument on WebsocketService.
(PR #5164)
CartesiaSTTService now connects with Cartesia-Version: 2026-03-01, the version that supports keyterm and returns structured JSON errors. This brings it in line with CartesiaTTSService and CartesiaTurnsSTTService, which already use that version.
(PR #5168)
SarvamLLMService now defaults to sarvam-105b, the only chat model Sarvam's API still serves. sarvam-30b, sarvam-30b-16k and sarvam-105b-32k are rejected server-side, so they're no longer accepted as the model setting.
(PR #5177)
BasetenLLMService now defaults to moonshotai/Kimi-K2.6. Set settings=BasetenLLMService.Settings(model=...) to pin a different model.
(PR #5178)
Deprecated XTTSService. Constructing one now emits a DeprecationWarning; it will be removed in 2.0.0. The Coqui XTTS streaming server it connects to has been unmaintained since February 2024 and pins a commit of the discontinued coqui-ai/TTS library, and the XTTS-v2 model it serves is licensed for non-commercial use only.
- KokoroTTSService and PiperTTSService are the maintained local TTS services. To stay on XTTS, the community-maintained pipecat-xtts-vllm package targets a current XTTSv2 serving stack.
(PR #5131)
Deprecated LLMSettings.filter_incomplete_user_turns and LLMSettings.user_turn_completion_config, which will be removed in 2.0.0. Turn completion is configured through the user turn strategy: pass user_turn_strategies=FilterIncompleteUserTurnStrategies() to LLMUserAggregatorParams, with FilterIncompleteUserTurnStrategies(config=...) for a custom UserTurnCompletionConfig. Setting either field at construction now emits a DeprecationWarning; it never took effect there.
(PR #5157)
Fixed cached DTMF audio being returned at the wrong sample rate. load_dtmf_audio() keyed its in-memory cache by button only, so the first sample rate a button was loaded at was reused for every later request. The cache is now keyed by (button, sample_rate).
(PR #4766)
Fixed an issue where sending several STTUpdateSettingsFrames in quick succession to DeepgramFluxSTTService (or the SageMaker transport) could be rejected by Deepgram Flux with Configure rejected: [unknown] Too many pending Configure messages.. Configure messages are now sent one at a time, waiting for each to be acknowledged (bounded by a 5s timeout) before the next goes out, so bursts of mid-stream settings updates no longer trip the server-side pending-Configure cap.
(PR #4791)
Fixed AWSBedrockLLMAdapter raising UnboundLocalError when a user message's content list places the image before the text (e.g. a UserImageRawFrame followed by a prompt).
(PR #4796)
Fixed FunctionCallUserMuteStrategy raising KeyError on a FunctionCallResultFrame / FunctionCallCancelFrame whose tool call id it was not tracking, which aborted the user aggregator's handling of that frame and surfaced an ErrorFrame to the application. Such frames arrive when the LLM invokes the built-in cancel_async_tool_call (excluded from FunctionCallsStartedFrame, but it still emits a result), when an async tool reports intermediate updates (a result frame per update, plus the final one), and when a bus bridge re-delivers a result another worker alreadyhandled. Untracked tool call ids are now ignored.
(PR #4824)
Fixed AWSBedrockLLMService dropping all but the last tool call in streaming responses. Parallel tool calls arrive as separate Bedrock content blocks, but the accumulator overwrote the prior block on each contentBlockStart, so only the final call ran. Tool blocks are now keyed by contentBlockIndex and finalized on contentBlockStop, so every requested function call is dispatched.
(PR #4918)
Fixed expand_currency (used by VoiceFormatter) leaking the 3rd and later fractional digits of a currency amount onto the subunit word. $5.500 expanded to five dollars and fifty cents0 and $3.567 to three dollars and fifty-six cents7, so TTS spoke the stray digit. The full fraction is now consumed and read to cent precision (e.g. five dollars and fifty cents), dropping sub-cent precision like a two-digit amount.
(PR #4969)
Fixed TurnAnalyzerUserTurnStopStrategy releasing user turns later than the configured STT ttfs_p99_latency budget. The safety-net timeout is now computed from an absolute deadline — the end of the user's speech plus ttfs_p99_latency — so time spent in end-of-turn analysis (e.g. MLturn-analyzer inference, typically 50–120ms) no longer extends the wait.
(PR #5007)
Fixed TurnAnalyzerUserTurnStopStrategy waiting out the STT safety-net timer when a transcript finalized while end-of-turn analysis was still running: the trigger conditions are re-checked as soon as the analyzer's verdict lands, releasing the turn immediately. With wait_for_transcript=False, the turn is likewise released directly on the analyzer's COMPLETE verdict.
(PR #5007)
Fixed TaskManager.current_tasks() undercounting live tasks, which left the dangling-task warnings logged at shutdown incomplete. The task registry was keyed by task name, and names are not unique — tasks started from the same method on the same object, such as the parallel function-call tasks, all share one name and evicted one another. The registry is now keyed by the task object.
(PR #5025)
Fixed services that send the system instruction as a separate parameter (e.g. AnthropicLLMService, AWSBedrockLLMService, GoogleLLMService) permanently rewriting a lone system message in the LLMContext to user. When a context's only message is a system message, it's convertedto user so the provider isn't sent an empty conversation history; that conversion now happens on a copy, so the context keeps its system role and later turns — once more messages have accumulated — still send it as the system instruction.
(PR #5027)
Fixed developer-role messages being permanently rewritten to user in the LLMContext by OpenAI-compatible services that don't support the role (e.g. QwenLLMService, OllamaLLMService, TogetherLLMService). These services convert developer to user on the way out to the provider; that conversion now happens on a copy, so the stored conversation history keeps its original roles for logging, persistence, and any other service reading the same context.
(PR #5027)
Fixed GeminiLiveLLMService crashing with WebSocket error 1007 when seeding a session from a context containing tool calls (e.g. after a multi-agent handoff or a reconnect). Gemini Live only accepts text and media content, so the new GeminiLiveLLMAdapter summarizes function calls andtheir responses as text.
⚠️ GeminiLiveLLMService now uses GeminiLiveLLMAdapter rather than
GeminiLLMAdapter, so any hand-crafted LLMSpecificMessages aimed at it
need llm="gemini-live" instead of llm="google". We recommend you use
LLMService.create_llm_specific_message(), which picks the right value for
you.
(PR #5028)
Fixed pipecat init omitting the video avatar provider's extra from the generated project's pipecat-ai dependency. Choosing Tavus, HeyGen, or Simli produced a bot.py importing that service while pyproject.toml requested only the transport and cascade extras, so the scaffolded botfailed to import after a fresh uv sync.
(PR #5031)
Fixed FilterIncompleteUserTurnStrategies going silent after a tool call when the LLM spoke a ✓ acknowledgement (e.g. "Let me look that up, one moment.") in the same response as the call. The per-turn one-spoken-completion guard — which stops the acoustic detector's duplicate inferences from voicing the same turn twice — also swallowed the post-tool response. A function call now resets that guard, so the post-tool answer is spoken and the acknowledgement still plays in full.
(PR #5063)
Fixed AssemblyAISTTService raising a ValueError at construction when prompt and keyterms_prompt were set together on U3 Pro models (u3-rt-pro, universal-3-5-pro). AssemblyAI's server accepts the combination on these models, so both parameters are now sent on the streaming connect URL; the client-side mutual-exclusivity check still applies to older models.
(PR #5084)
Fixed word-timestamp and streamed-token TTS providers producing incorrect or duplicated text/transcript output when two audio contexts were in flight at the same time (e.g. back-to-back TTSSpeakFrame utterances on a websocket TTS service). Previously the AggregatedFrameSequencer tracked pending-sentence and force-complete state globally, so a word event or force-complete for one context could bleed into another; state is now tracked per context. In streaming (token) mode, sentence-boundary frames were also being emitted twice per sentence; only one is now emitted.
(PR #5098)
Fixed streamed-token TTS providers with no word-timestamp support (e.g. DeepgramFluxTTSService in its default TextAggregationMode.TOKEN configuration) tracking each LLM token as its own spoken unit instead of grouping tokens into sentences, and never sending the RTVI "new segment" bot-output event. Previously the AggregatedFrameSequencer only grouped streamed tokens into sentences when the TTS service also supported word timestamps; providers without word timestamps now get the same per-sentence grouping, completing each sentence's slot as soon as it is confirmed (there is no word-timestamp signal to complete it progressively).
(PR #5102)
Fixed ElevenLabsTTSService pushing a spurious ErrorFrame (which could e.g. trigger an unwanted ServiceSwitcherStrategyFailover switch) when the server closed the websocket first during an intentional disconnect.
(PR #5103)
Fixed TTS services with pause_frame_processing=True (e.g. ElevenLabs, Deepgram, Azure, Rime, Inworld) permanently hanging the pipeline when the expected resume signal never arrived — for example, a provider reporting a successful completion with zero audio (a quota-exhausted key), or a resume racing ahead of the pause taking effect. The stuck pause blocked the terminal EndFrame from draining, which in turn blocked pipeline teardown, PipelineRunner.run()/WorkerRunner returning, and any cleanup that runs afterward. TTSService now starts a watchdog whenever it pauses frame processing and force-resumes (reporting a non-fatal ErrorFrame) if no BotStartedSpeakingFrame confirms audio within the new pause_watchdog_timeout_s parameter (default 3.0 seconds).
(PR #5106)
Fixed LLMFullResponseEndFrame being silently dropped for every turn after the first when two turns' audio contexts were in flight concurrently. The end frame is now looked up per audio context instead of being gated by a single shared "response started" flag, which only ever fired forwhichever turn's audio happened to finish first.
(PR #5108)
Fixed LLMFullResponseStartFrame/LLMFullResponseEndFrame arriving out of order at downstream processors (e.g. LLMAssistantAggregator) when a new LLM turn started while the previous turn's audio was still playing — reachable on TTS services that don't pause frame processing during synthesis (e.g. CartesiaTTSService), where a second, overlapping turn is possible. LLMFullResponseStartFrame is now routed through the same serialization queue as the rest of a turn's frames, so it can no longer be pushed ahead of an earlier turn's still-draining audio context.
(PR #5108)
Fixed SmallWebRTCConnection's outgoing app-message buffer never being flushed for data channels created by the remote peer. aiortc surfaces such channels already open, so the "open" event the flush was wired to never fired and buffered messages were stuck until disconnect; the buffer now also flushes when a channel arrives already open.
(PR #5112)
Fixed SmallWebRTCTransport silently dropping app messages sent before the peer connection was established — e.g. the RTVI user-mute-started emitted by MuteUntilFirstBotCompleteUserMuteStrategy at pipeline start, which left clients that connect slower than the pipeline starts (realnetworks; loopback usually wins the race) never showing the greeting-hold mute. Messages sent before the data channel is open are now buffered and delivered, in order, once it opens; messages sent while the connection is closing are discarded with a debug log.
(PR #5112)
Fixed DeepgramSTTService retrying forever with no error notification when the WebSocket connection kept failing (e.g. an invalid API key). A 4xx error from Deepgram now stops the retry loop immediately, and any other connection failure now reports a push_error on every attempt and gives up after 3 consecutive failures within 5 seconds of connecting, backing off exponentially between attempts.
(PR #5113)
Fixed SmallWebRTCTransport input audio ignoring a configured audio_in_channels=2: the resampler was hardcoded to downmix to mono regardless of the setting, so frames were labeled as stereo while containing mono data. The resampler now targets the configured channel layout.
(PR #5115)
Fixed SmallWebRTCTransport input audio being read as half-speed garbage when audio_in_sample_rate matches the wire rate (48000): stereo frames bypassed the mono-downmixing resampler and their interleaved bytes were labeled mono.
(PR #5115)
Fixed a Pipecat CLI issue where one broken plugin took down the whole CLI. Plugins are imported on every invocation, so a bad install (missing transitive dependency, version conflict, stale wheel) made every command — pipecat init included — exit with a message saying the cli extra wasn't installed, which was not the problem. Plugins now load in isolation: a failure is reported on stderr and skipped, and the rest of the CLI carries on. A plugin that isn't a typer.Typer is rejected the same way instead of failing later with an opaque AttributeError.
(PR #5121)
Fixed a Pipecat CLI issue where the hint for enabling an optional sub-CLI silently uninstalled other plugins. uv tool install --with replaces the tool environment rather than adding to it, so a hint naming only the missing plugin dropped every other one. The hint now repeats every installed plugin package.
(PR #5121)
Fixed SmallWebRTCTransport PATCH /api/offer requests crashing with an AssertionError when the browser sends an RFC 8840 end-of-candidates marker (an empty candidate string) for a media line, as Firefox does on every connection. The marker is now forwarded to aiortc as None instead of being parsed as a regular candidate.
(PR #5126)
Fixed word-by-word context attribution breaking after punctuation for TTS services that report word timestamps (e.g. InworldTTSService), which duplicated the affected sentence in the assistant context. When a provider's word-timestamp events don't repeat punctuation attached to the previous word — reporting "Yeah" then "I" for "Yeah, I can do that." — every word after the punctuation failed to match its slot, was emitted as an untracked passthrough frame without its original-text attribution, and the unmatched remainder was re-emitted when the audio context ended.
(PR #5128)
Fixed punctuation being duplicated in the assistant context when a TTS service reports it with the following word (", I") rather than the preceding one ("Yeah,"), which produced "Yeah, , I can do that.". Punctuation trailing a word is attributed to that word, so the repeat is now dropped from the following word instead of discarding that word's original-text attribution.
(PR #5128)
Fixed AzureTTSService and AzureHttpTTSService crashing at startup when Settings(language=None) was used: the synthesis language is no longer assigned to the Azure SDK (which rejects None), letting the service default apply.
(PR #5144)
Fixed DailyTransportClient.add_custom_video_track() crashing with an AttributeError: the video track factory was invoked without await, so the SDK received a coroutine instead of a track. Joining with an already-released client now reports a join error instead of raising a TypeError.
(PR #5144)
Fixed an InvalidSsmlException error in AWSPollyTTSService when the text to synthesize contained XML-reserved characters such as & or <. The text is now escaped before being embedded in the SSML request.
(PR #5144)
Fixed NvidiaSegmentedSTTService failing with gRPC NOT_FOUND against NVIDIA's hosted endpoint: the default NVCF function_id for canary-1b-asr now points at NVIDIA's current deployment.
(PR #5144)
Fixed responses to LLMMessagesAppendFrame(..., run_llm=True) being silently dropped when filter_incomplete_user_turns is enabled and the bot had already responded in the current user turn. The response was generated but never spoken, with nothing logged — most visibly breaking user-idle re-prompts, where an escalation ladder ended the session with an unspoken goodbye.
(PR #5146)
Fixed ServiceSwitcher (and therefore LLMSwitcher) delivering ServiceUpdateSettingsFrame only to the active service, so an update addressed to a specific service — LLMUpdateSettingsFrame(service=...), TTSUpdateSettingsFrame(service=...) — never reached its target unless that target happened to be active. An addressed update now reaches its service whether or not it is the active one. Updates with no target still apply to the active service alone, since settings values are often specific to one provider; mark one reach_inactive_services to send it to every service the switcher manages.
(PR #5155)
Fixed LLMTurnCompletionUserTurnStopStrategy configuring only the LLM active at pipeline start when its LLMs sit behind an LLMSwitcher. Once a switch made another LLM active, that LLM emitted none of the turn-completion markers the strategy waits on, and every user turn was closed bythe user_turn_stop_timeout watchdog instead. The startup update now reaches every LLM the switcher manages.
(PR #5155)
Fixed TurnAnalyzerUserTurnStopStrategy ending the user turn on every finalized transcript when the turn was started by a transcript rather than by a VAD frame — the case for MinWordsUserTurnStartStrategy, and for TranscriptionUserTurnStartStrategy in the default strategy set. One utterance split by the STT endpointer into several finalized transcripts produced an LLM inference per fragment, each landing as its own role: "user" message, so the bot replied to partial phrases.
(PR #5159)
Fixed total_tokens omitting cached input tokens on the Anthropic and AWS Bedrock LLM services. Both report their input count net of the prompt cache, so cache reads and cache writes were left out of the total — a large undercount for a voice agent, where the cached prefix comes to dominate the input within a few turns. total_tokens is now the gross figure on both, matching what the OpenAI and Google services already reported, and on Bedrock a usage report carrying only cache activity is no longer skipped. The per-field breakdown is unchanged; read prompt_tokens forthe net input figure.
(PR #5163)
Fixed DTMFAggregator emitting flushed TranscriptionFrames with finalized=False, which stalled user-turn stop strategies waiting for a final transcription.
(PR #5172)
Fixed UserIdleController ignoring UserIdleTimeoutUpdateFrame until the next re-arm: a running idle timer now restarts with the new duration, and enabling a positive timeout while waiting for the user to speak arms the timer immediately.
(PR #5173)
Fixed TTSService leaving frame processing permanently paused when a text filter strips the text to empty and pause_frame_processing is enabled.
(PR #5174)
Fixed NovitaLLMService emitting a token-usage MetricsFrame (and a debug log line) for every streamed chunk instead of one per completion, which over-counted a single turn dozens of times over for anything aggregating those metrics. Novita reports a cumulative usage snapshot on each chunk; the service now collapses them into one total reported when the completion finishes, including when it is interrupted mid-stream.
(PR #5180)
Fixed PerplexityLLMService and NvidiaLLMService discarding the cache-read and reasoning token counts their providers report. Both now pass the whole usage snapshot through, so cache_read_input_tokens and reasoning_tokens reach the token-usage MetricsFrame alongside the prompt and completion counts.
(PR #5180)
Fixed a KeyError: 'role' crash in AnthropicLLMService when using a model that thinks by default, such as Claude Sonnet 5 or Claude Opus 5. The first tool call failed, and so did every later inference on that context. These models return thinking blocks whose text is empty and whose signature carries the reasoning; such blocks are now preserved and passed back to Anthropic, as required within a tool-use turn. A thought carrying no signature has no valid thinking-block form and is skipped.
(PR #5183)
One column per month.