livekit-agents

Patch Changes

• update to livekit python 1.0 - 32e129ff1a4c3d28f363f4f2b2a355e29c8fe64d (@davidzhao)

Patch Changes

• fix decoder: if no data was pushed, close the output channel - #1881 (@jayeshp19)

Patch Changes

• fixed thread safety in AudioStreamDecoder - #1736 (@jeradf)

• cleanup AudioStreamDecoder resources - #1736 (@jeradf)

Patch Changes

• Remove unnecessary version pins - #1682 (@hauntsaninja)

• reduced retry interval to 2s - #1701 (@davidzhao)

Patch Changes

• use streaming AudioDecoder to handle compressed encoding - #1584 (@davidzhao)

• added a tts.prewarm method to start the connection pool early. - #1587 (@davidzhao)

• Raise ValueError in FallbackAdapter when streaming is not supported - #1609 (@jayeshp19)

• fixed a bug in AudioStreamDecoder where it could fail on close - #1587 (@davidzhao)

• support for livekit noise cancellation plugin in VoicePipelineAgent and MultimodalAgent - #1551 (@bcherry)

• fix: _play_speech get stuck due to orphan speech handle - #1555 (@SiyuanQi)

Patch Changes

• feat: connection pooling. speeds up generation with STT/TTS providers - #1538 (@davidzhao)

• handle process initialization failure - #1556 (@theomonnom)

Patch Changes

• Revert "fix(cli): update main_file path to use current directory" - #1509 (@theomonnom)

Patch Changes

• openai tts: switch to using Opus encoding - #1494 (@davidzhao)

• improve exception logging - #1490 (@jayeshp19)

• fix interrupting nested speech from before_llm_cb - #1504 (@longcw)

• add cache tokens in CompletionUsage dataclass - #1478 (@jayeshp19)

Patch Changes

• Allow shutdown callbacks to take reason - #1475 (@martin-purplefish)

Patch Changes

• fix agent transcription could not be disabled - #1448 (@davidzhao)

• Added an additional field in LLM capabilities class to check if model providers support function call history within chat context without needing function definitions. - #1441 (@jayeshp19)

• support agent.say inside the before_llm_cb - #1460 (@longcw)

Patch Changes

• gemini-realtime: fix input audio sample rate - #1411 (@jayeshp19)

• fix(pipeline_agent): clear user transcript when before_llm_cb returns false - #1423 (@s-hamdananwar)

• fix: fallbackadapter to correctly handle function calls - #1429 (@davidzhao)

• improved TTFB metrics for streaming TTS - #1431 (@davidzhao)

Patch Changes

• fix false positive interruption tripping up certain LLMs - #1410 (@davidzhao)

• fix: ensure llm.FallbackAdapter executes function calls - #1409 (@davidzhao)

Patch Changes

• add generate_reply api for multimodal agent - #1359 (@longcw)

• remove aiodns from livekit-agents - #1368 (@theomonnom)

Patch Changes

• Fix not awaiting forward task in TTS forwarder, leading to warnings. - #1339 (@martin-purplefish)

• reduces initial delay before model retries - #1337 (@davidzhao)

• fix the function calls without a text response are not added to chat ctx - #1349 (@longcw)

• add timeout for EOU inference requests made to the inference process - #1315 (@theomonnom)

• support disabling server VAD for OpenAI realtime model - #1347 (@longcw)

Patch Changes

• ensure job status updates contain the correct status - #1319 (@davidzhao)

Patch Changes

• expose worker_id in jobcontext - #1307 (@s-hamdananwar)

• improved handling of LLM errors, do not retry if already began - #1298 (@davidzhao)

• Do not pass function context if at max depth - #1306 (@martin-purplefish)

• avoid warnings when function depth matches limit - #1316 (@davidzhao)

• improve interruption handling, avoid agent from getting stuck - #1290 (@davidzhao)

• add manual interrupt method for pipeline agent - #1294 (@longcw)

• make multimodal class generic and support gemini live api - #1240 (@jayeshp19)

Patch Changes

• make max_endpoint_delay configurable - #1277 (@davidzhao)

• set USE_DOCSTRING as default for ai_callable - #1266 (@longcw)

• fix: do not log process warning when process not found - #1281 (@davidzhao)

• fix context when functions have been called - #1279 (@jmugicagonz)

Patch Changes

• avoid duplicated chat ctx for function calls with messages - #1254 (@longcw)

Patch Changes

• Moved create_ai_function_info to function_context.py for better reusability and reduce repetation - #1260 (@jayeshp19)

• added streaming audio decoder for compressed audio. - #1236 (@davidzhao)

• Add JPEG quality param to image encoder - #1249 (@bcherry)

• Add support for OpenAI's "detail" parameter to ChatImage - #1213 (@bcherry)

  Add support for data URLs on ChatImage in the Anthropic plugin.

• fix: correctly parse function argument types - #1221 (@jayeshp19)

• Fix center_aspect_fit bug, add scale_aspect_fit and scale_aspect_fill resizing options. - #1222 (@bcherry)

  Make scale_aspect_fit the new default resizing option for video frames.

Patch Changes

• improvements to endpointing latency - #1212 (@davidzhao)

• Improvements to end of turn plugin, ensure STT language settings. - #1195 (@davidzhao)

• fix duplicated agent speech commit for message with function call - #1192 (@longcw)

• fix: Handle optional func args in tool calls when set to None - #1211 (@jayeshp19)

Patch Changes

• fix release - #1176 (@theomonnom)

Minor Changes

• add nested speech handles, now agent.say works during a function call - #1130 (@longcw)

Patch Changes

• feat: stt retry & stt.FallbackAdapter - #1114 (@theomonnom)

• expose LiveKitAPI from the a JobContext - #1159 (@theomonnom)

• add extra chat messages to the end of the function call outputs - #1165 (@longcw)

• Add retries to recover from text mode to audio model for realtime API - #1121 (@longcw)

• prepare for release - #1160 (@theomonnom)

• add max_job_memory_usage and will kill the job if it exceeds the limit - #1136 (@longcw)

• support for custom tool use in LLMs - #1102 (@jayeshp19)

• feat: tts retry & tts.FallbackAdapter - #1074 (@theomonnom)

• Expose multimodal agent metrics - #1080 (@longcw)

• preload mp3 decoder for TTS plugins - #1129 (@jayeshp19)

• feat: llm retry & llm.FallbackAdapter - #1132 (@theomonnom)

• feat: inference process & end of utterance plugin - #1133 (@theomonnom)

• vertex ai support with openai library - #1084 (@jayeshp19)

Patch Changes

• add PeriodicCollector utility for metrics - #1094 (@davidzhao)

Patch Changes

• Fix interrupt_min_words handling - #1062 (@davidzhao)

• pipelineagent: fix speech_committed never called - #1078 (@theomonnom)

• Allow setting agent attributes when accepting job - #1076 (@davidzhao)

• handles error in function calls - #1057 (@jayeshp19)

• Include job count in WorkerStatus and pass in worker for load_fnc - #1046 (@keepingitneil)

• Fix delay calculation - #1081 (@martin-purplefish)

• sync the Realtime API converstation items and add set_chat_ctx - #1015 (@longcw)

• added metrics for idle time - #1064 (@jayeshp19)

Patch Changes

• Fix stack dump on closed stream - #1023 (@martin-purplefish)

• fix: invalid request on anthropic - #1018 (@theomonnom)

• fix: IndexError on tts metrics - #1028 (@theomonnom)

Minor Changes

• prepare for release - #1007 (@theomonnom)

Patch Changes

• Fix race in load calc initialization - #969 (@martin-purplefish)

• Fix incorrect load computation on docker instances - #972 (@martin-purplefish)

• stt: reduce bandwidth usage by reducing sample_rate to 16khz - #920 (@theomonnom)

• Reorganized metrics, added create_metrics_logger - #1009 (@davidzhao)

• pipelineagent: expose timing metrics & api errors wip - #957 (@theomonnom)

• Allow kind to be list or single value - #1006 (@keepingitneil)

• fix before_llm_cb not handling coroutines returning False - #961 (@Tanesan)

• expose transcriptions for multimodal agents - #1001 (@longcw)

• Fix stack dump on room shutdown - #989 (@martin-purplefish)

• Add exception logging for tool calls - #923 (@martin-purplefish)

• Skip egress by default in participant-related utilities on JobContext - #1005 (@keepingitneil)

• pipeline-agent: avoid nested function calls - #935 (@theomonnom)

• expose usage metrics - #984 (@theomonnom)

• fix jobs never reloading - #934 (@theomonnom)

• voicepipeline: support recursive/chained function calls - #970 (@theomonnom)

Patch Changes

• Fix split_paragraphs and simple-rag example - #896 (@davidzhao)

• Fix bug where if the tts_source was a string but before_tts_cb returned AsyncIterable[str], the transcript would not be synthesized. - #906 (@martin-purplefish)

• Allow forcing interruptions of incomplete audio - #891 (@martin-purplefish)

• Include chat context on collected tool calls - #897 (@martin-purplefish)

Patch Changes

• use rtc.combine_audio_frames - #841 (@theomonnom)

• Fix agent state to not change to listening when user speaks - #857 (@martin-purplefish) Fixed canceling uncancelable speech Fixed bug where agent would get stuck with uninterruptable speech.

• Fix bug where empty audio would cause agent to get stuck. - #836 (@martin-purplefish)

• fix: handle when STT does not return any speech - #854 (@davidzhao)

• Fix watcher reloaded processes double connecting to rooms - #822 (@keepingitneil)

• voice-pipeline: avoid stacked replies when interruptions is disallowed - #869 (@theomonnom)

• disable preemptive_synthesis by default - #867 (@theomonnom)

• Fixed bug where agent would get stuck on non-interruptable speech - #850 (@martin-purplefish)

• use EventEmitter from rtc - #879 (@theomonnom)

• AudioByteStream: avoid empty frames on flush - #840 (@theomonnom)

• improve worker logs - #878 (@theomonnom)

• voice-pipeline: fix tts_forwarder not always being closed - #871 (@theomonnom)

• bump livekit-rtc to v0.17.5 - #880 (@theomonnom)

• Fixed bug where agent would freeze if before_llm_cb returned false - #865 (@martin-purplefish)