NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI · #610 most downloaded on PyPI
Datadog APM client library
Last release 4 days ago
23 Sep 2026
Ships on a steady schedule
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
5 versions withdrawn
withdrawn after publishing
10 years old
851 releases · first in 2017
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
BaseEvaluator class, providing more flexibility and structure for implementing evaluation logic. The EvaluatorContext stores the context of the evaluation, including the dataset record and span information. Additionally, class-based summary evaluators are supported via BaseSummaryEvaluator, which receives a SummaryEvaluatorContext containing aggregated inputs, outputs, expected outputs, and per-row evaluation results.DD_TRACE_LOG_LEVEL to control the ddtrace logger level, following the log levels available in the logging module.pathlib.Path.open() for App and API Protection Exploit Prevention.DD_TRACE_TORNADO_ENABLED=true or DD_PATCH_MODULES=tornado:trueFallbackStreamWrapper (introduced for mid-stream fallback support) that caused an AttributeError when attempting to access the .handler attribute. The integration now gracefully handles both the original response format and wrapped responses by falling back to ddtrace's own stream wrapping when needed.asyncio task stacks could contain duplicated frames when the task was on-CPU is now fixed. The stack now correctly shows each frame only once.gevent.joinall is called.StreamedRunResult.stream_responses() method which was introduced in pydantic-ai==0.8.1. This was leading to agent spans not being finished.LLMObs.experiment was overly constrained due to the use of an invariant List type. The argument now uses the covariant Sequence type, allowing users to pass in a list of evaluator functions with narrower return type.One column per quarter.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
BaseEvaluator class, providing more flexibility and structure for implementing evaluation logic. The EvaluatorContext stores the context of the evaluation, including the dataset record and span information. Additionally, class-based summary evaluators are supported via BaseSummaryEvaluator, which receives a SummaryEvaluatorContext containing aggregated inputs, outputs, expected outputs, and per-row evaluation results.DD_TRACE_LOG_LEVEL to control the ddtrace logger level, following the log levels available in the logging module.FallbackStreamWrapper (introduced for mid-stream fallback support) that caused an AttributeError when attempting to access the .handler attribute. The integration now gracefully handles both the original response format and wrapped responses by falling back to ddtrace's own stream wrapping when needed.asyncio task stacks could contain duplicated frames when the task was on-CPU is now fixed. The stack now correctly shows each frame only once.gevent.joinall is called.StreamedRunResult.stream_responses() method which was introduced in pydantic-ai==0.8.1. This was leading to agent spans not being finished.LLMObs.experiment was overly constrained due to the use of an invariant List type. The argument now uses the covariant Sequence type, allowing users to pass in a list of evaluator functions with narrower return type.Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
BaseEvaluator class, providing more flexibility and structure for implementing evaluation logic. The EvaluatorContext stores the context of the evaluation, including the dataset record and span information. Additionally, class-based summary evaluators are supported via BaseSummaryEvaluator, which receives a SummaryEvaluatorContext containing aggregated inputs, outputs, expected outputs, and per-row evaluation results.FallbackStreamWrapper (introduced for mid-stream fallback support) that caused an AttributeError when attempting to access the .handler attribute. The integration now gracefully handles both the original response format and wrapped responses by falling back to ddtrace's own stream wrapping when needed.asyncio task stacks could contain duplicated frames when the task was on-CPU is now fixed. The stack now correctly shows each frame only once.StreamedRunResult.stream_responses() method which was introduced in pydantic-ai==0.8.1. This was leading to agent spans not being finished.Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
gevent.joinall is called.Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Tracer.data_streams_processor is deprecated and will be removed in v5.0.0. Use ddtrace.data_streams.data_streams_processor() instead.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
⚠️ An issue was detected with Memory Profiling in this release. ⚠️ An issue was detected with Datastreams Monitoring in this release.
httpx library from 0.17.1 to 0.25.0. If you are using httpx versions prior to 0.25.0, upgrade to httpx>=0.25.0 to ensure compatibility with ddtrace.Tracer.data_streams_processor is deprecated and will be removed in v5.0.0. Use ddtrace.data_streams.data_streams_processor() instead.threading.Condition locking type profiling in Python. The Lock profiler now provides visibility into threading.Condition usage, helping identify contention in multi-threaded applications using condition variables.EvaluatorResult class that evaluator functions can return to provide additional fields beyond just the evaluation value. This enables evaluators to include reasoning, assessment, metadata, and tags alongside the evaluation result.RuntimeError caused by capturing the __dict__ attribute of an object that mutates during iteration.PicklingError when using multiprocessing.Manager() with lock profiling enabled on Python 3.14+. Python 3.14 changed the default multiprocessing start method from fork to forkserver, which requires all objects to be pickleable. The lock profiler's wrappers now properly support pickle serialization.tracer=None.Tracer.data_streams_processor is deprecated and will be removed in v5.0.0. Use ddtrace.data_streams.data_streams_processor() instead.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
httpx library from 0.17.1 to 0.25.0. If you are using httpx versions prior to 0.25.0, upgrade to httpx>=0.25.0 to ensure compatibility with ddtrace.Tracer.data_streams_processor is deprecated and will be removed in v5.0.0. Use ddtrace.data_streams.data_streams_processor() instead.threading.Condition locking type profiling in Python. The Lock profiler now provides visibility into threading.Condition usage, helping identify contention in multi-threaded applications using condition variables.EvaluatorResult class that evaluator functions can return to provide additional fields beyond just the evaluation value. This enables evaluators to include reasoning, assessment, metadata, and tags alongside the evaluation result.RuntimeError caused by capturing the __dict__ attribute of an object that mutates during iteration.PicklingError when using multiprocessing.Manager() with lock profiling enabled on Python 3.14+. Python 3.14 changed the default multiprocessing start method from fork to forkserver, which requires all objects to be pickleable. The lock profiler's wrappers now properly support pickle serialization.tracer=None.Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
FallbackStreamWrapper (introduced for mid-stream fallback support) that caused an AttributeError when attempting to access the .handler attribute. The integration now gracefully handles both the original response format and wrapped responses by falling back to ddtrace's own stream wrapping when needed.asyncio task stacks could contain duplicated frames when the task was on-CPU is now fixed. The stack now correctly shows each frame only once.Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Bug Fixes
The Hooks class (config. .hooks) is deprecated and will be removed in v5.0. All hook methods (register(), on(), deregister(), emit()) are now no-op an…
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Hooks class (config.<integration>.hooks) is deprecated and will be removed in v5.0. All hook methods (register(), on(), deregister(), emit()) are now no-op and no longer affect span behavior. To interact with spans, use ddtrace.trace_utils.get_current_span() or ddtrace.trace_utils.get_current_root_span() instead.initialize requests and their responses on modelcontextprotocol/python-sdk servers.source:otel tagging for evaluations when OpenTelemetry (OTel) tracing is enabled when DD_TRACE_OTEL_ENABLED=true is set. This tag allows the backend to wait for OTel span conversion before processing evaluations.DD_MCP_CAPTURE_INTENT environment variable.asyncio.BoundedSemaphore lock type profiling in Python Lock Profiler.asyncio.Condition locking type profiling in Python. The Lock profiler now provides visibility into asyncio.Condition usage, helping identify contention in async applications using condition variables.asyncio.Semaphore lock type profiling in Python Lock Profiler.asyncio.shield.asyncio.TaskGroup.aws.httpapi span name for v2 apis when the API Gateway sets the x-dd-proxy header to aws-httpapi. Additionally, the tag http.route and the resource name of the span now contains the api resource path instead of the path when propagated with the x-dd-proxy-resource-path header.DD_TRACE_REMOVE_INTEGRATION_SERVICE_NAMES_ENABLED support, which was previously ignored.https:// scheme prefix as part of the http.url tag; this caused the entire url to be parsed as the http path.isDefined to result in an evaluation error.DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED=true), ensuring remote configuration is received before process forking occurs.None metadata in Ray job submission caused a crash.AttributeError when calling tag_agent_manifest.annotation_context blocks caused annotations to fail after the first operation in subsequent contexts. Previously, the trace context created by the first annotation_context remained active after exiting, causing the second context to reuse a stale context ID. This resulted in annotations not being applied to spans after the first batch call in the second annotation_context block.activate_distributed_headers() where distributed requests missing a LLM Observability trace ID would be incorrectly propagated twice.TypeError during profiling. One example of this is neo4j's AsyncRLock, which inherits from asyncio.Lock: https://github.com/neo4j/neo4j-python-driver/blob/6.x/src/neo4j/_async_compat/concurrency.py#L45uvloop and forking has been resolved._psutil_linux.abi3.so file in an injected environment.The Hooks class (config. .hooks) is deprecated and will be removed in v5.0. All hook methods (register(), on(), deregister(), emit()) are now no-op an…
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Hooks class (config.<integration>.hooks) is deprecated and will be removed in v5.0. All hook methods (register(), on(), deregister(), emit()) are now no-op and no longer affect span behavior. To interact with spans, use ddtrace.trace_utils.get_current_span() or ddtrace.trace_utils.get_current_root_span() instead.initialize requests and their responses on modelcontextprotocol/python-sdk servers.source:otel tagging for evaluations when OpenTelemetry (OTel) tracing is enabled when DD_TRACE_OTEL_ENABLED=true is set. This tag allows the backend to wait for OTel span conversion before processing evaluations.DD_MCP_CAPTURE_INTENT environment variable.asyncio.BoundedSemaphore lock type profiling in Python Lock Profiler.asyncio.Condition locking type profiling in Python. The Lock profiler now provides visibility into asyncio.Condition usage, helping identify contention in async applications using condition variables.asyncio.Semaphore lock type profiling in Python Lock Profiler.asyncio.shield.asyncio.TaskGroup.aws.httpapi span name for v2 apis when the API Gateway sets the x-dd-proxy header to aws-httpapi. Additionally, the tag http.route and the resource name of the span now contains the api resource path instead of the path when propagated with the x-dd-proxy-resource-path header.DD_TRACE_REMOVE_INTEGRATION_SERVICE_NAMES_ENABLED support, which was previously ignored.https:// scheme prefix as part of the http.url tag; this caused the entire url to be parsed as the http path.isDefined to result in an evaluation error.DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED=true), ensuring remote configuration is received before process forking occurs.None metadata in Ray job submission caused a crash.AttributeError when calling tag_agent_manifest.annotation_context blocks caused annotations to fail after the first operation in subsequent contexts. Previously, the trace context created by the first annotation_context remained active after exiting, causing the second context to reuse a stale context ID. This resulted in annotations not being applied to spans after the first batch call in the second annotation_context block.activate_distributed_headers() where distributed requests missing a LLM Observability trace ID would be incorrectly propagated twice.TypeError during profiling. One example of this is neo4j's AsyncRLock, which inherits from asyncio.Lock: https://github.com/neo4j/neo4j-python-driver/blob/6.x/src/neo4j/_async_compat/concurrency.py#L45uvloop and forking has been resolved._psutil_linux.abi3.so file in an injected environment.The Hooks class (config. .hooks) is deprecated and will be removed in v5.0. All hook methods (register(), on(), deregister(), emit()) are now no-op an…
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Hooks class (config.<integration>.hooks) is deprecated and will be removed in v5.0. All hook methods (register(), on(), deregister(), emit()) are now no-op and no longer affect span behavior. To interact with spans, use ddtrace.trace_utils.get_current_span() or ddtrace.trace_utils.get_current_root_span() instead.initialize requests and their responses on modelcontextprotocol/python-sdk servers.source:otel tagging for evaluations when OpenTelemetry (OTel) tracing is enabled when DD_TRACE_OTEL_ENABLED=true is set. This tag allows the backend to wait for OTel span conversion before processing evaluations.DD_MCP_CAPTURE_INTENT environment variable.asyncio.BoundedSemaphore lock type profiling in Python Lock Profiler.asyncio.Condition locking type profiling in Python. The Lock profiler now provides visibility into asyncio.Condition usage, helping identify contention in async applications using condition variables.asyncio.Semaphore lock type profiling in Python Lock Profiler.asyncio.shield.asyncio.TaskGroup.aws.httpapi span name for v2 apis when the API Gateway sets the x-dd-proxy header to aws-httpapi. Additionally, the tag http.route and the resource name of the span now contains the api resource path instead of the path when propagated with the x-dd-proxy-resource-path header.DD_TRACE_REMOVE_INTEGRATION_SERVICE_NAMES_ENABLED support, which was previously ignored.https:// scheme prefix as part of the http.url tag; this caused the entire url to be parsed as the http path.isDefined to result in an evaluation error.DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED=true), ensuring remote configuration is received before process forking occurs.None metadata in Ray job submission caused a crash.AttributeError when calling tag_agent_manifest.annotation_context blocks caused annotations to fail after the first operation in subsequent contexts. Previously, the trace context created by the first annotation_context remained active after exiting, causing the second context to reuse a stale context ID. This resulted in annotations not being applied to spans after the first batch call in the second annotation_context block.activate_distributed_headers() where distributed requests missing a LLM Observability trace ID would be incorrectly propagated twice.TypeError during profiling. One example of this is neo4j's AsyncRLock, which inherits from asyncio.Lock: https://github.com/neo4j/neo4j-python-driver/blob/6.x/src/neo4j/_async_compat/concurrency.py#L45uvloop and forking has been resolved._psutil_linux.abi3.so file in an injected environment.The Hooks class (config. .hooks) is deprecated and will be removed in v5.0. All hook methods (register(), on(), deregister(), emit()) are now no-op an…
Estimated end-of-life date, accurate to within three months: 07-2027 See the support level definitions for more information.
Hooks class (config.<integration>.hooks) is deprecated and will be removed in v5.0. All hook methods (register(), on(), deregister(), emit()) are now no-op and no longer affect span behavior. To interact with spans, use ddtrace.trace_utils.get_current_span() or ddtrace.trace_utils.get_current_root_span() instead.initialize requests and their responses on modelcontextprotocol/python-sdk servers.source:otel tagging for evaluations when OpenTelemetry (OTel) tracing is enabled when DD_TRACE_OTEL_ENABLED=true is set. This tag allows the backend to wait for OTel span conversion before processing evaluations.asyncio.BoundedSemaphore lock type profiling in Python Lock Profiler.asyncio.Condition locking type profiling in Python. The Lock profiler now provides visibility into asyncio.Condition usage, helping identify contention in async applications using condition variables.asyncio.Semaphore lock type profiling in Python Lock Profiler.aws.httpapi span name for v2 apis when the API Gateway sets the x-dd-proxy header to aws-httpapi. Additionally, the tag http.route and the resource name of the span now contains the api resource path instead of the path when propagated with the x-dd-proxy-resource-path header.DD_TRACE_REMOVE_INTEGRATION_SERVICE_NAMES_ENABLED support, which was previously ignored.https:// scheme prefix as part of the http.url tag; this caused the entire url to be parsed as the http path.isDefined to result in an evaluation error.DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED=true), ensuring remote configuration is received before process forking occurs.None metadata in Ray job submission caused a crash.AttributeError when calling tag_agent_manifest.TypeError during profiling. One example of this is neo4j's AsyncRLock, which inherits from asyncio.Lock: https://github.com/neo4j/neo4j-python-driver/blob/6.x/src/neo4j/_async_compat/concurrency.py#L45uvloop and forking has been resolved.Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
anthropic: Handle empty content field when tracing the beta client.
LLM Observability: Resolves an issue in activate_distributed_headers() where distributed requests missing a LLM Observability trace ID would be incorrectly propagated twice.
profiling: Fixes a bug where code that sub-classes our wrapped locks crashes with TypeError during profiling. One example of this is neo4j's AsyncRLock, which inherits from asyncio.Lock: https://github.com/neo4j/neo4j-python-driver/blob/6.x/src/neo4j/_async_compat/concurrency.py#L45
profiling: This fix resolves a race condition leading to incorrect stacks being reported for asyncio parent/child Tasks (e.g. when using asyncio.gather).
profiling: Fixes a PicklingError when using multiprocessing.Manager() with lock profiling enabled on Python 3.14+. Python 3.14 changed the default multiprocessing start method from fork to forkserver, which requires all objects to be pickleable. The lock profiler's wrappers now properly support pickle serialization.
Estimated end-of-life date, accurate to within three months: 07-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 07-2027 See the support level definitions for more information.
_psutil_linux.abi3.so file in an injected environment.<!-- -->
annotation_context blocks caused annotations to fail after the first operation in subsequent contexts. Previously, the trace context created by the first annotation_context remained active after exiting, causing the second context to reuse a stale context ID. This resulted in annotations not being applied to spans after the first batch call in the second annotation_context block.<!-- -->
<!-- -->
uvloop and forking has been resolved.<!-- -->
<!-- -->
Estimated end-of-life date, accurate to within three months: 07-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 07-2027 See the support level definitions for more information.
None metadata in Ray job submission caused a crash.Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
DD_CODE_ORIGIN_FOR_SPANS_ENABLED=true is setDeprecated support for Tornado versions older than v6.1. Use Tornado v6.1 or later.
⚠️ An issue was detected with memory profiling in this release. Please consider upgrading to v4.1.3 or newer
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
DD_CODE_ORIGIN_FOR_SPANS_ENABLED=true is setdataset_name, project_name, project_id, experiment_name.ExperimentResult class' rows and summary_evaluations attributes are deprecated and will be removed in the next major release. ExperimentResult.rows/summary_evaluations attributes will only store the results of the first run iteration for multi-run experiments. Use the ExperimentResult.runs attribute instead to access experiment results and summary evaluations.threading.BoundedSemaphore locking type profiling in Python. The implementation follows the same approach as threading.Semaphore, properly handling internal lock detection to prevent double-counting of the underlying threading.Lock object.threading.Semaphore locking type profiling in Python. The Lock profiler now detects and marks "internal" Lock objects, i.e. those that are part of implementation of higher-level locking types. One example of such higher-level primitive is threading.Semaphore, which is implemented with threading.Condition, which itself uses threading.Lock internally. Marking internal lock as "internal" will prevent it from being sampled, ensuring that the high-level (e.g. Semaphore) sample is processed.process_id tag to profiles. The value of this tag is the current process ID (PID).asyncio.wait.codeobject.co_qualname in memory profiler and lock profiler flamegraphs for Python 3.11+. Stack profiler has already been using this. This aligns the user experience across different profile types.asyncio.as_completed util in the Profiler.asyncio.wait in the Profiler. This makes it possible to track dependencies between Tasks/Coroutines that await/are awaited through asyncio.wait.client.beta.messages.create() and client.beta.messages.stream()). This feature requires Anthropic client version 0.37.0 or higher.aiokafka>=0.9.0. See the aiokafka<https://ddtrace.readthedocs.io/en/stable/integrations.html#aiokafka> documentation for more information.thread=False is no longer required when performing monkey-patching with gevent via gevent.monkey.patch_all.responses endpoint (available in OpenAI SDK >= 1.87.0).runs argument, to assess the true performance of an experiment in the face of the non determinism of LLMs. Use the new ExperimentResult class' runs attribute to access the results and summary evaluations by run iteration.RunnableLambda instances._Py_DumpTracebackThreads function is not available.IndexError in partial flush when the finished span counter was out of sync with actual finished spans.DD_TRACE_PARTIAL_FLUSH_MIN_SPANS values less than 1 now default to 1 with a warning.ray.init() at the top of their scripts were not properly instrumented, resulting in incomplete traces. To ensure full tracing capabilities, use ddtrace-run when starting your Ray cluster: DD_PATCH_MODULES="ray:true,aiohttp:false,grpc:false,requests:false" ddtrace-run ray start --head.gsutil toolLLMObs.export_span() would raise when LLMObs is disabled.self was being annotated as an input parameter using LLM Observability function decorators.LLMObs.annotation_context() properties (tags, prompt, and name) were not applied to subsequent LLM operations within the same context block. This occurred when multiple sequential operations (such as Langchain batch calls with structured outputs) were performed, causing only the first operation to receive the annotations.AttributeError when trying to access the name or description attributes of a tool.opentelemetry.trace.get_current_span() or NonRecordingSpan. Spans are now kept and appear in the UI unless explicitly dropped by the Agent or sampling rules.frame.f_locals while trying to retrieve class name of a PyFrameObject.asyncio.gather).Deprecated support for Tornado versions older than v6.1. Use Tornado v6.1 or later.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
dataset_name, project_name, project_id, experiment_name.ddtrace.contrib.tornado module. Configure tracing using environment variables and import ddtrace.auto instead.ExperimentResult class' rows and summary_evaluations attributes are deprecated and will be removed in the next major release. ExperimentResult.rows/summary_evaluations attributes will only store the results of the first run iteration for multi-run experiments. Use the ExperimentResult.runs attribute instead to access experiment results and summary evaluations.threading.BoundedSemaphore locking type profiling in Python. The implementation follows the same approach as threading.Semaphore, properly handling internal lock detection to prevent double-counting of the underlying threading.Lock object.threading.Semaphore locking type profiling in Python. The Lock profiler now detects and marks "internal" Lock objects, i.e. those that are part of implementation of higher-level locking types. One example of such higher-level primitive is threading.Semaphore, which is implemented with threading.Condition, which itself uses threading.Lock internally. Marking internal lock as "internal" will prevent it from being sampled, ensuring that the high-level (e.g. Semaphore) sample is processed.process_id tag to profiles. The value of this tag is the current process ID (PID).asyncio.wait.codeobject.co_qualname in memory profiler and lock profiler flamegraphs for Python 3.11+. Stack profiler has already been using this. This aligns the user experience across different profile types.asyncio.as_completed util in the Profiler.asyncio.wait in the Profiler. This makes it possible to track dependencies between Tasks/Coroutines that await/are awaited through asyncio.wait.client.beta.messages.create() and client.beta.messages.stream()). This feature requires Anthropic client version 0.37.0 or higher.aiokafka>=0.9.0. See the aiokafka<https://ddtrace.readthedocs.io/en/stable/integrations.html#aiokafka> documentation for more information.thread=False is no longer required when performing monkey-patching with gevent via gevent.monkey.patch_all.responses endpoint (available in OpenAI SDK >= 1.87.0).runs argument, to assess the true performance of an experiment in the face of the non determinism of LLMs. Use the new ExperimentResult class' runs attribute to access the results and summary evaluations by run iteration.RunnableLambda instances._Py_DumpTracebackThreads function is not available.IndexError in partial flush when the finished span counter was out of sync with actual finished spans.DD_TRACE_PARTIAL_FLUSH_MIN_SPANS values less than 1 now default to 1 with a warning.ray.init() at the top of their scripts were not properly instrumented, resulting in incomplete traces. To ensure full tracing capabilities, use ddtrace-run when starting your Ray cluster: DD_PATCH_MODULES="ray:true,aiohttp:false,grpc:false,requests:false" ddtrace-run ray start --head.gsutil toolLLMObs.export_span() would raise when LLMObs is disabled.self was being annotated as an input parameter using LLM Observability function decorators.LLMObs.annotation_context() properties (tags, prompt, and name) were not applied to subsequent LLM operations within the same context block. This occurred when multiple sequential operations (such as Langchain batch calls with structured outputs) were performed, causing only the first operation to receive the annotations.AttributeError when trying to access the name or description attributes of a tool.opentelemetry.trace.get_current_span() or NonRecordingSpan. Spans are now kept and appear in the UI unless explicitly dropped by the Agent or sampling rules.frame.f_locals while trying to retrieve class name of a PyFrameObject.asyncio.gather).Deprecated support for Tornado versions older than v6.1. Use Tornado v6.1 or later.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
dataset_name, project_name, project_id, experiment_name.ddtrace.contrib.tornado module. Configure tracing using environment variables and import ddtrace.auto instead.ExperimentResult class' rows and summary_evaluations attributes are deprecated and will be removed in the next major release. ExperimentResult.rows/summary_evaluations attributes will only store the results of the first run iteration for multi-run experiments. Use the ExperimentResult.runs attribute instead to access experiment results and summary evaluations.threading.BoundedSemaphore locking type profiling in Python. The implementation follows the same approach as threading.Semaphore, properly handling internal lock detection to prevent double-counting of the underlying threading.Lock object.threading.Semaphore locking type profiling in Python. The Lock profiler now detects and marks "internal" Lock objects, i.e. those that are part of implementation of higher-level locking types. One example of such higher-level primitive is threading.Semaphore, which is implemented with threading.Condition, which itself uses threading.Lock internally. Marking internal lock as "internal" will prevent it from being sampled, ensuring that the high-level (e.g. Semaphore) sample is processed.process_id tag to profiles. The value of this tag is the current process ID (PID).asyncio.wait.codeobject.co_qualname in memory profiler and lock profiler flamegraphs for Python 3.11+. Stack profiler has already been using this. This aligns the user experience across different profile types.asyncio.as_completed util in the Profiler.asyncio.wait in the Profiler. This makes it possible to track dependencies between Tasks/Coroutines that await/are awaited through asyncio.wait.client.beta.messages.create() and client.beta.messages.stream()). This feature requires Anthropic client version 0.37.0 or higher.aiokafka>=0.9.0. See the aiokafka<https://ddtrace.readthedocs.io/en/stable/integrations.html#aiokafka> documentation for more information.thread=False is no longer required when performing monkey-patching with gevent via gevent.monkey.patch_all.responses endpoint (available in OpenAI SDK >= 1.87.0).runs argument, to assess the true performance of an experiment in the face of the non determinism of LLMs. Use the new ExperimentResult class' runs attribute to access the results and summary evaluations by run iteration.RunnableLambda instances._Py_DumpTracebackThreads function is not available.IndexError in partial flush when the finished span counter was out of sync with actual finished spans.DD_TRACE_PARTIAL_FLUSH_MIN_SPANS values less than 1 now default to 1 with a warning.ray.init() at the top of their scripts were not properly instrumented, resulting in incomplete traces. To ensure full tracing capabilities, use ddtrace-run when starting your Ray cluster: DD_PATCH_MODULES="ray:true,aiohttp:false,grpc:false,requests:false" ddtrace-run ray start --head.gsutil toolLLMObs.export_span() would raise when LLMObs is disabled.self was being annotated as an input parameter using LLM Observability function decorators.LLMObs.annotation_context() properties (tags, prompt, and name) were not applied to subsequent LLM operations within the same context block. This occurred when multiple sequential operations (such as Langchain batch calls with structured outputs) were performed, causing only the first operation to receive the annotations.AttributeError when trying to access the name or description attributes of a tool.opentelemetry.trace.get_current_span() or NonRecordingSpan. Spans are now kept and appear in the UI unless explicitly dropped by the Agent or sampling rules.frame.f_locals while trying to retrieve class name of a PyFrameObject.asyncio.gather).Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
activate_distributed_headers() where distributed requests missing a LLM Observability trace ID would be incorrectly propagated twice.<!-- -->
TypeError during profiling.
One example of this is neo4j's AsyncRLock, which inherits from asyncio.Lock:
https://github.com/neo4j/neo4j-python-driver/blob/6.x/src/neo4j/_async_compat/concurrency.py#L45<!-- -->
asyncio.gather).Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
<!-- -->
uvloop and forking has been resolved.<!-- -->
profiling: This fix resolves an issue where memory profiler module fails to load when the system doesn't have libatomic installed.
profiling: This fix ensures the profiler now correctly tracks dependencies between Tasks and Coroutines that are awaiting or being awaited via asyncio.wait.
Fix exposure event deduplication to use (flag_key, subject_id) as cache key instead of (flag_key, variant_key, allocation_key). This ensures different
Fixes critical memory safety issue in IAST when used with forked worker processes (MCP servers with Gunicorn and Uvicorn). Workers previously crashed
Code Security:
tracing:
LLM Observability:
AttributeError when trying to access the name or description attributes of a tool.AAP:
lib-injection:
gsutil toolprofiling:
frame.f_locals while trying to retrieve class name of a PyFrameObject.To find which of these your code relies on, follow the "deprecation warnings" instructions here.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
This is a major-version release that contains many backwards-incompatible changes to public APIs. To find which of these your code relies on, follow the "deprecation warnings" instructions here.
dd-trace-py now includes an OpenFeature provider implementation, enabling feature flag evaluation through the OpenFeature API.
This integration is under active design and development. Functionality and APIs are experimental and may change without notice. For more information, see the Datadog documentation at https://docs.datadoghq.com/feature_flags/#overview
ddtrace.Pin object with mongoengine. With this change, the ddtrace library no longer directly supports mongoengine. Mongoengine will be supported through the pymongo integration.pytest_benchmark and pytest_bdd integrations. These plugins are now supported by the regular pytest integration.DD_DYNAMIC_INSTRUMENTATION_UPLOAD_FLUSH_INTERVAL variable.DD_EXCEPTION_DEBUGGING_ENABLED variable.Span.set_tag_str has been removed, use Span.set_tag instead.Span.set_struct_tag has been removed.Span.get_struct_tag has been removed.Span._pprint has been removedSpan.finished setter was removed, please use Span.finish() method instead.Tracer.on_start_span method has been removed.Tracer.deregister_on_start_span method has been removed.ddtrace.trace.Pin has been removed.Span.finish_with_ancestors was removed with no replacement.Span.set_tag typing is now set_tag(key: str, value: Optional[str] = None) -> NoneSpan.get_tag typing is now get_tag(key: str) -> Optional[str]Span.set_tags typing is now set_tags(tags: dict[str, str]) -> NoneSpan.get_tags typing is now get_tags() -> dict[str, str]Span.set_metric typing is now set_metric(key: str, value: int | float) -> NoneSpan.get_metric typing is now get_metric(key: str) -> Optional[int | float]Span.set_metrics typing is now set_metrics(metrics: Dict[str, int | float]) -> NoneSpan.get_metrics typing is now get_metrics() -> dict[str, int | float]Span.record_exception's timestamp and escaped parameters are removedLLMObs.annotate(), LLMObs.export_span(), LLMObs.submit_evaluation(), LLMObs.inject_distributed_headers(), and LLMObs.activate_distributed_headers() now raise exceptions instead of logging. LLM Observability auto-instrumentation is not affected.LLMObs.submit_evaluation_for() has been removed. Please use LLMObs.submit_evaluation() instead for submitting evaluations. To migrate:
LLMObs.submit_evaluation_for(...) users: rename to LLMObs.submit_evaluation(...)LLMObs.submit_evaluation_for(...) users: rename the span_context argument to span, i.e. LLMObs.submit_evaluation(span_context={"span_id": ..., "trace_id": ...}, ...) to LLMObs.submit_evaluation(span={"span_id": ..., "trace_id": ...}, ...)DD_PROFILING_STACK_V2_ENABLED is now removed.freezegun integration is now removed.opentracer packagegoogle_generativeai integration has been removed as the google_generativeai library has reached end-of-life.
As an alternative, you can use the recommended google_genai library and corresponding integration instead.tiktoken library, and instead
will default to having their token counts estimated if not explicitly provided in the OpenAI response object. To guarantee accurate streamed token metrics, set stream_options={"include_usage": True} in the OpenAI request.DD_DJANGO_TRACING_MINIMAL now defaults to true). Django ORM, cache, and template instrumentation are disabled by default to eliminate duplicate span creation since library integrations for database drivers (psycopg, MySQLdb, sqlite3), cache clients (redis, memcached), template renderers (Jinja2), and other supported libraries continue to be traced. This reduces performance overhead by removing redundant Django-layer instrumentation. To restore all Django instrumentation, set DD_DJANGO_TRACING_MINIMAL=false, or enable individual features using DD_DJANGO_INSTRUMENT_DATABASES=true, DD_DJANGO_INSTRUMENT_CACHES=true, and DD_DJANGO_INSTRUMENT_TEMPLATES=true.DD_DJANGO_INSTRUMENT_DATABASES=true (default false), database instrumentation now merges Django-specific tags into database driver spans created by supported integrations (psycopg, sqlite3, MySQLdb, etc.) instead of creating duplicate Django database spans. If the database cursor is not already wrapped by a supported integration, Django wraps it and creates a span. This change reduces overhead and duplicate spans while preserving visibility into database operations.ddtrace.settings package. Environment variables should be used to adjust settings.HttpPropagator.injectDEFAULT_RUNTIME_METRICS_INTERVAL.ddtrace.contrib.tornado module. Configure tracing using environment variables and import ddtrace.auto instead.urllib3 and requests. It does not require enabling APM instrumentation for urllib3 anymore.threading.RLock (reentrant lock) profiling. The Lock profiler now tracks both threading.Lock and threading.RLock usage, providing comprehensive lock contention visibility for Python applications.version argument to LLMObs.pull_datasetversion and latest_version to provide information on the version of the dataset that is being worked with and the latest global version of the dataset, respectivelyLock, RLock, Event).HTTPS_PROXY.assessment argument in submit_evaluation().langchain integration would incorrectly mark Azure OpenAI calls as duplicate llm operations even if the openai integration was enabled.DD_PROFILING_API_TIMEOUT doesn't have any effect, and is marked to be removed in upcoming 4.0 release. New environment variable DD_PROFILING_API_TIMEOUT_MS is introduced to configure timeout for uploading profiles to the backend. The default value is 10000 ms (10 seconds)_acquire method of the Lock profiler (note: this only occurs when assertions are enabled.)version field. The version field is now omitted unless explicitly set by the user.
assessment now refers to whether the evaluation itself passes or fails according to your application, rather than the validity of the evaluation result.
The langchain integration will trace Azure OpenAI spans as workflow spans if there is an equivalent llm span from the openai integration.IndexError.async for model in client.models.list()) caused a TypeError: 'async for' requires an object with __aiter__ method, got coroutine. See issue #14574.KeyError exceptions in test runs when gevent is detected within the environment.ray.init().ray.data._internal to the module denylist.ValueError: Formatting field not found in record: 'dd.service'.ResourceWarning in multiprocess scenarios.wrapt library dependency from the Lock Profiler implementation, improving performance and reducing overhead during lock instrumentation.To find which of these your code relies on, follow the "deprecation warnings" instructions here.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
This is a major-version release that contains many backwards-incompatible changes to public APIs. To find which of these your code relies on, follow the "deprecation warnings" instructions here.
dd-trace-py now includes an OpenFeature provider implementation, enabling feature flag evaluation through the OpenFeature API.
This integration is under active design and development. Functionality and APIs are experimental and may change without notice. For more information, see the Datadog documentation at https://docs.datadoghq.com/feature_flags/#overview
ddtrace.Pin object with mongoengine. With this change, the ddtrace library no longer directly supports mongoengine. Mongoengine will be supported through the pymongo integration.pytest_benchmark and pytest_bdd integrations. These plugins are now supported by the regular pytest integration.DD_DYNAMIC_INSTRUMENTATION_UPLOAD_FLUSH_INTERVAL variable.DD_EXCEPTION_DEBUGGING_ENABLED variable.Span.set_tag_str has been removed, use Span.set_tag instead.Span.set_struct_tag has been removed.Span.get_struct_tag has been removed.Span._pprint has been removedSpan.finished setter was removed, please use Span.finish() method instead.Tracer.on_start_span method has been removed.Tracer.deregister_on_start_span method has been removed.ddtrace.trace.Pin has been removed.Span.finish_with_ancestors was removed with no replacement.Span.set_tag typing is now set_tag(key: str, value: Optional[str] = None) -> NoneSpan.get_tag typing is now get_tag(key: str) -> Optional[str]Span.set_tags typing is now set_tags(tags: dict[str, str]) -> NoneSpan.get_tags typing is now get_tags() -> dict[str, str]Span.set_metric typing is now set_metric(key: str, value: int | float) -> NoneSpan.get_metric typing is now get_metric(key: str) -> Optional[int | float]Span.set_metrics typing is now set_metrics(metrics: Dict[str, int | float]) -> NoneSpan.get_metrics typing is now get_metrics() -> dict[str, int | float]Span.record_exception's timestamp and escaped parameters are removedLLMObs.annotate(), LLMObs.export_span(), LLMObs.submit_evaluation(), LLMObs.inject_distributed_headers(), and LLMObs.activate_distributed_headers() now raise exceptions instead of logging. LLM Observability auto-instrumentation is not affected.LLMObs.submit_evaluation_for() has been removed. Please use LLMObs.submit_evaluation() instead for submitting evaluations. To migrate:
LLMObs.submit_evaluation_for(...) users: rename to LLMObs.submit_evaluation(...)LLMObs.submit_evaluation_for(...) users: rename the span_context argument to span, i.e. LLMObs.submit_evaluation(span_context={"span_id": ..., "trace_id": ...}, ...) to LLMObs.submit_evaluation(span={"span_id": ..., "trace_id": ...}, ...)DD_PROFILING_STACK_V2_ENABLED is now removed.freezegun integration is now removed.opentracer packagegoogle_generativeai integration has been removed as the google_generativeai library has reached end-of-life.
As an alternative, you can use the recommended google_genai library and corresponding integration instead.tiktoken library, and instead
will default to having their token counts estimated if not explicitly provided in the OpenAI response object. To guarantee accurate streamed token metrics, set stream_options={"include_usage": True} in the OpenAI request.DD_DJANGO_TRACING_MINIMAL now defaults to true). Django ORM, cache, and template instrumentation are disabled by default to eliminate duplicate span creation since library integrations for database drivers (psycopg, MySQLdb, sqlite3), cache clients (redis, memcached), template renderers (Jinja2), and other supported libraries continue to be traced. This reduces performance overhead by removing redundant Django-layer instrumentation. To restore all Django instrumentation, set DD_DJANGO_TRACING_MINIMAL=false, or enable individual features using DD_DJANGO_INSTRUMENT_DATABASES=true, DD_DJANGO_INSTRUMENT_CACHES=true, and DD_DJANGO_INSTRUMENT_TEMPLATES=true.DD_DJANGO_INSTRUMENT_DATABASES=true (default false), database instrumentation now merges Django-specific tags into database driver spans created by supported integrations (psycopg, sqlite3, MySQLdb, etc.) instead of creating duplicate Django database spans. If the database cursor is not already wrapped by a supported integration, Django wraps it and creates a span. This change reduces overhead and duplicate spans while preserving visibility into database operations.ddtrace.settings package. Environment variables should be used to adjust settings.HttpPropagator.injectDEFAULT_RUNTIME_METRICS_INTERVAL.ddtrace.contrib.tornado module. Configure tracing using environment variables and import ddtrace.auto instead.urllib3 and requests. It does not require enabling APM instrumentation for urllib3 anymore.threading.RLock (reentrant lock) profiling. The Lock profiler now tracks both threading.Lock and threading.RLock usage, providing comprehensive lock contention visibility for Python applications.version argument to LLMObs.pull_datasetversion and latest_version to provide information on the version of the dataset that is being worked with and the latest global version of the dataset, respectivelyLock, RLock, Event).HTTPS_PROXY.assessment argument in submit_evaluation().langchain integration would incorrectly mark Azure OpenAI calls as duplicate llm operations even if the openai integration was enabled.DD_PROFILING_API_TIMEOUT doesn't have any effect, and is marked to be removed in upcoming 4.0 release. New environment variable DD_PROFILING_API_TIMEOUT_MS is introduced to configure timeout for uploading profiles to the backend. The default value is 10000 ms (10 seconds)_acquire method of the Lock profiler (note: this only occurs when assertions are enabled.)version field. The version field is now omitted unless explicitly set by the user.
assessment now refers to whether the evaluation itself passes or fails according to your application, rather than the validity of the evaluation result.
The langchain integration will trace Azure OpenAI spans as workflow spans if there is an equivalent llm span from the openai integration.IndexError.async for model in client.models.list()) caused a TypeError: 'async for' requires an object with __aiter__ method, got coroutine. See issue #14574.KeyError exceptions in test runs when gevent is detected within the environment.ray.init().ray.data._internal to the module denylist.ValueError: Formatting field not found in record: 'dd.service'.ResourceWarning in multiprocess scenarios.wrapt library dependency from the Lock Profiler implementation, improving performance and reducing overhead during lock instrumentation.To find which of these your code relies on, follow the "deprecation warnings" instructions here.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
This is a major-version release that contains many backwards-incompatible changes to public APIs. To find which of these your code relies on, follow the "deprecation warnings" instructions here.
dd-trace-py now includes an OpenFeature provider implementation, enabling feature flag evaluation through the OpenFeature API.
This integration is under active design and development. Functionality and APIs are experimental and may change without notice. For more information, see the Datadog documentation at https://docs.datadoghq.com/feature_flags/#overview
ddtrace.Pin object with mongoengine. With this change, the ddtrace library no longer directly supports mongoengine. Mongoengine will be supported through the pymongo integration.pytest_benchmark and pytest_bdd integrations. These plugins are now supported by the regular pytest integration.DD_DYNAMIC_INSTRUMENTATION_UPLOAD_FLUSH_INTERVAL variable.DD_EXCEPTION_DEBUGGING_ENABLED variable.Span.set_tag_str has been removed, use Span.set_tag instead.Span.set_struct_tag has been removed.Span.get_struct_tag has been removed.Span._pprint has been removedSpan.finished setter was removed, please use Span.finish() method instead.Tracer.on_start_span method has been removed.Tracer.deregister_on_start_span method has been removed.ddtrace.trace.Pin has been removed.Span.finish_with_ancestors was removed with no replacement.Span.set_tag typing is now set_tag(key: str, value: Optional[str] = None) -> NoneSpan.get_tag typing is now get_tag(key: str) -> Optional[str]Span.set_tags typing is now set_tags(tags: dict[str, str]) -> NoneSpan.get_tags typing is now get_tags() -> dict[str, str]Span.set_metric typing is now set_metric(key: str, value: int | float) -> NoneSpan.get_metric typing is now get_metric(key: str) -> Optional[int | float]Span.set_metrics typing is now set_metrics(metrics: Dict[str, int | float]) -> NoneSpan.get_metrics typing is now get_metrics() -> dict[str, int | float]Span.record_exception's timestamp and escaped parameters are removedLLMObs.annotate(), LLMObs.export_span(), LLMObs.submit_evaluation(), LLMObs.inject_distributed_headers(), and LLMObs.activate_distributed_headers() now raise exceptions instead of logging. LLM Observability auto-instrumentation is not affected.LLMObs.submit_evaluation_for() has been removed. Please use LLMObs.submit_evaluation() instead for submitting evaluations. To migrate:
LLMObs.submit_evaluation_for(...) users: rename to LLMObs.submit_evaluation(...)LLMObs.submit_evaluation_for(...) users: rename the span_context argument to span, i.e. LLMObs.submit_evaluation(span_context={"span_id": ..., "trace_id": ...}, ...) to LLMObs.submit_evaluation(span={"span_id": ..., "trace_id": ...}, ...)DD_PROFILING_STACK_V2_ENABLED is now removed.freezegun integration is now removed.opentracer packagegoogle_generativeai integration has been removed as the google_generativeai library has reached end-of-life.
As an alternative, you can use the recommended google_genai library and corresponding integration instead.tiktoken library, and instead
will default to having their token counts estimated if not explicitly provided in the OpenAI response object. To guarantee accurate streamed token metrics, set stream_options={"include_usage": True} in the OpenAI request.DD_DJANGO_TRACING_MINIMAL now defaults to true). Django ORM, cache, and template instrumentation are disabled by default to eliminate duplicate span creation since library integrations for database drivers (psycopg, MySQLdb, sqlite3), cache clients (redis, memcached), template renderers (Jinja2), and other supported libraries continue to be traced. This reduces performance overhead by removing redundant Django-layer instrumentation. To restore all Django instrumentation, set DD_DJANGO_TRACING_MINIMAL=false, or enable individual features using DD_DJANGO_INSTRUMENT_DATABASES=true, DD_DJANGO_INSTRUMENT_CACHES=true, and DD_DJANGO_INSTRUMENT_TEMPLATES=true.DD_DJANGO_INSTRUMENT_DATABASES=true (default false), database instrumentation now merges Django-specific tags into database driver spans created by supported integrations (psycopg, sqlite3, MySQLdb, etc.) instead of creating duplicate Django database spans. If the database cursor is not already wrapped by a supported integration, Django wraps it and creates a span. This change reduces overhead and duplicate spans while preserving visibility into database operations.ddtrace.settings package. Environment variables should be used to adjust settings.HttpPropagator.injectDEFAULT_RUNTIME_METRICS_INTERVAL.urllib3 and requests. It does not require enabling APM instrumentation for urllib3 anymore.threading.RLock (reentrant lock) profiling. The Lock profiler now tracks both threading.Lock and threading.RLock usage, providing comprehensive lock contention visibility for Python applications.version argument to LLMObs.pull_datasetversion and latest_version to provide information on the version of the dataset that is being worked with and the latest global version of the dataset, respectivelyLock, RLock, Event).HTTPS_PROXY.assessment argument in submit_evaluation().langchain integration would incorrectly mark Azure OpenAI calls as duplicate llm operations even if the openai integration was enabled.DD_PROFILING_API_TIMEOUT doesn't have any effect, and is marked to be removed in upcoming 4.0 release. New environment variable DD_PROFILING_API_TIMEOUT_MS is introduced to configure timeout for uploading profiles to the backend. The default value is 10000 ms (10 seconds)_acquire method of the Lock profiler (note: this only occurs when assertions are enabled.)version field. The version field is now omitted unless explicitly set by the user.
assessment now refers to whether the evaluation itself passes or fails according to your application, rather than the validity of the evaluation result.
The langchain integration will trace Azure OpenAI spans as workflow spans if there is an equivalent llm span from the openai integration.IndexError.async for model in client.models.list()) caused a TypeError: 'async for' requires an object with __aiter__ method, got coroutine. See issue #14574.KeyError exceptions in test runs when gevent is detected within the environment.ray.init().ray.data._internal to the module denylist.ValueError: Formatting field not found in record: 'dd.service'.ResourceWarning in multiprocess scenarios.wrapt library dependency from the Lock Profiler implementation, improving performance and reducing overhead during lock instrumentation.To find which of these your code relies on, follow the "deprecation warnings" instructions here.
Estimated end-of-life date, accurate to within three months: 05-2027 See the support level definitions for more information.
This is a major-version release that contains many backwards-incompatible changes to public APIs. To find which of these your code relies on, follow the "deprecation warnings" instructions here.
ddtrace.Pin object with mongoengine. With this change, the ddtrace library no longer directly supports mongoengine. Mongoengine will be supported through the pymongo integration.pytest_benchmark and pytest_bdd integrations. These plugins are now supported by the regular pytest integration.DD_DYNAMIC_INSTRUMENTATION_UPLOAD_FLUSH_INTERVAL variable.DD_EXCEPTION_DEBUGGING_ENABLED variable.Span.set_tag_str has been removed, use Span.set_tag instead.Span.set_struct_tag has been removed.Span.get_struct_tag has been removed.Span._pprint has been removedSpan.finished setter was removed, please use Span.finish() method instead.Tracer.on_start_span method has been removed.Tracer.deregister_on_start_span method has been removed.ddtrace.trace.Pin has been removed.Span.finish_with_ancestors was removed with no replacement.Span.set_tag typing is now set_tag(key: str, value: Optional[str] = None) -> NoneSpan.get_tag typing is now get_tag(key: str) -> Optional[str]Span.set_tags typing is now set_tags(tags: dict[str, str]) -> NoneSpan.get_tags typing is now get_tags() -> dict[str, str]Span.set_metric typing is now set_metric(key: str, value: int | float) -> NoneSpan.get_metric typing is now get_metric(key: str) -> Optional[int | float]Span.set_metrics typing is now set_metrics(metrics: Dict[str, int | float]) -> NoneSpan.get_metrics typing is now get_metrics() -> dict[str, int | float]Span.record_exception's timestamp and escaped parameters are removedLLMObs.annotate(), LLMObs.export_span(), LLMObs.submit_evaluation(), LLMObs.inject_distributed_headers(), and LLMObs.activate_distributed_headers() now raise exceptions instead of logging. LLM Observability auto-instrumentation is not affected.LLMObs.submit_evaluation_for() has been removed. Please use LLMObs.submit_evaluation() instead for submitting evaluations. To migrate:
LLMObs.submit_evaluation_for(...) users: rename to LLMObs.submit_evaluation(...)LLMObs.submit_evaluation_for(...) users: rename the span_context argument to span, i.e. LLMObs.submit_evaluation(span_context={"span_id": ..., "trace_id": ...}, ...) to LLMObs.submit_evaluation(span={"span_id": ..., "trace_id": ...}, ...)DD_PROFILING_STACK_V2_ENABLED is now removed.freezegun integration is now removed.opentracer packagegoogle_generativeai integration has been removed as the google_generativeai library has reached end-of-life.
As an alternative, you can use the recommended google_genai library and corresponding integration instead.tiktoken library, and instead
will default to having their token counts estimated if not explicitly provided in the OpenAI response object. To guarantee accurate streamed token metrics, set stream_options={"include_usage": True} in the OpenAI request.DD_DJANGO_TRACING_MINIMAL now defaults to true). Django ORM, cache, and template instrumentation are disabled by default to eliminate duplicate span creation since library integrations for database drivers (psycopg, MySQLdb, sqlite3), cache clients (redis, memcached), template renderers (Jinja2), and other supported libraries continue to be traced. This reduces performance overhead by removing redundant Django-layer instrumentation. To restore all Django instrumentation, set DD_DJANGO_TRACING_MINIMAL=false, or enable individual features using DD_DJANGO_INSTRUMENT_DATABASES=true, DD_DJANGO_INSTRUMENT_CACHES=true, and DD_DJANGO_INSTRUMENT_TEMPLATES=true.DD_DJANGO_INSTRUMENT_DATABASES=true (default false), database instrumentation now merges Django-specific tags into database driver spans created by supported integrations (psycopg, sqlite3, MySQLdb, etc.) instead of creating duplicate Django database spans. If the database cursor is not already wrapped by a supported integration, Django wraps it and creates a span. This change reduces overhead and duplicate spans while preserving visibility into database operations.ddtrace.settings package. Environment variables should be used to adjust settings.HttpPropagator.injectDEFAULT_RUNTIME_METRICS_INTERVAL.urllib3 and requests. It does not require enabling APM instrumentation for urllib3 anymore.threading.RLock (reentrant lock) profiling. The Lock profiler now tracks both threading.Lock and threading.RLock usage, providing comprehensive lock contention visibility for Python applications.version argument to LLMObs.pull_datasetversion and latest_version to provide information on the version of the dataset that is being worked with and the latest global version of the dataset, respectivelyLock, RLock, Event).HTTPS_PROXY.assessment argument in submit_evaluation().langchain integration would incorrectly mark Azure OpenAI calls as duplicate llm operations even if the openai integration was enabled.DD_PROFILING_API_TIMEOUT doesn't have any effect, and is marked to be removed in upcoming 4.0 release. New environment variable DD_PROFILING_API_TIMEOUT_MS is introduced to configure timeout for uploading profiles to the backend. The default value is 10000 ms (10 seconds)_acquire method of the Lock profiler (note: this only occurs when assertions are enabled.)version field. The version field is now omitted unless explicitly set by the user.
assessment now refers to whether the evaluation itself passes or fails according to your application, rather than the validity of the evaluation result.
The langchain integration will trace Azure OpenAI spans as workflow spans if there is an equivalent llm span from the openai integration.IndexError.async for model in client.models.list()) caused a TypeError: 'async for' requires an object with __aiter__ method, got coroutine. See issue #14574.KeyError exceptions in test runs when gevent is detected within the environment.ray.init().ray.data._internal to the module denylist.ValueError: Formatting field not found in record: 'dd.service'.ResourceWarning in multiprocess scenarios.wrapt library dependency from the Lock Profiler implementation, improving performance and reducing overhead during lock instrumentation.Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
contextvars) could cause use-after-free or double-free crashes (SIGSEGV) inside libddwaf. A per-context lock now serializes WAF calls on the same context.Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
gevent.joinall is called.Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
isDefined to result in an evaluation error.<!-- -->
TypeError during profiling. One example of this is neo4j's AsyncRLock, which inherits from asyncio.Lock: https://github.com/neo4j/neo4j-python-driver/blob/6.x/src/neo4j/_async_compat/concurrency.py#L45LLM Observability: Warning logs for incorrect usage of the LLM Observability SDK are deprecated and will be replaced with raised errors in ddtrace>=4.…
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
ddtrace>=4.0.0.<!-- -->
<!-- -->
profiling: This fix resolves an issue where memory profiler module fails to load when the system doesn't have libatomic installed.
profiling: This fix ensures the profiler now correctly tracks dependencies between Tasks and Coroutines that are awaiting or being awaited via asyncio.wait.
Resolves a potential deadlock when forking.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Deprecated support for Tornado versions older than v6.1. Use Tornado v6.1 or later.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
ddtrace.contrib.tornado module. Configure tracing using environment variables and import ddtrace.auto instead.AAP: This fix resolves an issue where the appsec layer was not compatible anymore with the lambda/serverless version of the tracer.
Code Security: Fixes critical memory safety issue in IAST when used with forked worker processes (MCP servers with Gunicorn and Uvicorn). Workers previously crashed with segmentation faults due to stale PyObject pointers in native taint maps after fork.
Dynamic instrumentation: fix issue with line probes matching the wrong source file when multiple source files from different Python path entries share the same name.
Exception replay: ensure exception information is captured when exceptions are raised by the GraphQL client library.
Lib-injection: do not inject into the gsutil tool
LLM Observability: Fixes an issue where the Google ADK integration would throw an AttributeError when trying to access the name or description attributes of a tool.
Profiling:
frame.f_locals while trying to retrieve class name of a PyFrameObject.Tracing:
IndexError in partial flush when the finished span counter was out of sync with actual finished spans.DD_TRACE_PARTIAL_FLUSH_MIN_SPANS values less than 1 now default to 1 with a warning.Span.finished setter is deprecated, use Span.finish() method instead.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Span.finished setter is deprecated, use Span.finish() method instead.Span.finish_with_ancestors() is deprecated with no alternative.ExperimentResult class' rows and summary_evaluations attributes are deprecated and will be removed in the next major release. ExperimentResult.rows/summary_evaluations attributes will only store the results of the first run iteration for multi-run experiments. Use the ExperimentResult.runs attribute instead to access experiment results and summary evaluations.runs argument, to assess the true performance of an experiment in the face of the non determinism of LLMs. Use the new ExperimentResult class' runs attribute to access the results and summary evaluations by run iteration.Lock, RLock, Event).ResourceWarning in multiprocess scenarios.wrapt library dependency from the Lock Profiler implementation, improving performance and reducing overhead during lock instrumentation.Span.finished setter is deprecated, use Span.finish() method instead.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Span.finished setter is deprecated, use Span.finish() method instead.Span.finish_with_ancestors() is deprecated with no alternative.ExperimentResult class' rows and summary_evaluations attributes are deprecated and will be removed in the next major release. ExperimentResult.rows/summary_evaluations attributes will only store the results of the first run iteration for multi-run experiments. Use the ExperimentResult.runs attribute instead to access experiment results and summary evaluations.runs argument, to assess the true performance of an experiment in the face of the non determinism of LLMs. Use the new ExperimentResult class' runs attribute to access the results and summary evaluations by run iteration.Lock, RLock, Event).ResourceWarning in multiprocess scenarios.wrapt library dependency from the Lock Profiler implementation, improving performance and reducing overhead during lock instrumentation.Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
CI Visibility: this fix resolves an issue where repo tags would be fetched while unshallowing to extract commit metadata, causing performance issues for repos with a large number of tags.
LLM Observability: fix support for HTTPS_PROXY.
Error Tracking: Modifies the way exception events are stored such that the exception id is stored instead of the exception object, to prevent TypeErrors with custom exception objects.
Span.set_struct_tag is deprecated and will be removed in v4.0.0 with no direct replacement.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
tiktoken library, and insteadstream_options={"include_usage": True} in the OpenAI request.Span.set_struct_tag is deprecated and will be removed in v4.0.0 with no direct replacement.Span.get_struct_tag is deprecated and will be removed in v4.0.0 with no direct replacement.Span.set_tag_str is deprecated and will be removed in version 4.0.0.
As an alternative to Span.set_tag_str, you can use Span.set_tag instead.DD_PROFILING_STACK_V2_ENABLED=false will no longer have an effect starting in 4.0.urllib3 and requests. It does not require enabling APM instrumentation for urllib3 anymore.threading.RLock (reentrant lock) profiling. The Lock profiler now tracks both threading.Lock and threading.RLock usage, providing comprehensive lock contention visibility for Python applications.version argument to LLMObs.pull_datasetversion and latest_version to provide information on the version of the dataset that is being worked with and the latest global version of the dataset, respectivelyversion field. The version field is now omitted unless explicitly set by the user.IndexError.assessment argument in submit_evaluation().
assessment now refers to whether the evaluation itself passes or fails according to your application, rather than the validity of the evaluation result.langchain integration would incorrectly mark Azure OpenAI calls as duplicate llm operations even if the openai integration was enabled.
The langchain integration will trace Azure OpenAI spans as workflow spans if there is an equivalent llm span from the openai integration.async for model in client.models.list()) caused a TypeError: 'async for' requires an object with __aiter__ method, got coroutine. See issue #14574.KeyError exceptions in test runs when gevent is detected within the environment.ray.init().ray.data._internal to the module denylist._acquire method of the Lock profiler (note: this only occurs when assertions are enabled.)DD_PROFILING_API_TIMEOUT doesn't have any effect, and is marked to be removed in upcoming 4.0 release. New environment variable DD_PROFILING_API_TIMEOUT_MS is introduced to configure timeout for uploading profiles to the backend. The default value is 10000 ms (10 seconds)ValueError: Formatting field not found in record: 'dd.service'.Span.set_struct_tag is deprecated and will be removed in v4.0.0 with no direct replacement.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
tiktoken library, and insteadstream_options={"include_usage": True} in the OpenAI request.Span.set_struct_tag is deprecated and will be removed in v4.0.0 with no direct replacement.Span.get_struct_tag is deprecated and will be removed in v4.0.0 with no direct replacement.Span.set_tag_str is deprecated and will be removed in version 4.0.0.
As an alternative to Span.set_tag_str, you can use Span.set_tag instead.DD_PROFILING_STACK_V2_ENABLED=false will no longer have an effect starting in 4.0.urllib3 and requests. It does not require enabling APM instrumentation for urllib3 anymore.threading.RLock (reentrant lock) profiling. The Lock profiler now tracks both threading.Lock and threading.RLock usage, providing comprehensive lock contention visibility for Python applications.version argument to LLMObs.pull_datasetversion and latest_version to provide information on the version of the dataset that is being worked with and the latest global version of the dataset, respectivelyversion field. The version field is now omitted unless explicitly set by the user.IndexError.assessment argument in submit_evaluation().
assessment now refers to whether the evaluation itself passes or fails according to your application, rather than the validity of the evaluation result.langchain integration would incorrectly mark Azure OpenAI calls as duplicate llm operations even if the openai integration was enabled.
The langchain integration will trace Azure OpenAI spans as workflow spans if there is an equivalent llm span from the openai integration.async for model in client.models.list()) caused a TypeError: 'async for' requires an object with __aiter__ method, got coroutine. See issue #14574.KeyError exceptions in test runs when gevent is detected within the environment.ray.init().ray.data._internal to the module denylist._acquire method of the Lock profiler (note: this only occurs when assertions are enabled.)DD_PROFILING_API_TIMEOUT doesn't have any effect, and is marked to be removed in upcoming 4.0 release. New environment variable DD_PROFILING_API_TIMEOUT_MS is introduced to configure timeout for uploading profiles to the backend. The default value is 10000 ms (10 seconds)ValueError: Formatting field not found in record: 'dd.service'.Span.set_struct_tag is deprecated and will be removed in v4.0.0 with no direct replacement.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Span.set_struct_tag is deprecated and will be removed in v4.0.0 with no direct replacement.Span.get_struct_tag is deprecated and will be removed in v4.0.0 with no direct replacement.Span.set_tag_str is deprecated and will be removed in version 4.0.0.
As an alternative to Span.set_tag_str, you can use Span.set_tag instead.DD_PROFILING_STACK_V2_ENABLED=false will no longer have an effect starting in 4.0.urllib3 and requests. It does not require enabling APM instrumentation for urllib3 anymore.threading.RLock (reentrant lock) profiling. The Lock profiler now tracks both threading.Lock and threading.RLock usage, providing comprehensive lock contention visibility for Python applications.version argument to LLMObs.pull_datasetversion and latest_version to provide information on the version of the dataset that is being worked with and the latest global version of the dataset, respectivelyversion field. The version field is now omitted unless explicitly set by the user.IndexError.assessment argument in submit_evaluation().
assessment now refers to whether the evaluation itself passes or fails according to your application, rather than the validity of the evaluation result.langchain integration would incorrectly mark Azure OpenAI calls as duplicate llm operations even if the openai integration was enabled.
The langchain integration will trace Azure OpenAI spans as workflow spans if there is an equivalent llm span from the openai integration.async for model in client.models.list()) caused a TypeError: 'async for' requires an object with __aiter__ method, got coroutine. See issue #14574.KeyError exceptions in test runs when gevent is detected within the environment.ray.init().ray.data._internal to the module denylist.DD_PROFILING_API_TIMEOUT doesn't have any effect, and is marked to be removed in upcoming 4.0 release. New environment variable DD_PROFILING_API_TIMEOUT_MS is introduced to configure timeout for uploading profiles to the backend. The default value is 10000 ms (10 seconds)ValueError: Formatting field not found in record: 'dd.service'.Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
CI Visibility: This fix addresses a performance issue where repository tags were fetched during the unshallow process to extract commit metadata, causing slowdowns in repositories with many tags.
LLM Observability: Resolves an issue in the bedrock integration where invoking cohere rerank models would result in missing spans due to output formatting index errors.
opentelemetry:
ray: This fix resolves an issue where the tracer raised an error when submitting Ray tasks without explicitly calling ray.init().
tracer: This fix resolves an issue where an application instrumented by ddtrace could crash at start. Fix compatibility with zope.event==6.0
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
version field. The version field is now omitted unless explicitly set by the user.<!-- -->
async for model in client.models.list()) caused a TypeError: 'async for' requires an object with __aiter__ method, got coroutine. See issue #14574.<!-- -->
IndexError.<!-- -->
<!-- -->
<!-- -->
ValueError: Formatting field not found in record: 'dd.service'.Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
CI Visibility: This fix resolves performance issue affecting coverage collection for Python 3.12+
LLM Observability
assessment argument in submit_evaluation().
assessment now refers to whether the evaluation itself passes or fails according to your application, rather than the validity of the evaluation result.langchain integration would incorrectly mark Azure OpenAI calls as duplicate llm operations even if the openai integration was enabled. The langchain integration will trace Azure OpenAI spans as workflow spans if there is an equivalent llm span from the openai integration.<!-- -->
<!-- -->
Pin to wrapt<2 until we can ensure full compatibility with the breaking changes.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
LLMObs.enableLLMObs.submit_evaluation_for() has been deprecated and will be removed in a future version. It will be replaced with LLMObs.submit_evaluation() which will take the signature of the original LLMObs.submit_evaluation_for() method in ddtrace version 4.0. Please use LLMObs.submit_evaluation() for submitting evaluations moving forward.
To migrate:
LLMObs.submit_evaluation_for(...) users: rename to LLMObs.submit_evaluation(...)LLMObs.submit_evaluation_for(...) users: rename the span_context argument to span, i.e. LLMObs.submit_evaluation(span_context={"span_id": ..., "trace_id": ...}, ...) to LLMObs.submit_evaluation(span={"span_id": ..., "trace_id": ...}, ...)Tracer.on_start_span and Tracer.deregister_on_start_span are deprecated and will be removed in v4.0.0 with no planned replacement.asyncio.create_task().asyncio and futures integrations are now enabled by default on LLMObs.enable(), which enables asynchronous context propagation for those libraries.LLMObs.submit_evaluation() and LLMObs.submit_evaluation_for() methods now accept a reasoning argument to denote an explanation of the evaluation results.parse() methods used for structured outputs.LLMObs.submit_evaluation_for() method now accepts a assessment argument to denote
whether or not the evaluation is valid or correct. Accepted values are either "pass" or "fail".parse() methods for structured outputs on chat.completions and responses endpoints (available in OpenAI SDK >= 1.92.0).track_user_id in the ATO SDK, which is equivalent to track_user but does not require the login, only the user id.requests with urllib3\<2.wrapt<2 until we can ensure full compatibility with the breaking changes.sumy package installs files under tests/* in site-packages, and this would cause any modules under tests.* to be considered third-party.AttributeError crash in certain configurations./v1/logs path is correctly added to prevent log payloads from being dropped by the Agent when using OTEL_EXPORTER_OTLP_ENDPOINT configuration. Metrics and traces are unaffected.Pin to wrapt<2 until we can ensure full compatibility with the breaking changes.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
LLMObs.enableLLMObs.submit_evaluation_for() has been deprecated and will be removed in a future version. It will be replaced with LLMObs.submit_evaluation() which will take the signature of the original LLMObs.submit_evaluation_for() method in ddtrace version 4.0. Please use LLMObs.submit_evaluation() for submitting evaluations moving forward.
To migrate:
LLMObs.submit_evaluation_for(...) users: rename to LLMObs.submit_evaluation(...)LLMObs.submit_evaluation_for(...) users: rename the span_context argument to span, i.e. LLMObs.submit_evaluation(span_context={"span_id": ..., "trace_id": ...}, ...) to LLMObs.submit_evaluation(span={"span_id": ..., "trace_id": ...}, ...)Tracer.on_start_span and Tracer.deregister_on_start_span are deprecated and will be removed in v4.0.0 with no planned replacement.asyncio.create_task().asyncio and futures integrations are now enabled by default on LLMObs.enable(), which enables asynchronous context propagation for those libraries.LLMObs.submit_evaluation() and LLMObs.submit_evaluation_for() methods now accept a reasoning argument to denote an explanation of the evaluation results.parse() methods used for structured outputs.LLMObs.submit_evaluation_for() method now accepts a assessment argument to denote
whether or not the evaluation is valid or correct. Accepted values are either "pass" or "fail".parse() methods for structured outputs on chat.completions and responses endpoints (available in OpenAI SDK >= 1.92.0).track_user_id in the ATO SDK, which is equivalent to track_user but does not require the login, only the user id.requests with urllib3\<2.wrapt<2 until we can ensure full compatibility with the breaking changes.sumy package installs files under tests/* in site-packages, and this would cause any modules under tests.* to be considered third-party.AttributeError crash in certain configurations./v1/logs path is correctly added to prevent log payloads from being dropped by the Agent when using OTEL_EXPORTER_OTLP_ENDPOINT configuration. Metrics and traces are unaffected.Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
version field. The version field is now omitted unless explicitly set by the user.<!-- -->
<!-- -->
<!-- -->
ValueError: Formatting field not found in record: 'dd.service'.Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
<!-- -->
langchain integration would incorrectly mark Azure OpenAI calls as duplicate llm operations even if the openai integration was enabled. The langchain integration will trace Azure OpenAI spans as workflow spans if there is an equivalent llm span from the openai integration.<!-- -->
Pin to wrapt<2 until we can ensure full compatibility with the breaking changes
wrapt<2 until we can ensure full compatibility with the breaking changesEstimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
sumy package installs files under tests/* in site-packages, and this would cause any modules under tests.* to be considered third-party.<!-- -->
<!-- -->
AttributeError crash in certain configurations.<!-- -->
<!-- -->
<!-- -->
<!-- -->
vertica: The vertica integration is deprecated and will be removed in a future version, around the same time that ddtrace drops support for Python 3.9…
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
opentelemetry: Adds default configurations for the OpenTelemetry Metrics API implementation to improve the Datadog user experience. This includes the following configurations:
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT is set to the default Datadog Agent endpoint, or localhost if not foundOTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE is set to deltaOTEL_METRIC_EXPORT_INTERVAL is set to 10000OTEL_METRIC_EXPORT_TIMEOUT is set to 7500LLM Observability: MCP integration also traces ClientSession contexts, ClientSession.initialize, and ClientSession.list_tools.
ray: This introduces a Ray core integration that traces Ray jobs, remote tasks, and actor method calls. Supported for Ray >= 2.46.0.
To enable tracing, start the Ray head with --tracing-startup-hook=ddtrace.contrib.ray:setup_tracing then submit jobs as usual.
AttributeError that could occur when tracing Google ADK agent runs, due to the agent model attribute not being defined for SequentialAgent class.ImportError for when using langchain_core>=0.3.76.DD_APM_TRACING_ENABLED=0 when using LLM Observability.vertica: The vertica integration is deprecated and will be removed in a future version, around the same time that ddtrace drops support for Python 3.9…
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
opentelemetry: Adds default configurations for the OpenTelemetry Metrics API implementation to improve the Datadog user experience. This includes the following configurations:
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT is set to the default Datadog Agent endpoint, or localhost if not foundOTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE is set to deltaOTEL_METRIC_EXPORT_INTERVAL is set to 10000OTEL_METRIC_EXPORT_TIMEOUT is set to 7500LLM Observability: MCP integration also traces ClientSession contexts, ClientSession.initialize, and ClientSession.list_tools.
ray: This introduces a Ray core integration that traces Ray jobs, remote tasks, and actor method calls. Supported for Ray >= 2.46.0.
To enable tracing, start the Ray head with --tracing-startup-hook=ddtrace.contrib.ray:setup_tracing then submit jobs as usual.
AttributeError that could occur when tracing Google ADK agent runs, due to the agent model attribute not being defined for SequentialAgent class.ImportError for when using langchain_core>=0.3.76.vertica: The vertica integration is deprecated and will be removed in a future version, around the same time that ddtrace drops support for Python 3.9…
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
opentelemetry: Adds default configurations for the OpenTelemetry Metrics API implementation to improve the Datadog user experience. This includes the following configurations:
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT is set to the default Datadog Agent endpoint, or localhost if not foundOTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE is set to deltaOTEL_METRIC_EXPORT_INTERVAL is set to 10000OTEL_METRIC_EXPORT_TIMEOUT is set to 7500LLM Observability: MCP integration also traces ClientSession contexts, ClientSession.initialize, and ClientSession.list_tools.
ray: This introduces a Ray core integration that traces Ray jobs, remote tasks, and actor method calls. Supported for Ray >= 2.46.0.
To enable tracing, start the Ray head with --tracing-startup-hook=ddtrace.contrib.ray:setup_tracing then submit jobs as usual.
AttributeError that could occur when tracing Google ADK agent runs, due to the agent model attribute not being defined for SequentialAgent class.ImportError for when using langchain_core>=0.3.76.Pin to wrapt<2 until we can ensure full compatibility with the breaking changes.
Pin to wrapt<2 until we can ensure full compatibility with the breaking changes.
CI Visibility: This fix resolves an issue where tests would be incorrectly detected as third-party code if a third-party package containing a folder with the same name as the tests folder was installed. For instance, the sumy package installs files under tests/* in site-packages, and this would cause any modules under tests.* to be considered third-party.
langchain: Resolves an issue where langchain patching would throw an ImportError for when using langchain_core>=0.3.76.
Untrusted Serialization detection, which will be displayed on your DataDog Vulnerability Explorer dashboard. See the Application Vulnerability Managem…
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
DD_DJANGO_TRACING_MINIMAL environment variable for performance-sensitive applications. When enabled, this disables Django ORM, cache, and template instrumentation while keeping middleware instrumentation enabled. This significantly reduces overhead by removing Django-specific spans while preserving visibility into the underlying database drivers, cache clients, and other integrations. For example, with this enabled, Django ORM query spans are disabled but database driver spans (e.g., psycopg, MySQLdb) will still be created. To enable minimal tracing, set DD_DJANGO_TRACING_MINIMAL=true.aws.partition tag onto AWS traces based on the region for the boto, botocore, and aiobotocore integrations.DD_CIVISIBILITY_ENABLED (with default value True) so it can be disabled to avoid sending traces to the Test Visibility product from the test runners.DD_TRACE_RESOURCE_RENAMING_ENABLED="true"usedforsecurity=False.tags and chat_template, and a new Prompt TypedDict class that would be used in annotation and annotation_context.DatadogSampler from getting recreated whenever the SpanAggregator is reset, and instead updates the rate limiter that the sampler uses.PyErr_Format.-m entries in the denylist.asyncio.Tasks.PyObject_Realloc call, which can lead to accessing freed memory.--skip-atexit is not set when --lazy or --lazy-apps is set on uWSGI<2.0.30. This is to prevent crashes from profiling native extension modules. See https://github.com/unbit/uwsgi/pull/2726 for details.c_char type raised a TypeError, this now uses the 'c' typecode for better compatibility across versions.DD_GIT_COMMIT_SHA and DD_GIT_REPOSITORY_URL are defined before using the git command.Untrusted Serialization detection, which will be displayed on your DataDog Vulnerability Explorer dashboard. See the Application Vulnerability Managem…
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
DD_DJANGO_TRACING_MINIMAL environment variable for performance-sensitive applications. When enabled, this disables Django ORM, cache, and template instrumentation while keeping middleware instrumentation enabled. This significantly reduces overhead by removing Django-specific spans while preserving visibility into the underlying database drivers, cache clients, and other integrations. For example, with this enabled, Django ORM query spans are disabled but database driver spans (e.g., psycopg, MySQLdb) will still be created. To enable minimal tracing, set DD_DJANGO_TRACING_MINIMAL=true.aws.partition tag onto AWS traces based on the region for the boto, botocore, and aiobotocore integrations.DD_CIVISIBILITY_ENABLED (with default value True) so it can be disabled to avoid sending traces to the Test Visibility product from the test runners.DD_TRACE_RESOURCE_RENAMING_ENABLED="true"usedforsecurity=False.tags and chat_template, and a new Prompt TypedDict class that would be used in annotation and annotation_context.DatadogSampler from getting recreated whenever the SpanAggregator is reset, and instead updates the rate limiter that the sampler uses.PyErr_Format.-m entries in the denylist.asyncio.Tasks.PyObject_Realloc call, which can lead to accessing freed memory.--skip-atexit is not set when --lazy or --lazy-apps is set on uWSGI<2.0.30. This is to prevent crashes from profiling native extension modules. See https://github.com/unbit/uwsgi/pull/2726 for details.c_char type raised a TypeError, this now uses the 'c' typecode for better compatibility across versions.DD_GIT_COMMIT_SHA and DD_GIT_REPOSITORY_URL are defined before using the git command.Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 08-2026 See the support level definitions for more information.
AAP:
CI Visibility: This fix solves an issue where the ITR skip count metric was aggregating skipped tests even when skipping level was set to suite. It will now count appropriately (skipped suites or skipped tests) depending on ITR skip level.
Estimated end-of-life date, accurate to within three months: 09-2026 See the support level definitions for more information.
Estimated end-of-life date, accurate to within three months: 09-2026 See the support level definitions for more information.
<!-- -->
<!-- -->
python -m and deny py_compile.Your coding agent can read these notes before it upgrades. Set up the MCP server →