NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI, #486 most downloaded on PyPI
Datadog APM client library
Latest stable
4.17.2
Last release 3 days ago
19 Nov 2026
Ships on a steady schedule
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
5 versions withdrawn
withdrawn after publishing
10 years old
871 releases · first in 2017
Drag across the ruler to compare, or tap a release to open it.
This fix resolves an issue where track_user was generating additional unexpected security activity for customers.
TypeError in encoding when truncating a large bytes object.DD_LLMOBS_ML_APP missing errors when LLM Observability was disabled.Algoliasearch: Fix for potential dangling reference that could trhow an exception. -
os.system or subprocess.Popen could return the wrong exception type.Your coding agent can read these notes before it upgrades. Set up the MCP server →
This fix resolves an issue in which traced nested generator functions had their execution order subtly changed in a way that affected the stack unwind
for v in g: yield v during the wrapping of generator functions where
full bidrectional communication with the sub-generator via
yield from g was appropriate. See PEP380 for an explanation of how
these two generator uses differ.Fix an issue where the trace ID exported from export_span was incorrect.
export_span was incorrect.Unvalidated Redirect detection for Django, Flask and FastAPI applications, which will be displayed on your DataDog Vulnerability Explorer dashboard. S…
completion and text_completion router methods.ml_app of the most recent LLM Observability span (or the global ml_app) when injecting distributed headers. In distributed services, uses the ml_app from the distributed trace headers.DD_DYNAMIC_INSTRUMENTATION_REDACTION_EXCLUDED_IDENTIFIERSDD_TAGS.LLM Observability
astream_events on a compiled graph would raise a KeyError.CI Visibility
crewai
TypeError to be thrown due to empty task contexts.tracing
ddtrace.trace.Context fails in coroutines. This regression was introduced in v3.7.0.TraceFilter and tracer.configure.unicode string is too large error.openai
None would result in unfinished spans.dynamic instrumentation
langgraph
astream_events on a compiled graph would cause missing spans.This fix resolves an issue where LLM interactions were not being traced when a non-default base URL was provided for the Anthropic, Bedrock, LangChain
This introduces report links to the pytest plugin. At the end of a test session, ddtrace shows links to the Datadog Test Optimization pages with the t
tracer.wrap(). Previously, calling tracer.current_span() inside a wrapped generator function would return None, leading to AttributeError when interacting with the span. Additionally, traces reported to Datadog showed incorrect durations, as span context was not maintained across generator iteration. This change ensures that tracer.wrap() now correctly handles both sync and async generators by preserving the tracing context throughout their execution and finalizing spans correctly. Users can now safely use tracer.current_span() within generator functions and expect accurate trace reporting.Adds a new configuration option DD_TRACE_SAFE_INSTRUMENTATION_ENABLED to enable safer patching of integrations.
When enabled, the tracer will check if the installed version of an integration is compatible with the explicit version range supported by that integration. If an incompatible version is detected, the integration will not be patched and an error message will be logged.
This feature is currently disabled by default, but will be enabled by default in a future version.
Currently the only supported integrations for this feature are: aiobotocore (>=1.0.0), fastapi (>=0.57.0), and elasticsearch (>=1.10).
profiling
docs
--lazy-apps config. For uWSGI<2.0.30 when --lazy-apps is set , we advise our customers to also set --skip-atexit to avoid crashes that could occur from our native extensions when worker processes are terminated.Fix resolves an issue where running from a GitHub action triggered on a tag push would cause the branch name to be null, causing errors when fetching
<!-- -->
Avoid excessive filtering of stacktrace locations when finding vulnerabilities. After this change, vulnerabilities that were previously discarded will…
DD_ERROR_TRACKING_HANDLED_ERRORS=allthird_partyDD_ERROR_TRACKING_HANDLED_ERRORS_INCLUDE=module1, module2, module3.submoduletracing
unicode string is too large error.ddtrace.trace.Context fails in coroutines. This regression was introduced in v3.7.0.CI Visibility
DD_CIVISIBILITY_ITR_ENABLED was not honored properly.profiling
SynchronizedSamplePool where pool could be null when calling into ddog_ArrayQueue_ functions, leading to segfaults in the uWSGI shutdownCode Security
LLM Observability
dynamic instrumentation
kafka
UNKNOWN_SERVER_ERROR (-1).code origin for spans
LLM Observability: add processor capability to process span inputs and outputs. See usage documentation \here\.
unicode string is too large error.Resolves an issue where the DD_CIVISIBILITY_ITR_ENABLED was not honored properly.
CI Visibility
DD_CIVISIBILITY_ITR_ENABLED was not honored properly.Tracing
ddtrace.trace.Context fails in coroutines. This regression was introduced in v3.7.0.Avoids excessive filtering of stacktrace locations when finding vulnerabilities. After this change, vulnerabilities that were previously discarded wil…
CI Visibility
Code Security
langchain v0.1.0 and above.Error Tracking
DD_ERROR_TRACKING_HANDLED_ERRORS=user|third_party|allDD_ERROR_TRACKING_HANDLED_ERRORS_INCLUDE=module1, module2, module3.submoduleLLM Observability
openai: Introduces tracing support for the OpenAI Responses endpoint.CI Visibility
pytest-xdist would not exit with the proper status code if ATR was enabled.xdist would report test suites as failing even when all tests pass.Code Origin
Code Security (IAST)
Dynamic Instrumentation
LLM Observability
Profiling
SynchronizedSamplePool where pool could be null when calling into ddog_ArrayQueue_ functions, leading to segfaults in the uWSGI shutdownTracing
kafka: Resolves an issue where message headers were sent to Kafka brokers that do not support them. Message headers are turned off when the Kafka server responds with UNKNOWN_SERVER_ERROR (-1).CI Visibility: This fix resolves an issue where the DD_CIVISIBILITY_ITR_ENABLED was not honored properly.
unicode string is too large error.CI Visibility: This fix resolves an issue where pytest-xdist would not exit with the proper status code if ATR was enabled.
The field representing the class name in IAST vulnerability location reporting was previously incorrectly named as class_name. This fix standardizes t…
AAP (ASM is now AAP)
ddtrace.appsec.track_user_sdk for manual instrumentation.CI Visibility
LLM Observability
Tracing
baggage. prefix The DD_TRACE_BAGGAGE_TAG_KEYS configuration allows users to specify a comma-separated list of baggage keys for span tagging, by default the value is set to user.id,account.id,session.id. When set to \*, all baggage keys will be converted into span tags. Setting it to an empty value disables baggage tagging.Code Security
class_name. This fix standardizes the naming and ensures that the correct field name is used (class).Dynamic Instrumentation
LLM Observability
openai: This fix resolves an issue where using client.beta.chat.completions.stream with openai patching caused an attribute error.client.beta.chat.completions.stream with LLM Observability enabled caused an attribute errorconverse_stream calls.converse_stream calls contained an extra empty output message.Profiling
Tracing
azure_functions: Resolves an issue where async functions throw an error when instrumented.datastreams: Resolves an issue where failure to decode the data streams context caused infinite loops in data streams checkpoints.futures: Resolves an edge case where trace context was not propagated to spans generated by the ThreadPoolExecutor.kafka: Fixes an issue where a producer or consumer initialized with an unpacked config resulted in TypeError, causing a failed connection. confluent-kafka supports both unpacked and packed config; this change allows initialization with either.telemetry: Improves periodic telemetry writer performance by removing unnecessary calls to importlib.metadata for reporting imported dependencies.AttributeError when reinitializing the DatadogSampler. This prevented sampling rules from being reset. Note: This only affected cases where sampling rules were an empty list. It did not impact cases with at least one rule or when rules were set to null.?* was not being matched correctly for DD_TRACE_SAMPLING_RULES tags, due to it matching on spans with no tag matching the specified key.RuntimeWarning from an unwaited coroutine during tab completion in IPython REPL when asyncio integration is active. Tracer now wraps an asyncio coroutine only when there is an active trace context.The field representing the class name in IAST vulnerability location reporting was previously incorrectly named as class_name. This fix standardizes t…
Code Security
class_name. This fix standardizes the naming and ensures that the correct field name is used (class).Tracing
futures: Resolves an edge case where trace context was not propagated to spans generated by the ThreadPoolExecutor.RuntimeWarning from an unwaited coroutine during tab completion in IPython REPL when asyncio integration is active. Tracer now wraps an asyncio coroutine only when there is an active trace context.Introduces a new user event sdk available through ddtrace.appsec.track_user_sdk for manual instrumentation.
AAP (ASM is now AAP)
ddtrace.appsec.track_user_sdk for manual instrumentation.LLM Observability
Tracing
baggage. prefix The DD_TRACE_BAGGAGE_TAG_KEYS configuration allows users to specify a comma-separated list of baggage keys for span tagging, by default the value is set to user.id,account.id,session.id. When set to \*, all baggage keys will be converted into span tags. Setting it to an empty value disables baggage tagging.Dynamic Instrumentation
LLM Observability
openai: This fix resolves an issue where using client.beta.chat.completions.stream with openai patching caused an attribute error.client.beta.chat.completions.stream with LLM Observability enabled caused an attribute errorconverse_stream calls.converse_stream calls contained an extra empty output message.Profiling
Tracing
azure_functions: Resolves an issue where async functions throw an error when instrumented.datastreams: Resolves an issue where failure to decode the data streams context caused infinite loops in data streams checkpoints.kafka: Fixes an issue where a producer or consumer initialized with an unpacked config resulted in TypeError, causing a failed connection. confluent-kafka supports both unpacked and packed config; this change allows initialization with either.telemetry: Improves periodic telemetry writer performance by removing unnecessary calls to importlib.metadata for reporting imported dependencies.AttributeError when reinitializing the DatadogSampler. This prevented sampling rules from being reset. Note: This only affected cases where sampling rules were an empty list. It did not impact cases with at least one rule or when rules were set to null.?* was not being matched correctly for DD_TRACE_SAMPLING_RULES tags, due to it matching on spans with no tag matching the specified key.Upgrades echion, improving the performance of stack sampler by replacing proc filesystem reads with clock_gettime().
Profiling
echion, improving the performance of stack sampler by replacing proc filesystem reads with clock_gettime().echion, improving the performance of the stack sampler by reusing memory and reducing the frequency of memory allocation.Tracing
LLM Observability
session_id on a LLM Observability span via LLMObs.annotate/annotation_context() as a tag only set it as a tag on the span, but did not update the span's actual session ID value.Profiling
process_vm_readv() syscalls for Python frame objects for Python versions 3.11+.echion to pick up a fix which resolves a potential crash. The stack sampler could read off of an empty queue of frames after failing to resolve specific frame information, triggering undefined behavior.Tracing
x-datadog-sampling-priority header. The sampling decision is now made using the updated sampling formula.Upgrades echion, improving the performance of stack sampler by replacing proc filesystem reads with clock_gettime().
Profiling
echion, improving the performance of stack sampler by replacing proc filesystem reads with clock_gettime().echion, improving the performance of the stack sampler by reusing memory and reducing the frequency of memory allocation.Tracing
LLM Observability
session_id on a LLM Observability span via LLMObs.annotate/annotation_context() as a tag only set it as a tag on the span, but did not update the span's actual session ID value.Profiling
process_vm_readv() syscalls for Python frame objects for Python versions 3.11+.echion to pick up a fix which resolves a potential crash. The stack sampler could read off of an empty queue of frames after failing to resolve specific frame information, triggering undefined behavior.Tracing
x-datadog-sampling-priority header. The sampling decision is now made using the updated sampling formula.azure_functions: Resolves an issue where spans generated from timer triggers did not include the trigger type in the resource name.
azure_functions: Resolves an issue where spans generated from timer triggers did not include the trigger type in the resource name.opentracing: Support for OpenTracing has been deprecated and will be removed in v4.0.0. Please use the OpenTelemetry API instead.
opentracing: Support for OpenTracing has been deprecated and will be removed in v4.0.0. Please use the OpenTelemetry API instead.DD_RUNTIME_METRICS_INTERVAL is deprecated and will be removed in the 4.0.0 release.LLM Observability
converse_stream calls to the Bedrock Converse API. converse_stream calls are traced as LLM spans with max_tokens, temperature inference parameters and input/output messages.botocore: Introduces capturing prompts, token usage, and inference parameters for converse_stream calls to the Bedrock API.crewai: Introduces tracing support for the CrewAI SDK, specifically tracing crew, task, agent, and tool executions. See the docs for more information.litellm: Introduces tracing support for the LiteLLM Python SDK's synchronous and asynchronous completion and text_completion methods. See the docs for more information.openai_agents: Introduces tracing support for OpenAI Agents SDK. Agent, guardrail, tool, handoff, response, and custom operations are traced. See the docs for more information.Tracing
Support for Database Monitoring (DBM) propagation in pymongo. Two propagation modes are supported:
This feature can be controlled using the DD_DBM_PROPAGATION_MODE environment variable with values full, service, or disabled. Example of enabling full or service propagation mode: DD_DBM_PROPAGATION_MODE=full|service
Note that this feature requires PyMongo 3.9 or higher.
A subset of the tracer configuration is now stored on disk throughout the tracer's lifetime. This enables the monitoring of which processes are instrumented.
azure_functions: Introduces tracing support for timer triggers.
langgraph: Introduces tracing support for LangGraph, specifically tracing graph and node executions. See the docs for more information.
Code Security
CI Visibility
CODEOWNERS file of the project being tested was looked up in the current working directory rather than the repository root.I/O operation on closed file error was raised while printing ddtrace logs during a pytest session, due to a conflict with pytest's log capture.Profiling
_self_acquired_at attribute.Tracing
Django, Flask, Cherrypy, wsgi, asgi, pyramid, molten, falcon, tornado, aiohttp, bottle, rq, and sanic integrations were unnecessarily running code to activate distributed tracing headers multiple times per request. This is fixed by only activating distributed tracing headers once per request.azure_functions: Resolves an issue where the function name set by the function_name decorator was ignored.azure_functions: Resolves an issue where a function would not start if the trigger_arg_nameparameter was set to anything other than req.azure_functions: Resolves an issue where timer triggers had a span kind of server instead of internalgraphql: Resolves AttributeError raised when used with older versions of graphql-core that do not have formatted property for SourceLocation.opentelemetry: Updates trace flags in opentelemetry.trace.Span.get_span_context() to match OpenTelemetry specification.subprocess: Removes unnecessary debug log variables.opentracing: Support for OpenTracing has been deprecated and will be removed in v4.0.0. Please use the OpenTelemetry API instead.
opentracing: Support for OpenTracing has been deprecated and will be removed in v4.0.0. Please use the OpenTelemetry API instead.DD_RUNTIME_METRICS_INTERVAL is deprecated and will be removed in the 4.0.0 release.LLM Observability
converse_stream calls to the Bedrock Converse API. converse_stream calls are traced as LLM spans with max_tokens, temperature inference parameters and input/output messages.botocore: Introduces capturing prompts, token usage, and inference parameters for converse_stream calls to the Bedrock API.crewai: Introduces tracing support for the CrewAI SDK, specifically tracing crew, task, agent, and tool executions. See the docs for more information.litellm: Introduces tracing support for the LiteLLM Python SDK's synchronous and asynchronous completion and text_completion methods. See the docs for more information.openai_agents: Introduces tracing support for OpenAI Agents SDK. Agent, guardrail, tool, handoff, response, and custom operations are traced. See the docs for more information.Tracing
Support for Database Monitoring (DBM) propagation in pymongo. Two propagation modes are supported:
This feature can be controlled using the DD_DBM_PROPAGATION_MODE environment variable with values full, service, or disabled. Example of enabling full or service propagation mode: DD_DBM_PROPAGATION_MODE=full|service
Note that this feature requires PyMongo 3.9 or higher.
A subset of the tracer configuration is now stored on disk throughout the tracer's lifetime. This enables the monitoring of which processes are instrumented.
azure_functions: Introduces tracing support for timer triggers.
langgraph: Introduces tracing support for LangGraph, specifically tracing graph and node executions. See the docs for more information.
Code Security
CI Visibility
CODEOWNERS file of the project being tested was looked up in the current working directory rather than the repository root.I/O operation on closed file error was raised while printing ddtrace logs during a pytest session, due to a conflict with pytest's log capture.Profiling
_self_acquired_at attribute.Tracing
Django, Flask, Cherrypy, wsgi, asgi, pyramid, molten, falcon, tornado, aiohttp, bottle, rq, and sanic integrations were unnecessarily running code to activate distributed tracing headers multiple times per request. This is fixed by only activating distributed tracing headers once per request.azure_functions: Resolves an issue where the function name set by the function_name decorator was ignored.azure_functions: Resolves an issue where a function would not start if the trigger_arg_nameparameter was set to anything other than req.azure_functions: Resolves an issue where timer triggers had a span kind of server instead of internalgraphql: Resolves AttributeError raised when used with older versions of graphql-core that do not have formatted property for SourceLocation.opentelemetry: Updates trace flags in opentelemetry.trace.Span.get_span_context() to match OpenTelemetry specification.subprocess: Removes unnecessary debug log variables.Fixes an infinite recursion from calling logging module when lock profiler fails to delete _self_acquired_at attribute.
Profiling
_self_acquired_at attribute.Tracing
Fixes an issue with PosixPath handling in path operations that could cause errors during taint tracking. This fix improves stability and slightly redu
patch_all is deprecated. As an alternative to patch_all, you can use import ddtrace.auto along with DD_PATCH_MODULES if specific module patching is ne…
echion which includes performance improvements.patch_all is deprecated. As an alternative to patch_all, you can use import ddtrace.auto along with DD_PATCH_MODULES if specific module patching is necessary.Dynamic Instrumentation
LLM Observability
integration tag to LLM Observability spans that are generated by an integration.agentless_enabled and DD_LLMOBS_AGENTLESS_ENABLED now defaults to None, and will be determined automatically based on the presence of a compatible Datadog agent. agentless_enabled and DD_LLMOBS_AGENTLESS_ENABLED can still be set explicitly to True or False</span>, but is now optional for agentless users.Profiling
Tracing
DD_TRACE_NATIVE_SPAN_EVENTS (disabled by default). This requires agent version 7.63.0 or later.http.referrer_hostname tag.Fixes an issue with gevent support and the typing module on CPython >= 3.12.
ASM
NotImplementedError that occurred when trying to deepcopy wrapped builtin functions (like open) while ASM or IAST were enabled. The error was caused by the wrapper not implementing the __deepcopy__ method.CI Visibility
Lib-Injection
LLM Observability
agentless_enabled=True or DD_LLMOBS_AGENTLESS_ENABLED=true.openai: Avoids creating spans for streamed Open AI chat and completion requests that use with_raw_response since the stream cannot be traced in these cases, leading to unfinished spans.Profiling
AttributeError: '_ProfiledThreadingLock' object has no attribute '_self_acquired_at'.Single-Step Instrumentation
Tracing
graphql: Fixes an issue in GraphQL patching that threw an error when locations (an Optional field) is None. This checks for locations before setting attributes.Nothing published for this version
patch_all is deprecated. As an alternative to patch_all, you can use import ddtrace.auto along with DD_PATCH_MODULES if specific module patching is ne…
echion which includes performance improvements.patch_all is deprecated. As an alternative to patch_all, you can use import ddtrace.auto along with DD_PATCH_MODULES if specific module patching is necessary.Dynamic Instrumentation
LLM Observability
integration tag to LLM Observability spans that are generated by an integration.agentless_enabled and DD_LLMOBS_AGENTLESS_ENABLED now defaults to None, and will be determined automatically based on the presence of a compatible Datadog agent. agentless_enabled and DD_LLMOBS_AGENTLESS_ENABLED can still be set explicitly to True or False</span>, but is now optional for agentless users.Profiling
Tracing
DD_TRACE_NATIVE_SPAN_EVENTS (disabled by default). This requires agent version 7.63.0 or later.http.referrer_hostname tag.Fixes an issue with gevent support and the typing module on CPython >= 3.12.
ASM
NotImplementedError that occurred when trying to deepcopy wrapped builtin functions (like open) while ASM or IAST were enabled. The error was caused by the wrapper not implementing the __deepcopy__ method.CI Visibility
Lib-Injection
LLM Observability
agentless_enabled=True or DD_LLMOBS_AGENTLESS_ENABLED=true.openai: Avoids creating spans for streamed Open AI chat and completion requests that use with_raw_response since the stream cannot be traced in these cases, leading to unfinished spans.Profiling
AttributeError: '_ProfiledThreadingLock' object has no attribute '_self_acquired_at'.Single-Step Instrumentation
Tracing
graphql: Fixes an issue in GraphQL patching that threw an error when locations (an Optional field) is None. This checks for locations before setting attributes.Fixes an issue with PosixPath handling in path operations that could cause errors during taint tracking. This fix improves stability and slightly redu
Avoids zombie process from telemetry sender on startup.
Lib-Injection
LLM Observability
agentless_enabled=True or DD_LLMOBS_AGENTLESS_ENABLED=true.Fixes a NotImplementedError that occurred when trying to deepcopy wrapped builtin functions (like open) while ASM or IAST were enabled. The error was
ASM
NotImplementedError that occurred when trying to deepcopy wrapped builtin functions (like open) while ASM or IAST were enabled. The error was caused by the wrapper not implementing the __deepcopy__ method.CI Visibility
Lib-Injection
LLM Observability
Includes dynamodb in the default list of cloud services for payload tagging, i.e. DD_TRACE_CLOUD_PAYLOAD_TAGGING_SERVICES. Note that cloud services pa
Includes dynamodb in the default list of cloud services for payload tagging, i.e. DD_TRACE_CLOUD_PAYLOAD_TAGGING_SERVICES. Note that cloud services payload tagging feature is still gated by DD_TRACE_CLOUD_REQUEST_PAYLOAD_TAGGING and DD_TRACE_CLOUD_RESPONSE_PAYLOAD_TAGGING.
LLM Observability: This introduces tracing for converse calls to the Bedrock Converse API. converse calls are traced as
LLM spans with max_tokens, temperature inference parameters and input/output messages.
botocore: This introduces capturing prompts, token usage, and inference parameters for converse calls to the Bedrock API.
SCA: This add support for extended heartbeat every 24h with dependencies payload.
kafka: Adds messaging.destination.name tag to kafka produce and consume spans.
LLM Observability: Avoids submitting spans to LLM Observability when a non-default base URL is detected in request for the Anthropic, Bedrock, LangChain, Open AI, and Azure Open AI integrations.
ASM: This introduces support for automatic instrumentation of session monitoring and blocking for Django.
ASM: This introduces new metrics span tags to keep track of truncations of WAF payloads.
runtime_metrics: Adds support for sending runtime metrics as gauge metrics (instead of distributions). To enable this feature set DD_TRACE_EXPERIMENTAL_FEATURES_ENABLED=DD_RUNTIME_METRICS_ENABLED.
runtime_metrics: Adds support for tagging runtime metrics with the current runtime ID. To enable tagging, set DD_TRACE_EXPERIMENTAL_RUNTIME_ID_ENABLED=True.
SCA: This fix resolves an issue where some dependencies where reported with an inaccurate name.
profiling: This fix resolves an issue where the Lock profiler would throw an AttributeError: '_ProfiledThreadingLock' object has no attribute '_self_acquired_at'.
tags: Updates DD_TAGS parsing to match the Datadog Agent and other SDKs. Key-value pairs can now be separated by both commas and spaces (e.g., DD_TAGS="key1:value1, key2:value2, key3:value3"). This reverts a change introduced in v2.11.0, which only allowed one separator type.
dynamic sampling: Ensures that dynamic sampling rates are always consistent with the rates received from the agent via remote configuration. Previously, setting a global sample rate and then sampling rules via remote config would result in both configurations being applied. With this fix only the most recent sampling configuration will be applied.
LLM Observability: This fix resolves an issue where enabling LLM Observability caused all botocore submodules to be patched instead
of only bedrock-runtime.
LLM Observability: Resolves issue where Pydantic model objects were encoded as unhelpful default placeholder text.
LLM Observability: Improves encoding of non-JSON serializable I/O objects by attempting to convert to string before resorting to default placeholder text.
tracing: Resolves an issue where trace information, such as span links, baggage, and trace-level tags (ex: sampling decision maker), could be lost when a new process was created while a trace was active.
internal: Fixes an issue where trimming a traceback to attach it to the span could result in the loss of the most recent frames.
pylibmc: fixes an issue where using Client(server=[url]) would throw the error __init__() got multiple values for argument 'servers'
tracing: Ensures remote sampling rules no longer overwrite existing samplers, preserving service-based sample rates set by the Agent and ensuring tracer rate limits configured by ddtrace products (e.g., ASM) are always respected.
DD_TRACE_AGENT_URLDD_DOGSTATSD_URLDD_TRACE_AGENT_HOSTNAMEDD_AGENT_HOSTDD_TRACE_AGENT_PORTDD_DOGSTATSD_PORTDD_AGENT_PORTDD_TRACE_AGENT_TIMEOUT_SECONDSOf the above impacted configurations, these had a precedence change:
DD_TRACE_AGENT_HOSTNAME over DD_AGENT_HOSTDD_TRACE_AGENT_PORT over DD_AGENT_PORTDD_DOGSTATSD_HOST over DD_AGENT_HOSTDD_DOGSTATSD_PORT over DD_AGENT_PORTSCA: Resolves an issue where some dependencies where reported with an inaccurate name.
ASM
Profiling
AttributeError: '_ProfiledThreadingLock' object has no attribute '_self_acquired_at'.Fixes an issue with PosixPath handling in path operations that could cause errors during taint tracking. This fix improves stability and slightly redu
Code Security
Lib-injection
LLM Observability
agentless_enabled=True or DD_LLMOBS_AGENTLESS_ENABLED=true.Tracing
Fixes a NotImplementedError that occurred when trying to deepcopy wrapped builtin functions (like open) while ASM or IAST were enabled. The error was
ASM
NotImplementedError that occurred when trying to deepcopy wrapped builtin functions (like open) while ASM or IAST were enabled. The error was caused by the wrapper not implementing the __deepcopy__ method.CI Visibility
LLM Observability
Profiling
AttributeError: '_ProfiledThreadingLock' object has no attribute '_self_acquired_at'.Tracing
Client(server=[url]) would throw the error __init__() got multiple values for argument 'servers'Fix for release script causing lib injection OCI images to not get published.
Nothing published for this version
Fix for release script causing lib injection OCI images to not get published.
This release is missing the OCI images due to issues with the release pipeline. If you need an OCI image with the features introduced in the 3.2 relea
This release is missing the OCI images due to issues with the release pipeline. If you need an OCI image with the features introduced in the 3.2 release line, please use v3.2.1 instead.
DD_EXPERIMENTAL_APPSEC_STANDALONE_ENABLED with DD_APM_TRACING_ENABLED. Feature set remains the same.Bumps libdatadog dependency to v16.0.3.
Bumps libdatadog dependency to v16.0.3.
ASM
DD_EXPERIMENTAL_APPSEC_STANDALONE_ENABLED with DD_APM_TRACING_ENABLED. Feature set remains the same.CI Visibility
sys.monitoring tools such as pytest-cov from being used.Profiling
Tracing
celery: When multiple broker URLs are provided as a list, use the first broker URL from a list to avoid parsing errors.SCA: Resolves an issue where some dependencies where reported with an inaccurate name.
ASM
CI Visibility
pytest-cov from being used.LLM Observability
Profiling
AttributeError: '_ProfiledThreadingLock' object has no attribute '_self_acquired_at'.Tracing
record_exception call as it was not following OTEL semantics.In some applications, unique cookie names are used per request, leading to a high number of unique cookie vulnerabilities. This fix improves detection…
Dynamic Instrumentation
LLM Observability
Deepseek for the model provider and span name for LLM Observability spans for calls to Deepseek models using the OpenAI SDK.LLMObs.export_span(), LLMObs.inject_distributed_headers(), LLMObs.annotate() to default to the current active LLM Observability-specific span if span is not provided.concurrent.futures.thread module.metadata argument to LLMObs.submit_evaluation_for, which enables users to attach arbitrary structured metadata to custom evaluations.Tracing
api gateway: Adds preview support to create inferred AWS API Gateway spans when certain AWS Gateway headers are passed into the request. Requires DD_TRACE_INFERRED_PROXY_SERVICES_ENABLED=true to enable. For enablement instructions, contact support.graphql: Add span events on GraphQL error spans, allowing multiple errors to be surfaced on the span.graphql: Adds user-provided extensions to graphql error span events through the DD_TRACE_GRAPHQL_ERROR_EXTENSIONS environment variable. This is disabled by default; when set it allows users to capture their extensions.record_exception method that adds an exception to a Span as a span event. Refer to Span.record_exception for more details.ASM
Code Security (IAST)
set_cookies function and adding more context. Warning: This change updates the vulnerability hash, meaning that if it was previously silenced in the web dashboard, it may reappear as a new vulnerability.LLM Observability
botocore: Moves bedrock token usage metrics on APM spans from the meta field under bedrock.usage.{prompt/completion}_tokens to the metrics field under bedrock.response.usage.{prompt/completion}_tokens.Profiling
Single-Step Instrumentation
boto3 from the minimum versions list to avoid blocking SSI on boto3 version.Tracing
asyncio: Fixes an issue where the name of a coroutine was being overridden by a ddtrace function.ot-baggage headers in wsgi apps were not properly recognized due to the http_ header prefix. This ensures consistent baggage propagation in wsgi compatible web frameworks.DD_TRACE_CLOUD_PAYLOAD_TAGGING_SERVICES env variable was not parsed correctlyddtrace.config.<undefined_attribute> would create an IntegrationConfig object instead of raising an AttributeError.bytecode>=0.15.1 for Python 3.12 to avoid known bug.ddtrace internal logger initialization mutating an unlocked shared resource.ddtrace.patch(...) is called with an invalid argument.pymongo: Changes the mongodb.query span tag to json so it can be parsed by the agent.__str__.ModuleNotFoundError errors raised when a OpenTracing tracer is initialized with a AsyncioScopeManager.API Gateway: Add preview support to create inferred AWS API Gateway spans when certain AWS Gateway headers are passed into the request. Requires DD_TRACE_INFERRED_PROXY_SERVICES_ENABLED=true to enable. For enablement instructions, contact support.
LLM Observability: Properly labels Deepseek for the model provider and span name for LLM Observability spans for calls to Deepseek models using the OpenAI SDK.
tracing: Introduces a record_exception method that adds an exception to a Span as a span event. Refer to [Span.record_exception](https://ddtrace.readthedocs.io/en/stable/api.html#ddtrace.trace.Span.record_exception) for more details.
LLM Observability: Introduces an improved automated context management system for LLM Observability-specific spans. Also modifies LLMObs.export_span(), LLMObs.inject_distributed_headers(), LLMObs.annotate() to default to the current active LLM Observability-specific span if span is not provided.
LLM Observability: Introduces automatic distributed tracing support for LLM Observability traces involving the concurrent.futures.thread module.
Symbol Database is now enabled by default.
graphql: Add span events on GraphQL error spans, allowing multiple errors to be surfaced on the span.
graphql: Adds user-provided extensions to graphql error span events through the <span class="title-ref">DD_TRACE_GRAPHQL_ERROR_EXTENSIONS</span> environment variable. This is disabled by default; when set it allows users to capture their extensions.
LLM Observability: This introduces <span class="title-ref">metadata</span> argument to <span class="title-ref">LLMObs.submit_evaluation_for</span>, which enables users to
attach arbitrary structured metadata to custom evaluations.
bytecode>=0.15.1 for Python 3.12 to avoid known bug.boto3 from the minimum versions list to avoid blocking SSI on boto3 version.ddtrace.config.<undefined_attribute> would create an IntegrationConfig object instead of raising an AttributeError.bedrock.usage.{prompt/completion}_tokens to the metrics field under bedrock.response.usage.{prompt/completion}_tokens.ddtrace internal logger initialization mutating an unlocked shared resource.ddtrace.patch(...) is called with an invalid argument.In some applications, unique cookie names are used per request, leading to a high number of unique cookie vulnerabilities. This fix improves detection…
Dynamic Instrumentation
LLM Observability
Deepseek for the model provider and span name for LLM Observability spans for calls to Deepseek models using the OpenAI SDK.LLMObs.export_span(), LLMObs.inject_distributed_headers(), LLMObs.annotate() to default to the current active LLM Observability-specific span if span is not provided.concurrent.futures.thread module.metadata argument to LLMObs.submit_evaluation_for, which enables users to attach arbitrary structured metadata to custom evaluations.Tracing
api gateway: Adds preview support to create inferred AWS API Gateway spans when certain AWS Gateway headers are passed into the request. Requires DD_TRACE_INFERRED_PROXY_SERVICES_ENABLED=true to enable. For enablement instructions, contact support.graphql: Add span events on GraphQL error spans, allowing multiple errors to be surfaced on the span.graphql: Adds user-provided extensions to graphql error span events through the DD_TRACE_GRAPHQL_ERROR_EXTENSIONS environment variable. This is disabled by default; when set it allows users to capture their extensions.record_exception method that adds an exception to a Span as a span event. Refer to Span.record_exception for more details.ASM
Code Security (IAST)
set_cookies function and adding more context. Warning: This change updates the vulnerability hash, meaning that if it was previously silenced in the web dashboard, it may reappear as a new vulnerability.LLM Observability
botocore: Moves bedrock token usage metrics on APM spans from the meta field under bedrock.usage.{prompt/completion}_tokens to the metrics field under bedrock.response.usage.{prompt/completion}_tokens.Profiling
Single-Step Instrumentation
boto3 from the minimum versions list to avoid blocking SSI on boto3 version.Tracing
asyncio: Fixes an issue where the name of a coroutine was being overridden by a ddtrace function.ot-baggage headers in wsgi apps were not properly recognized due to the http_ header prefix. This ensures consistent baggage propagation in wsgi compatible web frameworks.DD_TRACE_CLOUD_PAYLOAD_TAGGING_SERVICES env variable was not parsed correctlyddtrace.config.<undefined_attribute> would create an IntegrationConfig object instead of raising an AttributeError.bytecode>=0.15.1 for Python 3.12 to avoid known bug.ddtrace internal logger initialization mutating an unlocked shared resource.ddtrace.patch(...) is called with an invalid argument.pymongo: Changes the mongodb.query span tag to json so it can be parsed by the agent.__str__.ModuleNotFoundError errors raised when a OpenTracing tracer is initialized with a AsyncioScopeManager.XSS detection for Django applications and Jinja2 (Flask and FastAPI applications), which will be displayed on your DataDog Vulnerability Explorer dash…
We are happy to announce the release of v3.0.0 of ddtrace. This release drops support for Python 3.7 and many previously-deprecated parts of the library interface.
The 2.x release line enters Maintenance Mode with the release of 3.0.0, meaning that it will only receive bugfix changes on its last few minor releases, and 2.21 will be the last 2.x minor version. See the versioning policy for support level definitions.
item.nodeid.TestClass::test_method) for class-based tests.DD_PYTEST_USE_NEW_PLUGIN_BETA environment variable is no longer used.DD_LLMOBS_APP_NAME.parameters argument in LLMObs.annotate() has been removed. Use metadata instead.DD_TRACE_PROPAGATION_STYLE=b3 single header is no longer supported. Use b3 instead.ddtrace.tracer must be used.Tracer.configure(...).Before upgrading to v3, check for deprecation warnings in your application using the latest v2 release (2.21.0):
Enable warnings as errors in pytest:
pytest -W "error::ddtrace.DDTraceDeprecationWarning" tests.py
Set the following environment variable to enable all deprecation warnings:
PYTHONWARNINGS=all python app.py
The following environment variables have been removed or replaced in v3. Use the table below to update your configuration:
| v2 | Removed in v3 | Replaced/Changed in v3 |
|---|---|---|
| LLM Observability | ||
DD_LLMOBS_APP_NAME |
Use DD_LLMOBS_ML_APP instead |
|
_DD_LLMOBS_EVALUATOR_SAMPLING_RULES |
Use DD_LLMOBS_EVALUATOR_SAMPLING_RULES instead |
|
_DD_LLMOBS_EVALUATORS |
Use DD_LLMOBS_EVALUATORS instead |
|
| Tracing | ||
DD_ANALYTICS_ENABLED |
No replacement, this setting is now a no-op. Datadog Analytics is no longer supported. See the ingestion controls documentation for more detail. | |
DD_HTTP_CLIENT_TAG_QUERY_STRING |
Use DD_TRACE_HTTP_CLIENT_TAG_QUERY_STRING. |
|
DD_TRACE_ANALYTICS_ENABLED |
No replacement, this setting is now a no-op. Datadog Analytics is no longer supported. See the ingestion controls documentation for more detail. | |
DD_TRACE_METHODS (using [] notation) |
Must use : notation instead, For example: mod.submod:method1,method2;mod.submod:Class.method1. (docs) |
|
DD_TRACE_PROPAGATION_STYLE=”b3 single header” |
Use DD_TRACE_PROPAGATION_STYLE=b3 for identical behavior |
|
DD_TRACE_SAMPLE_RATE |
Use DD_TRACE_SAMPLING_RULES (docs). |
|
DD_TRACE_SPAN_AGGREGATOR_RLOCK |
No replacement (feature removed). Config variable is now a no-op. | |
| CI Visibility | ||
DD_PYTEST_USE_NEW_PLUGIN_BETA |
No replacement (new pytest plugin is not in beta anymore). |
Use the following patterns to identify the deprecated environment variables in a code base:
git grep -P -e "DD_LLMOBS_APP_NAME" \
-e "_DD_LLMOBS_EVALUATOR_SAMPLING_RULES" \
-e "_DD_LLMOBS_EVALUATORS" \
-e "DD_TRACE_PROPAGATION_STYLE=.*b3 single header" \
-e "DD_TRACE_SAMPLE_RATE" \
-e "DD_TRACE_API_VERSION=v0.3" \
-e "DD_ANALYTICS_ENABLED" \
-e "DD_TRACE_ANALYTICS_ENABLED" \
-e "DD_HTTP_CLIENT_TAG_QUERY_STRING" \
-e "DD_TRACE_SPAN_AGGREGATOR_RLOCK" \
-e "DD_TRACE_METHODS=.*\[\]"
Replace deprecated settings with the recommended alternatives where applicable.
NOTE: The changes to environment variables apply only to the configuration of the dd-trace-py library and not the Datadog Agent.
The following methods and module attributes have been removed or changed. Unless noted, removal means that the functionality pointed to by the removed interface has been disabled.
| v2 | v3 |
|---|---|
| Tracing | |
Tracer.configure(enabled: Optional[bool] = None, hostname: Optional[str] = None, port: Optional[int] = None, uds_path: Optional[str] = None, https: Optional[bool] = None, sampler: Optional[BaseSampler] = None, context_provider: Optional[DefaultContextProvider] = None, wrap_executor: Optional[Callable] = None, priority_sampling: Optional[bool] = None, settings: Optional[Dict[str, Any]] = None, dogstatsd_url: Optional[str] = None, writer: Optional[TraceWriter] = None, partial_flush_enabled: Optional[bool] = None, partial_flush_min_spans: Optional[int] = None, api_version: Optional[str] = None, compute_stats_enabled: Optional[bool] = appsec_enabled: Optional[bool] = None, iast_enabled: Optional[bool] = None, appsec_standalone_enabled: Optional[bool] = None, trace_processors: Optional[List[TraceProcessor]] = None ) |
Tracer.configure(context_provider: Optional[BaseContextProvider] = None, compute_stats_enabled: Optional[bool] = None, appsec_enabled: Optional[bool] = None, iast_enabled: Optional[bool] = None, apm_tracing_disabled: Optional[bool] = None, trace_processors: Optional[List[TraceProcessor]] = None) |
ddtrace.opentracer.tracer.Tracer()._dd_tracer |
ddtrace.tracer |
ddtrace.tracer.sampler.rules[].choose_matcher(lambda: True) |
ddtrace.tracer.sampler.rules[].choose_matcher(“foo”) Callables and regex patterns are no longer allowed as arguments to choose_matcher |
Span.sampled |
Span.context.sampling_priority > 0 |
| LLM Observability | |
LLMObs.annotate(parameters=foo) |
LLMObs.annotate(metadata=foo) |
Much of the library’s functionality has been compatible with Python 3.13 since version 2.20. Some pieces of functionality do not yet work with Python 3.13. Here’s what does not work under Python 3.13 as of dd-trace-py 3.0. 'Not tested' indicates that while the feature may be compatible with Python 3.13, its functionality has not been verified, as it is not included in our automated test coverage.
dd-trace-py doesn’t work on Windows with Python 3.13anthropicconsulfreezegungeventgoogle_generativeaigunicornlangchainmysqlclientopentracingpsycopgpsycopg2pymysqlpytestpytest-bddpytest-benchmarksanicseleniumsqlite3starlettetornadovertexaiBy following these guidelines, you can transition to dd-trace-py v3 gradually, ensuring minimal disruption to your services while taking advantage of the latest features and improvements. Happy migrating!
Removes code and tests related to Python 3.7, breaking the library's compatibility with this Python version.
ASM
Code Security (IAST)
LLM Observability
valkey: Adds automatic instrumentation of the Valkey package. Thank you AhmadMasry!
openai: Introduces tracing support to the OpenAI integration for Python versions 3.12 and 3.13.
Introduces an integration with the RAGAS evaluation framework to continuously monitor the performance of context-augmented LLM generations in production. The integration supports evaluating LLM inferences with the following RAGAS metrics:
To learn more, see the LLM Observability evaluations guide.
CI Visibility
Official release of the new version of the pytest plugin, introducing the following features:
NOTE: this new version of the plugin introduces breaking changes:
module, suite, and test names are now parsed from the item.nodeid attributeA beta version of the plugin had been available since v2.18.0, and could be enabled via the DD_PYTEST_USE_NEW_PLUGIN_BETA environment variable. The new version is now the default, and the environment variable is not used anymore.
LLM Observability
langchain: Removes the langchain.tokens.total_cost span metric for OpenAI calls. For continued cost estimation of OpenAI calls, enable LLM Observability.
langchain: Removes prompt-completion log sampling from the LangChain integration. To continue logging prompt completions, enable LLM Observability.
langchain: Removes integration metrics from the LangChain integration. To continue tracking operational metrics from the OpenAI integration, enable LLM Observability or use trace metrics instead.
langchain: removed tracing support for langchain.chains.base.Chains. To maintain support, migrate to using Langchain's LCEL instead.
- langchain: Drops tracing and LLM Observability support for Langchain v0.0.x versions.
openai: Removes prompt-completion log sampling from the OpenAI integration. To continue logging prompt completions, enable LLM Observability.
openai: Removes integration metrics from the OpenAI integration. To continue tracking operational metrics from the OpenAI integration, enable LLM Observability or use trace metrics instead.
openai: Drops tracing and LLM Observability support for OpenAI 0.x versions. To maintain support, upgrade to openai>=1.0.
Support for the deprecated environment variable DD_LLMOBS_APP_NAME has been removed.
Removes the deprecated parameters argument from LLMObs annotate(). Use metadata instead.
Profiling
Tracing
DD_TRACE_SAMPLE_RATE, use DD_TRACE_SAMPLING_RULES instead.DD_TRACE_API_VERSION=v0.3, the default v0.5 version is used instead.DD_ANALYTICS_ENABLED, Datadog Analytics is no longer supported.DD_TRACE_ANALYTICS_ENABLED, Datadog Analytics is no longer supported.DD_HTTP_CLIENT_TAG_QUERY_STRING, DD_TRACE_HTTP_CLIENT_TAG_QUERY_STRING should be used instead.DD_TRACE_SPAN_AGGREGATOR_RLOCK, disabling the span aggregator rlock is no longer supported.aws_lambda: Drops support for aws_lambda<=6.105.0.DD_TRACE_METHODS with the '[]' notation. Ensure DD_TRACE_METHODS use the ':' notation instead".ddtracer parameter from ddtrace.opentracer.tracer.Tracer().DD_TRACE_PROPAGATION_STYLE to b3 single header is no longer supported. Use b3 instead.ddtrace.tracer.sampler.rules[].choose_matcher(...) method and removes the timestamp_ns parameter from ddtrace.internal.rate_limiter.RateLimiter.is_allowed().Span.sampled propertyddtrace.trace.tracer to access the global tracer instance.ddtrace.trace.PinTracer.configure(...) method and removes the Tracer.sampler attribute.ddtrace.trace.Tracer can not be reinitialized.ASM
LLM Observability
_DD_LLMOBS_EVALUATORS environment variable is deprecated and will be removed in ddtrace 3.0.0. As an alternative to _DD_LLMOBS_EVALUATORS, you can use DD_LLMOBS_EVALUATORS instead. To migrate, replace _DD_LLMOBS_EVALUATORS with DD_LLMOBS_EVALUATORS._DD_LLMOBS_EVALUATOR_SAMPLING_RULES environment variable is deprecated and will be removed in ddtrace 3.0.0. As an alternative to _DD_LLMOBS_EVALUATOR_SAMPLING_RULES, you can use DD_LLMOBS_EVALUATOR_SAMPLING_RULES instead. To migrate, replace _DD_LLMOBS_EVALUATOR_SAMPLING_RULES with DD_LLMOBS_EVALUATOR_SAMPLING_RULES.Tracing
ddtrace.provider.BaseContextProvider to ddtrace.trace.BaseContextProvider. The ddtrace.provider module is deprecated and will be removed in v3.0.0.ddtrace.constants module:
ANALYTICS_SAMPLE_RATE_KEYSAMPLE_RATE_METRIC_KEYSAMPLING_PRIORITY_KEYSAMPLING_AGENT_DECISIONSAMPLING_RULE_DECISIONSAMPLING_LIMIT_DECISIONORIGIN_KEYUSER_ID_KEYHOSTNAME_KEYRUNTIME_FAMILYBASE_SERVICE_KEYSPAN_MEASURED_KEYKEEP_SPANS_RATE_KEYMULTIPLE_IP_HEADERSCONFIG_ENDPOINT_ENVCONFIG_ENDPOINT_RETRIES_ENVCONFIG_ENDPOINT_TIMEOUT_ENVddtrace.settings.config module and deprecates the following ddtrace.config attributes:
http, use DD_TRACE_HEADER_TAGS environment variable instead.http_server, use DD_TRACE_HTTP_SERVER_ERROR_STATUSES environment variable instead.trace_headers, this attribute is internal to the tracer.header_is_traced, this attribute is internal to the tracer.convert_rc_trace_sampling_rules, this attribute is internal to the tracer.enable_remote_configuration, use DD_REMOTE_CONFIGURATION_ENABLED environment variable instead.get_from, use ddtrace.trace.Pin to set instance level configurations.ddtrace.config.ASM
CI Visibility
Code security (IAST)
callonce decorator could trigger an import loopLLM Observability
openai: Fixes a patching issue where asynchronous moderation endpoint calls resulted in coroutine scheduling errors.vertexai: Resolves an issue with chat.send_message() where the content keyword argument was not parsed correctly.LLMObs.annotate() on the same span caused overwriting of metrics, metadata, and prompt dictionaries. Now, calling LLMObs.annotate() will merge new metrics/metadata/prompt values with any existing ones.LLMObs.enable() to configure LLM Observability without environment variables would not automatically propagate distributed tracing headers.Profiling
DD_PROFILING_TIMELINE_ENABLED and DD_PROFILING_STACK_V2_ENABLED.Tracing
aiohttp: Adds the environment variable DD_AIOHTTP_CLIENT_DISABLE_STREAM_TIMING_FOR_MEM_LEAK to address a potential memory leak in the aiohttp integration. When set to true, this flag may cause streamed response span timing to be inaccurate. The flag defaults to false.cassandra, cherrypy, flask_cache, starlette: Ensures a deprecation warning is not raised when patching these integrations via ddtrace-run and import ddtrace.auto.ddtrace package.Ensures that no module from ASM are loaded when ASM is disabled or unavailable.
ASM
Code Security
Tracing
Removes deprecated parameters from Tracer.configure(...) method and removes the Tracer.sampler attribute.
LLM Observability
langchain: Removes the langchain.tokens.total_cost span metric for OpenAI calls. For continued cost estimation of OpenAI calls, enable LLM Observability.langchain: Removes prompt-completion log sampling from the LangChain integration. To continue logging prompt completions, enable LLM Observability.langchain: Removes integration metrics from the LangChain integration. To continue tracking operational metrics from the OpenAI integration, enable LLM Observability or use trace metrics instead.openai: Removes prompt-completion log sampling from the OpenAI integration. To continue logging prompt completions, enable LLM Observability.openai: Removes integration metrics from the OpenAI integration. To continue tracking operational metrics from the OpenAI integration, enable LLM Observability or use trace metrics instead.Tracing
ddtrace.trace.tracer to access the global tracer instance.ddtrace.trace.Tracer can not be reinitialized.Tracer.configure(...) method and removes the Tracer.sampler attribute.DD_TRACE_SAMPLE_RATE, use DD_TRACE_SAMPLING_RULES instead.DD_TRACE_API_VERSION=v0.3, the default v0.5 version is used instead.DD_ANALYTICS_ENABLED, Datadog Analytics is no longer supported.DD_TRACE_ANALYTICS_ENABLED, Datadog Analytics is no longer supported.DD_HTTP_CLIENT_TAG_QUERY_STRING, DD_TRACE_HTTP_CLIENT_TAG_QUERY_STRING should be used instead.DD_TRACE_SPAN_AGGREGATOR_RLOCK, disabling the span aggregator rlock is no longer supported.DD_TRACE_METHODS with the [] notation. Ensure DD_TRACE_METHODS use the : notation instead.ddtracer parameter from ddtrace.opentracer.tracer.Tracer().ddtrace.tracer.sampler.rules[].choose_matcher(...) method and removes the timestamp_ns parameter from ddtrace.internal.rate_limiter.RateLimiter.is_allowed().Span.sampled property.valkey: Adds automatic instrumentation of the Valkey package. Thank you AhmadMasry!aiohttp: Adds the environment variable DD_AIOHTTP_CLIENT_DISABLE_STREAM_TIMING_FOR_MEM_LEAK to address a potential memory leak in the aiohttp integration. When set to true, this flag may cause streamed response span timing to be inaccurate. The flag defaults to false.XSS detection for Django applications, which will be displayed on your DataDog Vulnerability Explorer dashboard. See the Application Vulnerability Man…
CI Visibility
Official release of the new version of the pytest plugin, introducing the following features:
NOTE: this new version of the plugin introduces breaking changes:
module, suite, and test names are now parsed from the item.nodeid attributeA beta version of the plugin had been available since v2.18.0, and could be enabled via the DD_PYTEST_USE_NEW_PLUGIN_BETA environment variable. The new version is now the default, and the environment variable is not used anymore.
LLM Observability
DD_LLMOBS_APP_NAME has been removed.parameters argument from LLMObs annotate(). Use metadata instead.openai: Drops tracing and LLM Observability support for OpenAI 0.x versions. To maintain support, upgrade to openai>=1.0.langchain.chains.base.Chains. To maintain support, migrate to using Langchain's LCEL instead.Profiling
Tracing
DD_TRACE_PROPAGATION_STYLE to b3 single header is no longer supported. Use b3 instead.aws_lambda: Drops support for aws_lambda<=6.105.0.ASM
DD_APPSEC_AUTOMATED_USER_EVENTS_TRACKING is now removed. Please use the new env var DD_APPSEC_AUTO_USER_INSTRUMENTATION_MODE instead.LLM Observability
\_DD_LLMOBS_EVALUATORS environment variable is deprecated and will be removed in ddtrace 3.0.0.\_DD_LLMOBS_EVALUATORS, you can use <span class="title-ref">DD_LLMOBS_EVALUATORS</span> instead. To migrate, replace \_DD_LLMOBS_EVALUATORS with DD_LLMOBS_EVALUATORS.\_DD_LLMOBS_EVALUATOR_SAMPLING_RULES environment variable is deprecated and will be removed in ddtrace 3.0.0.\_DD_LLMOBS_EVALUATOR_SAMPLING_RULES, you can use DD_LLMOBS_EVALUATOR_SAMPLING_RULES instead. To migrate, replace \_DD_LLMOBS_EVALUATOR_SAMPLING_RULES with DD_LLMOBS_EVALUATOR_SAMPLING_RULES.Tracing
ddtrace.provider.BaseContextProvider to ddtrace.trace.BaseContextProvider. The ddtrace.provider module is deprecated and will be removed in v3.0.0.ddtrace.constants module:
ANALYTICS_SAMPLE_RATE_KEYSAMPLE_RATE_METRIC_KEYSAMPLING_PRIORITY_KEYSAMPLING_AGENT_DECISIONSAMPLING_RULE_DECISIONSAMPLING_LIMIT_DECISIONORIGIN_KEYUSER_ID_KEYHOSTNAME_KEYRUNTIME_FAMILYBASE_SERVICE_KEYSPAN_MEASURED_KEYKEEP_SPANS_RATE_KEYMULTIPLE_IP_HEADERSCONFIG_ENDPOINT_ENVCONFIG_ENDPOINT_RETRIES_ENVCONFIG_ENDPOINT_TIMEOUT_ENVddtrace.settings.config module and deprecates the following ddtrace.config attributes:
http, use DD_TRACE_HEADER_TAGS environment variable instead.http_server, use DD_TRACE_HTTP_SERVER_ERROR_STATUSES environment variable instead.trace_headers, this attribute is internal to the tracer.header_is_traced, this attribute is internal to the tracer.convert_rc_trace_sampling_rules, this attribute is internal to the tracer.enable_remote_configuration, use DD_REMOTE_CONFIGURATION_ENABLED environment variable instead.get_from, use ddtrace.trace.Pin to set instance level configurations.ddtrace.config.Removes code and tests related to Python 3.7, breaking the library's compatibility with this Python version.
ASM
Code Security (IAST)
LLM Observability
ASM
CI Visibility
Code security (IAST)
LLM Observability
LLMObs.annotate() on the same span caused overwriting of metrics, metadata, and prompt dictionaries. Now, calling LLMObs.annotate() will merge new metrics/metadata/prompt values with any existing ones.LLMObs.enable() to configure LLM Observability without environment variables would not automatically propagate distributed tracing headers.openai: Fixes a patching issue where asynchronous moderation endpoint calls resulted in coroutine scheduling errors.vertexai: Resolves an issue with chat.send_message() where the content keyword argument was not parsed correctly.Profiling
DD_PROFILING_TIMELINE_ENABLED and DD_PROFILING_STACK_V2_ENABLED.cassandra , cherrypy, flask_cache, starlette: Ensures a deprecation warning is not raised when patching these integrations via ddtrace-run and import ddtrace.auto.ddtrace package.Pin to wrapt<2 until we can ensure full compatibility with the breaking changes.
Estimated end-of-life date, accurate to within three months: 11-2025 See the support level definitions for more information.
wrapt<2 until we can ensure full compatibility with the breaking changes.Estimated end-of-life date: 10-2025 See the support level definitions for more information.
Estimated end-of-life date: 10-2025 See the support level definitions for more information.
Estimated end-of-life date: 10-2025 See the support level definitions for more information.
Estimated end-of-life date: 10-2025 See the support level definitions for more information.
TypeError in encoding when truncating a large bytes object.unicode string is too large error.Resolved an issue where the CODEOWNERS file of the project being tested was looked up in the current working directory rather than the repository root
CODEOWNERS file of the project being tested was looked up in the current working directory rather than the repository root.TypeError, causing a failed connection. confluent-kafka supports both unpacked and packed config; this change allows initialization with either.SynchronizedSamplePool where pool could be null when calling into ddog_ArrayQueue_ functions, leading to segfaults in the uWSGI shutdown?* was not being matched correctly for DD_TRACE_SAMPLING_RULES tags, due to it matching on spans with no tag matching the specified key.Avoid excessive filtering of stacktrace locations when finding vulnerabilities. After this change, vulnerabilities that were previously discarded will…
Fixes incompatibility between code origin and dynamic instrumentation probe on exit span functions.
Fixes an issue with PosixPath handling in path operations that could cause errors during taint tracking. This fix improves stability and slightly redu
Code Security
Profiling
_self_acquired_at attribute.Tracing
Fixes a NotImplementedError that occurred when trying to deepcopy wrapped builtin functions (likeopen) while ASM or IAST were enabled. The error was c
ASM
NotImplementedError that occurred when trying to deepcopy wrapped builtin functions (likeopen) while ASM or IAST were enabled. The error was caused by the wrapper not implementing the __deepcopy__ method.LLM Observability
agentless_enabled=True or DD_LLMOBS_AGENTLESS_ENABLED=true.Profiling
AttributeError: '_ProfiledThreadingLock' object has no attribute '_self_acquired_at'.Tracing
pylibmc: Fixes an issue where using Client(server=[url]) would throw the error __init__() got multiple values for argument 'servers'Nothing published for this version
This release is missing the OCI images due to issues with the release pipeline. If you need an OCI image with the features introduced in the 2.14 rele
This release is missing the OCI images due to issues with the release pipeline. If you need an OCI image with the features introduced in the 2.14 release line, please use v2.14.5 instead.
configurations: Resolves an issue where DD_TRACE_CLOUD_PAYLOAD_TAGGING_SERVICES env variable was not parsed correctly
DD_TRACE_CLOUD_PAYLOAD_TAGGING_SERVICES env variable was not parsed correctlyFix performance overhead of Python distribution parsing for internal telemetry.
Internal:
ASM:
Library Injection:
boto3 from the minimum versions list to avoid blocking SSI on boto3 version.boto3 from the minimum versions list to avoid blocking SSI on boto3 version.Ensure that no module from ASM are loaded when ASM is disabled or unavailable.
ASM
Code Security
Profiling
Tracing
DD_AIOHTTP_CLIENT_DISABLE_STREAM_TIMING_FOR_MEM_LEAK to address a potential memory leak in the aiohttp integration. When set to true, this flag may cause streamed response span timing to be inaccurate. The flag defaults to false.__str__.Moves ddtrace.provider.BaseContextProvider to ddtrace.trace.BaseContextProvider. The ddtrace.provider module is deprecated and will be removed in v3.0…
openai: Introduces tracing support to the OpenAI integration for Python versions 3.12 and 3.13.ddtrace.provider.BaseContextProvider to ddtrace.trace.BaseContextProvider. The ddtrace.provider module is deprecated and will be removed in v3.0.0.ddtrace.constants module:
ANALYTICS_SAMPLE_RATE_KEYSAMPLE_RATE_METRIC_KEYSAMPLING_PRIORITY_KEYSAMPLING_AGENT_DECISIONSAMPLING_RULE_DECISIONSAMPLING_LIMIT_DECISIONORIGIN_KEYUSER_ID_KEYHOSTNAME_KEYRUNTIME_FAMILYBASE_SERVICE_KEYSPAN_MEASURED_KEYKEEP_SPANS_RATE_KEYMULTIPLE_IP_HEADERSCONFIG_ENDPOINT_ENVCONFIG_ENDPOINT_RETRIES_ENVCONFIG_ENDPOINT_TIMEOUT_ENVddtrace.settings.config module and deprecates the following ddtrace.config attributes:
http, use DD_TRACE_HEADER_TAGS environment variable instead.http_server, use DD_TRACE_HTTP_SERVER_ERROR_STATUSES environment variable instead.trace_headers, this attribute is internal to the tracer.header_is_traced, this attribute is internal to the tracer.convert_rc_trace_sampling_rules, this attribute is internal to the tracer.enable_remote_configuration, use DD_REMOTE_CONFIGURATION_ENABLED environment variable instead.get_from, use ddtrace.trace.Pin to set instance level configurations.CI Visibility
Code Security
callonce decorator could trigger an import loopLLM Observability
LLMObs.enable() to configure LLM Observability without environment variables would not automatically propagate distributed tracing headers.openai: Fixes a patching issue where asynchronous moderation endpoint calls resulted in coroutine scheduling errors.vertexai: Resolves an issue with chat.send_message() where the content keyword argument was not parsed correctly.Profiling
SystemError from the memory profiler returning NULL when collecting eventscassandra, cherrypy, flask_cache, starlette: Ensures a deprecation warning is not raised when patching these integrations via ddtrace-run and import ddtrace.auto.One column per quarter.