NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI, #486 most downloaded on PyPI
Datadog APM client library
Latest stable
4.17.2
Last release 3 days ago
19 Nov 2026
Ships on a steady schedule
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
5 versions withdrawn
withdrawn after publishing
10 years old
871 releases · first in 2017
Drag across the ruler to compare, or tap a release to open it.
Nothing published for this version
Resolves the issue where tracer flares would not be generated if unexpected types were received in the AGENT_CONFIG remote configuration product.
AGENT_CONFIG remote configuration product.Fixes an issue where the OpenAI and LangChain integrations would still submit integration metrics even in agentless mode. Integration metrics are now
LLMObs.enable(agentless_enabled=True) or setting DD_LLMOBS_AGENTLESS_ENABLED=1.Initializer object is always reset and freed before the Python runtime.DD_PROFILING_EXPORT_LIBDD_ENABLED or DD_PROFILING_TIMELINE_ENABLED.Your coding agent can read these notes before it upgrades. Set up the MCP server →
library injection: Resolves an issue where the version of attrs installed by default on some Ubuntu installations was treated as incompatible with lib
attrs installed by default on some Ubuntu installations was treated as incompatible with library injectionAttributeError exceptions were being caught, interfering with the proper application cycle.SSI: This fix ensures injection denylist is included in published OCI package.
CI Visibility: Resolves an issue where exceptions other than timeouts and connection errors raised while fetching the list of skippable tests for ITR
.git was incorrectly being stripped from repository URLs when extracting service names, resulting in g, i, or t being removed (eg: test-environment.git incorrectly becoming test-environmen)session_id was being defaulted to trace_id which was causing unexpected UI behavior.ASM: Improves internal stability for the new fingerprinting feature.
openai: Introduces model tag for openai integration metrics for consistency with the OpenAI SaaS Integration. It has the same value as openai.request.
model tag for openai integration metrics for consistency with the OpenAI SaaS Integration. It has the same value as openai.request.model.celery or gunicorn workers. The LLM Observability writer thread now automatically restarts when a forked process is detected.asyncio.TimeoutErrors were not being propagated correctly from canceled OpenAI API requests.tracing(django): This fix resolves a bug where ddtrace was exhausting a Django stream response before returning it to user.
TypeError exception at runtime.Already mutably borrowed error by reverting back to pure-python rate limiter.ddtrace package files were published with incorrect file attributes.DD_PROFILING_STACK_V2_ENABLED set.ASM: The environment variable DD_APPSEC_AUTOMATED_USER_EVENTS_TRACKING is deprecated and will be removed in the next major release. Instead of DD_APPS…
ASM: This update introduces new Auto User Events support.
ASM’s [Account TakeOver (ATO) detection](https://docs.datadoghq.com/security/account_takeover_protection) is now automatically monitoring [all compatible user authentication frameworks](https://docs.datadoghq.com/security/application_security/enabling/compatibility/) to detect attempted or leaked user credentials during an ATO campaign.
To do so, the monitoring of the user activity is extended to now collect all forms of user IDs, including non-numerical forms such as usernames or emails. This is configurable with 3 different working modes: <span class="title-ref">identification</span> to send the user IDs in clear text; <span class="title-ref">anonymization</span> to send anonymized user IDs; or <span class="title-ref">disabled</span> to completely turn off any type of user ID collection (which leads to the disablement of the ATO detection).
The default collection mode being used is <span class="title-ref">identification</span> and this is configurable in your remote service configuration settings in the [service catalog]( https://app.datadog.com/security/appsec/inventory/services?tab=capabilities) (clicking on a service), or with the service environment variable <span class="title-ref">DD_APPSEC_AUTO_USER_INSTRUMENTATION_MODE</span>.
You can read more [here](https://docs.datadoghq.com/security/account_takeover_protection).
New local configuration environment variables include:
Additionally, an optional argument for the public API <span class="title-ref">track_user_login_success_event</span> and `track_user_login_failure_event`: <span class="title-ref">login_events_mode="auto"</span>. This allows manual instrumentation to follow remote configuration settings, enabling or disabling manual instrumentation with a single remote action on the Datadog UI.
Also prevents non numerical user ids to be reported by default without user instrumentation in Django.
Anthropic: Adds support for tracing message calls using tools.
LLM Observability: Adds support for tracing Anthropic messages using tool calls.
botocore: Adds support for overriding the default service name in botocore by either setting the environment variable DD_BOTOCORE_SERVICE or configuring it via <span class="title-ref">ddtrace.config.botocore["service"]</span>.
azure: Removes the restrictions on the tracer to only run the mini-agent on the consumption plan. The mini-agent now runs regardless of the hosting plan
ASM: Adds Threat Monitoring support for gRPC.
Code Security: add propagation for GRPC server sources.
LLM Observability: This introduces improved support for capturing tool call responses from the OpenAI and Anthropic integrations.
LLM Observability: This introduces the agentless mode configuration for LLM Observability. To enable agentless mode, set the environment variable DD_LLMOBS_AGENTLESS_ENABLED=1, or use the enable option LLMObs.enable(agentless_enabled=True).
LLM Observability: Function decorators now support tracing asynchronous functions.
LLM Observability: This introduces automatic input/output annotation for task/tool/workflow/agent/retrieval spans traced by function decorators. Note that manual annotations for input/output values will override automatic annotations.
LLM Observability: The OpenAI integration now submits embedding spans to LLM Observability.
LLM Observability: All OpenAI model parameters specified in a completion/chat completion request are now captured.
LLM Observability: This changes OpenAI-generated LLM Observability span names from openai.request to openai.createCompletion, openai.createChatCompletion, and openai.createEmbedding for completions, chat completions, and embeddings spans, respectively.
LLM Observability: This introduces the agent proxy mode for LLM Observability. By default, LLM Observability spans will be sent to the Datadog agent and then forwarded to LLM Observability. To continue submitting data directly to LLM Observability without the Datadog agent, set DD_LLMOBS_AGENTLESS_ENABLED=1 or set programmatically using LLMObs.enable(agentless_enabled=True).
LLM Observability: The Langchain integration now submits embedding spans to LLM Observability.
LLM Observability: The LLMObs.annotate() method now replaces non-JSON serializable values with a placeholder string [Unserializable object: <string representation of object>] instead of rejecting the annotation entirely.
pylibmc: adds traces for memcached add command
ASM: This introduces fingerprinting with libddwaf 1.19.1
Database Monitoring: Adds Database Monitoring (DBM) trace propagation for postgres databases used through Django.
langchain: Tags tool calls on chat completions.
LLM Observability: Adds retry logic to the agentless span writer to mitigate potential networking issues, like timeouts or dropped connections.
ASM: This introduces Command Injection support for Exploit Prevention on os.system only.
ASM: This introduces suspicious attacker blocking with libddwaf 1.19.1
patch() and unpatch() are deprecated and will be removed in version 3.0.0.patch() and unpatch() are deprecated and will be removed in version 3.0.0.patch() and unpatch() are deprecated and will be removed in version 3.0.0.patch() and unpatch() are deprecated and will be removed in version 3.0.0.patch() and unpatch() are deprecated and will be removed in version 3.0.0.patch() and unpatch() are deprecated and will be removed in version 3.0.0.patch() and unpatch() are deprecated and will be removed in version 3.0.0.patch() and unpatch() are deprecated and will be removed in version 3.0.0.patch() and unpatch() are deprecated and will be removed in version 3.0.0.patch() and unpatch() are deprecated and will be removed in version 3.0.0.DD_EXCEPTION_DEBUGGING_ENABLED environment variable has been deprecated in favor of DD_EXCEPTION_REPLAY_ENABLED. The old environment variable will be removed in a future major release.LLM Observability: Fixes an issue in the OpenAI integration where integration metrics would still be submitted even if LLMObs.enable(agentless_enabled=True) was set.
Code Security: add null pointer checks when creating new objects ids.
Code Security: add encodings.idna to the IAST patching denylist to avoid problems with gevent.
Code Security: add the boto package to the IAST patching denylist.
Code Security: fix two small memory leaks with Python 3.11 and 3.12.
CI Visibility: Fixes an issue where the pytest plugin would crash if the git binary was absent
CI Visibility: fixes incorrect URL for telemetry intake in EU that was causing missing telemetry data and SSL error log messages.
celery: changes error.message span tag to no longer include the traceback that is already included in the error.stack span tag.
CI Visibility: fixes source file information that would be incorrect in certain decorated / wrapped scenarios and forces paths to be relative to the repository root, if present.
futures: Fixes inconsistent behavior with concurrent.futures.ThreadPoolExecutor context propagation by passing the current trace context instead of the currently active span to tasks. This prevents edge cases of disconnected spans when the task executes after the parent span has finished.
kafka: Fixes ArgumentError raised when injecting span context into non-existent Kafka message headers.
botocore: Fixes Botocore Kinesis span parenting to use active trace context if a propagated child context is not found instead of empty context.
langchain: This fix resolves an issue where the wrong langchain class name was being used to check for Pinecone vectorstore instances.
LLM Observability: This resolves a typing hint error in the ddtrace.llmobs.utils.Documents helper class constructor where type hints did not accept input dictionaries with integer or float values.
LLM Observability: This fix resolves an issue where the OpenAI, Anthropic, and AWS Bedrock integrations were always setting temperature and max_tokens parameters to LLM invocations. The OpenAI integration in particular was setting the wrong temperature default values. These parameters are now only set if provided in the request.
opentelemetry: Resolves circular imports raised by the OpenTelemetry API when the ddcontextvars_context entrypoint is loaded. This resolves an incompatibility introduced in opentelemetry-api==1.25.0.
opentelemetry: Resolves an issue where the get_tracer function would raise a TypeError when called with the attribute argument. This resolves an incompatibility introduced in opentelemetry-api==1.26.0.
psycopg: Ensures traced async cursors return an asynchronous iterator object.
redis: This fix resolves an issue in the redis exception handling where an UnboundLocalError was raised instead of the expected BaseException.
ASM: This fix resolves an issue where the <span class="title-ref">requests</span> integration would not propagate when apm is opted out (i.e. in ASM Standalone).
profiling: Fixes an issue where task information coming from echion was encoded improperly, which could segfault the application.
tracing: fixes a potential crash where using partial flushes and tracer.configure() could result in an IndexError
tracer: This fix resolves an issue where the tracer was not starting properly on a read-only file system.
internal: fixes an issue where some pathlib functions return OSError on Windows.
ASM: This fix resolves an issue where the WAF could be disabled if the ASM_DD rule file was not found in Remote Config.
flask: Fix scenarios when using flask-like frameworks would cause a crash because of patching issues on startup.
Code Security: Logs warning instead of throwing an exception in the native module if IAST is not enabled by env var.
Code Security: fix potential infinite loop with path traversal when the analyze quota has been exceeded.
wsgi: Ensures the status of wsgi Spans are not set to error when a StopIteration exception is raised marked the span as an error. With this change, StopIteration exceptions in this context will be ignored.
langchain: tag non-dict inputs to LCEL chains appropriately. Non-dict inputs are stringified, and dict inputs are tagged by key-value pairs.
tracing: Updates DD_HEADER_TAGS and DD_TAGS to support the following formats: key1,key2,key3, key1:val,key2:val,key3:val3, key1:val key2:val key3:val3, and key1 key2 key3. Key value pairs that do not match an expected format will be logged and ignored by the tracer.
loguru: This fix avoids copying attributes from a log record's "extras" field to the record's top level if those attributes were not added by the Datadog integration.
opentelemetry: Resolves an edge case where distributed tracing headers could be generated before a sampling decision is made, resulting in dropped spans in downstream services.
profiling: captures lock usages with with context managers, e.g. with lock:
profiling: propagates runtime_id tag to libdatadog exporter. It is a unique string identifier for the profiled process. For example, Thread Timeline visualization uses it to distinguish different processes.
profiling: show lock init location in Lock Name and hide profiler internal frames from Stack Frame in Timeline Details tab.
ASM: This fix resolves an issue where ASM one click feature could fail to deactivate ASM.
redis: This fix resolves an issue in redis utils where a variable may not be declared within a try/catch
LLMObs.submit_evaluation() requires a Datadog API key to send custom evaluations to LLM Observability. If an API key is not set using either DD_API_KEY or LLMObs.enable(api_key="<api-key>"), this method will log a warning and return None.Nothing published for this version
Nothing published for this version
CI Visibility: Resolves an issue where exceptions other than timeouts and connection errors raised while fetching the list of skippable tests for ITR
.git was incorrectly being stripped from repository URLs when extracting service names, resulting in g, i, or t being removed (eg: test-environment.git incorrectly becoming test-environmen)asyncio.TimeoutErrors were not being propagated correctly from canceled OpenAI API requests.DD_PROFILING_STACK_V2_ENABLED is set.tracing(django): Resolves a bug where ddtrace was exhausting a Django stream response before returning it to user.
ddtrace was exhausting a Django stream response before returning it to user.Already mutably borrowed error by reverting back to pure-python rate limiter.Nothing published for this version
SSI: Fixes incorrect file permissions on lib-injection images.
ASM: This fix resolves an issue where the WAF could be disabled if the ASM_DD rule file was not found in Remote Config.
lib-injection: This fix resolves an issue with docker layer caching and the final lib-injection image size.
with context managers, e.g. with lock:runtime_id tag to libdatadog exporter. It is a unique string identifier for the profiled process. For example, Thread Timeline visualization uses it to distinguish different processes.langchain: This fix resolves an issue where the wrong langchain class name was being used to check for Pinecone vectorstore instances.
ddcontextvars_context entrypoint is loaded. This resolves an incompatibility introduced in opentelemetry-api==1.25.0.get_tracer function would raise a TypeError when called with the attribute argument. This resolves an incompatibility introduced in opentelemetry-api==1.26.0.Code Security: Expands SSRF vulnerability support for Code Security and Exploit Prevention for the modules urllib3, http.client, webbrowser and urllib…
botocore: Adds support for overriding the default service name in botocore by either setting the environment variable DD_BOTOCORE_SERVICE or configuring it via ddtrace.config.botocore["service"].
Database Monitoring: Adds Database Monitoring (DBM) trace propagation for postgres databases used through Django.
Anthropic: Adds support for tracing message calls using tools.
LLM Observability: Adds support for tracing Anthropic messages using tool calls.
azure: Removes the restrictions on the tracer to only run the mini-agent on the consumption plan. The mini-agent now runs regardless of the hosting plan
Anthropic: Adds support for tracing synchronous and asynchronous message streaming.
LLM Observability: Adds support for tracing synchronous and asynchronous message streaming.
SSI: Introduces generic safeguards for automatic instrumentation when using single step install in the form of early exit conditions. Early exit from instrumentation is triggered if a version of software in the environment is not explicitly supported by ddtrace. The Python runtime itself and many Python packages are checked for explicit support on the basis of their version.
langchain: Introduces support for langchain==0.2.0 by conditionally patching the langchain-community module if available, which is an optional dependency for langchain>=0.2.0. See the langchain integration docs for more details.
LLM Observability: Adds support to automatically submit Anthropic chat messages to LLM Observability.
tracer: This introduces the tracer flare functionality. Currently the tracer flare includes the tracer logs and tracer configurations.
Code Security: Expands SSRF vulnerability support for Code Security and Exploit Prevention for the modules urllib3, http.client, webbrowser and urllib.request.
ASM: This introduces full support for exploit prevention in the python tracer.
with monitoring and blocking feature, telemetry and span metrics reports.
ASM: This introduces SQL injection support for exploit prevention.
anthropic: This introduces tracing support for anthropic chat messages.
See the docs for more information.
ASM: This introduces "Standalone ASM", a feature that disables APM in the tracer but keeps ASM enabled. In order to enable it, set the environment variables DD_APPSEC_ENABLED=1 and DD_EXPERIMENTAL_APPSEC_STANDALONE_ENABLED=1.
LLM Observability: This introduces the LLM Observability SDK, which enhances the observability of Python-based LLM applications. See the LLM Observability Overview or the SDK documentation for more information about this feature.
opentelemetry: Adds support for span events.
tracing: Ensures the following OpenTelemetry environment variables are mapped to an equivalent Datadog configuration (datadog environment variables taking precedence in cases where both are configured):
OTEL_SERVICE_NAME -> DD_SERVICE
OTEL_LOG_LEVEL -> DD_TRACE_DEBUG
OTEL_PROPAGATORS -> DD_TRACE_PROPAGATION_STYLE
OTEL_TRACES_SAMPLER -> DD_TRACE_SAMPLE_RATE
OTEL_TRACES_EXPORTER -> DD_TRACE_ENABLED
OTEL_METRICS_EXPORTER -> DD_RUNTIME_METRICS_ENABLED
OTEL_LOGS_EXPORTER -> none
OTEL_RESOURCE_ATTRIBUTES -> DD_TAGS
OTEL_SDK_DISABLED -> DD_TRACE_OTEL_ENABLED
otel: Adds support for generating Datadog trace metrics using OpenTelemetry instrumentations
builtins.open function is experimental and may not be stable. This aspect is not replaced by default.grpc.aio clients and servers is experimental and may not be stable. This integration is now disabled by default.DD_LLMOBS_APP_NAME is deprecated and will be removed in the next major version of ddtrace. As an alternative to DD_LLMOBS_APP_NAME, you can use DD_LLMOBS_ML_APP instead. See the SDK setup documentation for more details on how to configure the LLM Observability SDK.Code Security: Logs warning instead of throwing an exception in the native module if IAST is not enabled by env var.
redis: This fix resolves an issue in redis utils where a variable may not be declared within a try/catch
Code Security: Adds the boto package to the IAST patching denylist.
celery: Changes error.message span tag to no longer include the traceback that is already included in the error.stack span tag.
CI Visibility: Fixes source file information that would be incorrect in certain decorated / wrapped scenarios and forces paths to be relative to the repository root, if present.
LLM Observability: This resolves a typing hint error in the ddtrace.llmobs.utils.Documents helper class constructor where type hints did not accept input dictionaries with integer or float values.
LLM Observability: This fix resolves an issue where the OpenAI, Anthropic, and AWS Bedrock integrations were always setting temperature and max_tokens parameters to LLM invocations. The OpenAI integration in particular was setting the wrong temperature default values. These parameters are now only set if provided in the request.
redis: This fix resolves an issue in the redis exception handling where an UnboundLocalError was raised instead of the expected BaseException.
ASM: This fix resolves an issue where the requests integration would not propagate when apm is opted out (i.e. in ASM Standalone).
profiling: Fixes an issue where task information coming from echion was encoded improperly, which could segfault the application.
tracing: Fixes a potential crash where using partial flushes and tracer.configure() could result in an IndexError.
flask: Fixes scenarios when using flask-like frameworks would cause a crash because of patching issues on startup.
wsgi: Ensures the status of wsgi Spans are not set to error when a StopIteration exception is raised marked the span as an error. With this change, StopIteration exceptions in this context will be ignored.
langchain: Tags non-dict inputs to LCEL chains appropriately. Non-dict inputs are stringified, and dict inputs are tagged by key-value pairs.
langchain: Fixes an issue of langchain patching errors due to the langchain-community module becoming an optional dependency in langchain>=0.2.0. The langchain integration now conditionally patches langchain-community methods if it is available. See the langchain integration docs for more details.
ASM: This fix resolves an issue where an org could not customize actions through remote config.
ASM: Protects against potentially returning None when tainting a gRPC message.
botocore: This fix adds additional key name checking and appropriate defaults for responses from Cohere and Amazon models.
Tracer: This fix resolves an issue where importing asyncio after a trace has already been started will reset the currently active span.
CI Visibility: Fixes traces that were not properly being sent in agentless mode, and were otherwise not properly attached to the test that started them
grpc: Fixes a bug in the grpc.aio support specific to streaming responses.
openai: This fix resolves an issue where specifying n=None for streamed chat completions resulted in a TypeError.
openai: This fix removes patching for the edits and fine tunes endpoints, which have been removed from the OpenAI API.
openai: This fix resolves an issue where streamed OpenAI responses raised errors when being used as context managers.
tracing: Ensures span links generated by distributed tracing headers record the correct sampling decision.
telemetry: This fix resolves an issue when using pytest + gevent where the telemetry writer was eager initialized by pytest entrypoints loading of our plugin causing a potential dead lock.
tracing: Fixes an issue where DD_TRACE_SPAN_TRACEBACK_MAX_SIZE was not applied to exception tracebacks.
Code Security: This fixes a bug in the AST patching process where ImportError exceptions were being caught, interfering with the proper application cycle if an ImportError was expected."
Code Security: Ensure IAST propagation does not raise side effects related to Magic methods.
Code Security: Fixes a potential memory corruption when the context was reset.
langchain: This fix resolves an issue where specifying inputs as a keyword argument for batching on chains caused a crash.
Code Security: Avoids calling terminate on the extend and join aspect when an exception is raised.
tracing: Ensures spans are rate limited at the expected rate (100 spans per second by default). Previously long running spans would set the rate limiter to set an invalid window and this could cause the next trace to be dropped.
RemoteConfig: This fix resolves an issue where remote config did not work for the tracer when using an agent that would add a flare item to the remote config payload. With this fix, the tracer will now correctly pull out the lib_config we need from the payload in order to implement remote config changes properly.
opentelemetry: Records exceptions on spans in a manner that is consistent with the otel specification
tracing: Ensures W3C tracecontext headers take precedence over all other header formats when incoming headers reference different spans in the same trace.
LLM Observability: The SDK allowed users to submit an unsupported numerical evaluation metric type. All evaluation metric types submitted with numerical type will now be automatically converted to a score type. As an alternative to using the numerical type, use score instead.
lib-injection: Updates base Alpine image to 3.20.
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
CI Visibility: Resolves an issue where exceptions other than timeouts and connection errors raised while fetching the list of skippable tests for ITR
.git was incorrectly being stripped from repository URLs when extracting service names, resulting in g, i, or t being removed (eg: test-environment.git incorrectly becoming test-environmen)DD_PROFILING_STACK_V2_ENABLED is set.ASM: This fix resolves an issue where the WAF could be disabled if the ASM_DD rule file was not found in Remote Config.
with context managers, e.g. with lock:runtime_id tag to libdatadog exporter. It is a unique string identifier for the profiled process. For example, Thread Timeline visualization uses it to distinguish different processes.langchain: This fix resolves an issue where the wrong langchain class name was being used to check for Pinecone vectorstore instances.
ddcontextvars_context entrypoint is loaded. This resolves an incompatibility introduced in opentelemetry-api==1.25.0.get_tracer function would raise a TypeError when called with the attribute argument. This resolves an incompatibility introduced in opentelemetry-api==1.26.0.redis exception handling where an UnboundLocalError was raised instead of the expected BaseException.langchain patching errors due to the langchain-community module becoming an optional dependency in langchain>=0.2.0. The langchain integration now conditionally patches langchain-community methods if it is available. See the langchain integration docs for more details.Code Security: Adds encodings.idna to the IAST patching denylist to avoid problems with gevent.
encodings.idna to the IAST patching denylist to avoid problems with gevent.error.message span tag to no longer include the traceback that is already included in the error.stack span tag.ddtrace.llmobs.utils.Documents helper class constructor where type hints did not accept input dictionaries with integer or float values.temperature and max_tokens parameters to LLM invocations. The OpenAI integration in particular was setting the wrong temperature default values. These parameters are now only set if provided in the request.echion was encoded improperly, which could segfault the application.tracer.configure() could result in an IndexError.pathlib functions return OSErrorg on Windows.StopIteration exception is raised marked the span as an error. With this change, StopIteration exceptions in this context will be ignored.numerical evaluation metric type. All evaluation metric types submitted with numerical type will now be automatically converted to a score type. As an alternative to using the numerical type, use score instead.futures: Fixes inconsistent behavior with concurrent.futures.ThreadPoolExecutor context propagation by passing the current trace context instead of th
concurrent.futures.ThreadPoolExecutor context propagation by passing the current trace context instead of the currently active span to tasks. This prevents edge cases of disconnected spans when the task executes after the parent span has finished.Removes the deprecated sqlparse dependency.
…security monitoring that detects and blocks vulnerability exploits. This introduces full support for exploit prevention in the python tracer.
LLM Observability: This introduces the LLM Observability SDK, which enhances the observability of Python-based LLM applications. See the LLM Observability Overview or the SDK documentation for more information about this feature.
ASM: Application Security Management (ASM) introduces its new "Exploit Prevention" feature in public beta, a new type of in-app security monitoring that detects and blocks vulnerability exploits. This introduces full support for exploit prevention in the python tracer.
with monitoring and blocking features, telemetry, and span metrics reports.
opentelemetry: Adds support for span events.
tracing: Ensures the following OpenTelemetry environment variables are mapped to an equivalent Datadog configuration (datadog environment variables taking precedence in cases where both are configured):
OTEL_SERVICE_NAME -> DD_SERVICE
OTEL_LOG_LEVEL -> DD_TRACE_DEBUG
OTEL_PROPAGATORS -> DD_TRACE_PROPAGATION_STYLE
OTEL_TRACES_SAMPLER -> DD_TRACE_SAMPLE_RATE
OTEL_TRACES_EXPORTER -> DD_TRACE_ENABLED
OTEL_METRICS_EXPORTER -> DD_RUNTIME_METRICS_ENABLED
OTEL_RESOURCE_ATTRIBUTES -> DD_TAGS
OTEL_SDK_DISABLED -> DD_TRACE_OTEL_ENABLED
otel: Adds support for generating Datadog trace metrics using OpenTelemetry instrumentations
aiomysql, asyncpg, mysql, mysqldb, pymysql: Adds Database Monitoring (DBM) for remaining mysql and postgres integrations lacking support.
(aiomysql, aiopg): Implements span service naming determination to be consistent with other database integrations.
ASM: This introduces the capability to enable or disable SCA using the environment variable DD_APPSEC_SCA_ENABLED. By default this env var is unset and in that case it doesn't affect the product.
Code Security: Taints strings from gRPC messages.
botocore: This introduces tracing support for bedrock-runtime embedding operations.
Vulnerability Management for Code-level (IAST): Enables IAST in the application. Needed to start application with ddtrace-run [your-application-run-command] prior to this release. Now, you can also activate IAST with the patch_all function.
langchain: This adds tracing support for LCEL (LangChain Expression Language) chaining syntax. This change specifically adds synchronous and asynchronous tracing support for the invoke and batch methods.
builtins.open function is experimental and may not be stable. This aspect is not replaced by default.grpc.aio clients and servers is experimental and may not be stable. This integration is now disabled by default.DD_LLMOBS_APP_NAME is deprecated and will be removed in the next major version of ddtrace. As an alternative to DD_LLMOBS_APP_NAME, you can use DD_LLMOBS_ML_APP instead. See the SDK setup documentation for more details on how to configure the LLM Observability SDK.asyncio after a trace has already been started will reset the currently active span.grpc.aio integration specific to streaming responses.n=None for streamed chat completions resulted in a TypeError.DD_TRACE_SPAN_TRACEBACK_MAX_SIZE was not applied to exception tracebacks.terminate on the extend and join aspect when an exception is raised.pytest + gevent where the telemetry writer was eager initialized by pytest entry points loading of our plugin causing a potential dead lock.ImportError exceptions were being caught, interfering with the proper application cycle if an ImportError was expected."taint_structure.FileNotFoundError.Chain.invoke() instead of Chain.__call__() resulted in the an ArgumentError due to an argument name change for inputs between the two methods.langchain_community package does not allow automatic submodule importing.lib_config entrygrpc.aio interceptors are registeredheaders.items()) does not duplicate them.tracer.get_log_correlation_context() incorrectly returned a 128-bit trace_id even with DD_TRACE_128_BIT_TRACEID_LOGGING_ENABLED set to False (the default), breaking log correlation. It now returns a 64-bit trace_id.protobuf to load in injected environments,protobuf versions. The profiler will now detect when injection is used and try loading with the native exporter. If that fails, it will self-disable rather than loading protobuf.pymongo.pool.validate_sessionflask and uwsgi.Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
opentelemetry: Resolves circular imports raised by the OpenTelemetry API when the ddcontextvars_context entrypoint is loaded. This resolves an incompa
ddcontextvars_context entrypoint is loaded. This resolves an incompatibility introduced in opentelemetry-api==1.25.0.get_tracer function would raise a TypeError when called with the attribute argument. This resolves an incompatibility introduced in opentelemetry-api==1.26.0.ASM: This fix resolves an issue where an org could not customize actions through remote config.
None for streamed chat completions resulted in a TypeError.tracer.configure() could result in an IndexErrorDD_TRACE_SPAN_TRACEBACK_MAX_SIZE was not applied to exception tracebacks.asyncio after a trace has already been started will reset the currently active span.with context managers, e.g. with lock:runtime_id tag to libdatadog exporter. It is a unique string identifier for the profiled process. For example, Thread Timeline visualization uses it to distinguish different processes.Code Security: Security tracing for the builtins.open function is experimental and may not be stable. This aspect is not replaced by default.
builtins.open function is experimental and may not be stable. This aspect is not replaced by default.grpc.aio clients and servers is experimental and may not be stable. This integration is now disabled by default.telemetry: This fix resolves an issue when using pytest + gevent where the telemetry writer was eager initialized by pytest entrypoints loading of our
pytest + gevent where the telemetry writer was eager initialized by pytest entrypoints loading of our plugin causing a potential dead lock.profiling: Fixes a defect where the deprecated path to the Datadog span type was used by the profiler.
tracer.get_log_correlation_context() incorrectly returned a 128-bit trace_id even with DD_TRACE_128_BIT_TRACEID_LOGGING_ENABLED set to False (the default), breaking log correlation. It now returns a 64-bit trace_id.tracing: This fix resolves an issue where sampling rules were not matching correctly on float values that had a 0 decimal value. Sampling rules now ev
tracing: This fix resolves an issue where sampling rules were not matching correctly on float values that had a 0 decimal value. Sampling rules now evaluate such values as integers.
langchain: This fix resolves an issue where the LangChain integration always attempted to patch LangChain partner
libraries, even if they were not available.
langchain: This fix resolves an issue where tracing Chain.invoke() instead of Chain.__call__() resulted in the an ArgumentError due to an argument name change for inputs between the two methods.
langchain: This fix adds error handling for checking if a traced LLM or chat model is an OpenAI instance, as the langchain_community package does not allow automatic submodule importing.
internal: This fix resolves an error regarding the remote config module with payloads missing a lib_config entry
profiling: fix a bug that caused the HTTP exporter to crash when attempting to serialize tags.
grpc: Resolves segfaults raised when grpc.aio interceptors are registered
Code Security: Ensure that when tainting the headers of a Flask application, iterating over the headers (i.e., with <span class="title-ref">headers.items()</span>) does not duplicate them.
Code Security: to enable IAST in the application, you had to start it with the command ddtrace-run [your-application-run-command] so far. Now, you can
ddtrace-run [your-application-run-command] so far. Now, you can also activate IAST with the patch_all function.FileNotFoundError.Vulnerability Management for Code-level (IAST): Some native exceptions were not being caught correctly by the python tracer. This fix remove those exc…
tracing: This release adds support for lazy sampling, essentially moving when we make a sampling decision for a trace to the latest possible moment. These include the following: 1. Before encoding a trace chunk to be sent to the agent 2. Before making an outgoing request via HTTP, gRPC, or a DB call for any automatically instrumented integration 3. Before running os.fork() For most users this change shouldn't have any impact on their traces, but it does allow for more flexibility in sampling (see features release note). It should be noted that if a user has application egress points that are not automatically instrumented, to other Datadog components (downstream instrumented services, databases, or execution context changes), and rely on the Python tracer to make the sampling decision (don't have an upstream service doing this), they will need to manually run the sampler for those traces, or use HttpPropagator.inject(). For more information please see the following: https://ddtrace.readthedocs.io/en/stable/advanced_usage.html#distributed-tracing https://ddtrace.readthedocs.io/en/stable/advanced_usage.html#tracing-context-management
langchain==0.1.0. Note that this does not have tracing support for deprecated langchain operations. Please follow the langchain upgrade guide or the langchain integration :ref: <span class="title-ref">docs<langchain></span> to enable full tracing support.dramatiq library.DD_TRACE_SAMPLING_RULES based on any span attribute (service, resource, tags, name)regardless of when the value for the attribute is set. This change is particularly beneficial for sampling on tags, since the vast majority of tags are set after the span is created. Since sampling was previously done at span creation time, this meant that those tags could not be used for sampling decisions.DD_PROFILING_STACK_V2_ENABLED=true. This new sampler should resolve segfault issues on Python 3.11 and later, while also decreasing the latency contribution of the profiler in many situations, and also improving the accuracy of stack-sampling data. This feature is currently only available on Linux using CPython 3.8 or greater. Requires DD_PROFILING_EXPORT_LIBDD_ENABLED=true to be set.confluent_kafka.Consumer.consume(). Previously only <span class="title-ref">confluent_kafka.Consumer.poll</span> was instrumented.ddtrace.contrib.asyncio.AsyncioContextProvider. ddtrace fully support tracing across asyncio tasks. Asyncio no longer requires additional configurations.tracer.sampler is deprecated and will be removed in the next major version release. To manually sample please call tracer.sample instead.ddtrace.contrib.gevent.provider.GeventContextProvider. Drops support for <span class="title-ref">gevent<20.12.0</span> and <span class="title-ref">greenlet<1.0</span>.Vulnerability Management for Code-level (IAST): Some native exceptions were not being caught correctly by the python tracer. This fix remove those exceptions to avoid fatal error executions.
otel: Ensures that the last datadog parent_id is added to w3c distributed tracing headers generated by the OpenTelemetry API.
ASM: This fix resolves an issue where a valid user may trigger a failed login event.
ASM: always clear the DDWaf context at the end of the span to avoid gc-induced latency spikes at the end of some requests.
ASM: This fix resolves an issue where django login failure events may send wrong information of user existence.
CI Visibility: fixes an issue where git author or committer names containing commas (eg: "Lastname, Firstname") would not work (and log an error) due to the use of comma as a separator.
propagation: This fix resolves an issue where the sampling decision-maker tag in tracestate propagation headers was clobbered by a default value.
datastreams: Changed DSM processor error logs to debug logs for a statement which is retried. If all retries fail, the stack trace is included
internal telemetry: Ensures heartbeat events are sent at regular intervals even when no other events are being sent.
Fix an incompatibility between the handling of namespace module imports and parts of the functionalities of the standard library importlib module.
internal: This fix resolves an issue where importing the ddtrace.appsec._iast._patches module would fail raising an ImportError
internal: This fix resolves an issue where importing the ddtrace.internal.peer_service module would fail raising an ImportError
langchain: Ensures langchain vision APIs are correctly instrumented
Fix for the declaration of dependencies for the package.
internal: This fix resolves an issue where importing the ddtrace.contrib.botocore.services module would fail raising an ImportError
profiling: handle unexpected stack data to prevent the profiler from stopping.
starlette: Fix a bug that crashed background tasks started from functions without a <span class="title-ref">__name__</span> attribute
ASM: This fix resolves an issue where the asgi middleware could crash with a RuntimeError "Unexpected message received".
ASM: This fix resolves an issue with Flask instrumentation causing CPU leak with ASM, API Security and Telemetry enabled.
Vulnerability Management for Code-level (IAST): Addresses an issue where the IAST native module was imported even though IAST was not enabled.
Vulnerability Management for Code-level (IAST): This fix addresses an issue where tainting objects may fail due to context not being created in the current span.
Vulnerability Management for Code-level (IAST): This fix addresses an issue where AST patching would generate code that fails to compile, thereby preventing the application from starting correctly.
Vulnerability Management for Code-level (IAST): This fix addresses AST patching issues where other subscript operations than Load were being unintentionally patched, leading to compilation errors for the patched module.
Vulnerability Management for Code-level (IAST): Fixes an issue where an atexit handler could lead to a segmentation fault.
Vulnerability Management for Code-level (IAST): This fix addresses an issue where a vulnerability would be reported at line 0 if we couldn't extract the proper line number, whereas the default line number should be -1.
kafka: This fix resolves an issue where None messages from confluent-kafka could cause crashes in the Kafka integration.
appsec: This fix resolves an issue in which the library attempted to finalize twice a context object used by the Application Security Management product.
tracing: Removes allow_false argument from ddtrace samplers. allow_false allows datadog samplers to return a value that differs from the sampling decision, this behavior is not supported.
profiling: This fixes a free(): invalid pointer error which would arise as a result of incorrectly linking the C++ runtime.
starlette: Ensures correct URL tag is set for starlette v0.34.0 and above.
structlog: Fixes error where multiple loggers would duplicate processors. Also adds processors injection when resetting to defaults.
Nothing published for this version
Nothing published for this version
profiling: Fixes a defect where the deprecated path to the Datadog span type was used by the profiler.
tracer.get_log_correlation_context() incorrectly returned a 128-bit trace_id even with DD_TRACE_128_BIT_TRACEID_LOGGING_ENABLED set to False (the default), breaking log correlation. It now returns a 64-bit trace_id.internal: This fix resolves an error regarding the remote config module with payloads missing a lib_config entry
lib_config entrypymongo.pool.validate_sessionCode Security: fix setting the wrong source on map elements tainted from taint_structure .
FileNotFoundError.Vulnerability Management for Code-level (IAST): This fix addresses an issue where tainting objects may fail due to context not being created in the cu…
ddtrace.internal.peer_service module would fail raising an ImportError__name__ attributeProfiling: This fix resolves an issue where the profiler was forcing protobuf to load in injected environments, causing crashes in configurations whic
kafka: Adds tracing and DSM support for confluent_kafka.Consumer.consume(). Previously only confluent_kafka.Consumer.poll was instrumented.
confluent_kafka.Consumer.consume(). Previously only <span class="title-ref">confluent_kafka.Consumer.poll</span> was instrumented.ddtrace.contrib.botocore.services module would fail raising an ImportErrorNothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
One column per quarter.