Summary
This release adds OTLP span metrics support, an inspectable tracer for testing, AppSec command-injection RASP protection, and several new integrations, while hardening trace-context propagation and dropping support for Go versions below 1.26.
Application Performance Monitoring (APM)
OTLP span metrics
Adds an end-to-end OTLP span metrics pipeline: a new config surface (#4895), stats-to-histogram conversion (#4897), an HTTP exporter (#4899), wiring into the tracer's flush path (#4900), and a startup-log export status report (#5062).
Inspectable tracer
Adds a test-only inspectable tracer (testtracer) that consolidates the four previous ad-hoc ways of inspecting spans and mocking the tracer in tests. #4512
Baggage and tracestate propagation hardening
Three targeted fixes to the tracer's HTTP-header propagation layer: percent-encoding of legacy ot-baggage-* values on inject to stop raw control bytes from reaching outbound requests, enforcement of the same item/byte limits on ot-baggage-* as the W3C baggage header, and a size bound on the tracestate header (4096 bytes total, 512 bytes per non-Datadog entry) to prevent unbounded downstream propagation and storage. #5063, #5120, #5162
New go.uber.org/zap integration
Adds a new contrib package providing structured-logging correlation for go.uber.org/zap. #4729
Minimum Go version raised to 1.26
Breaking/support-policy change: the minimum supported Go version is now 1.26.0. Go 1.25 and earlier are no longer supported. #5227
Application and API Protection (AAP)
Command injection (CMDi) RASP
Adds RASP protection against command-injection attacks that spawn processes via os.StartProcess. #4978
go-libddwaf v5.0.0
Upgrades the WAF engine to v5.0.0 across all modules, moving to subcontext-based ephemeral runs. #4985
LLM Observability
MCP intent capture and trace redaction
Adds intent-capture support to the mark3labs/mcp-go and modelcontextprotocol/go-sdk integrations, so MCP tool calls carry the caller's intent into LLM Observability traces, plus support for redacting tool output in those traces. #4939, #4941, #4944, #4945, #4947, #4976
Serverless
GCP Pub/Sub contrib additions
Adds instrumentation for GCP Pub/Sub push subscriptions (#4835), inferred span links (#4840), and admin endpoints (#4957).
What's Changed
Application Performance Monitoring (APM)
- feat(internal/version): v2.11.0-dev by @darccio in #4969
- fix(contrib/confluentinc/confluent-kafka-go/kafka.v2): match existing behavior for Producers when instrumenting via Orchestrion [shadow] by @darccio in #4967
- fix(contrib/gofiber/fiber.v2): clone zero-copy fasthttp strings in span tags by @rarguelloF in #4995
- feat: inspectable tracer by @darccio in #4512
- fix(haproxy): pass SPOE configuration path directly by @e-n-0 in #5042
- fix(contrib/gofiber/fiber.v2): suppress fasthttp double-instrumentation on all listen paths by @darccio in #5045
- feat(contrib/go.uber.org/zap): add integration by @rarguelloF in #4729
- fix(internal/orchestrion): don't hand out a finished span as the GLS parent by @kakkoyun in #5071
- feat(contrib/google.golang.org/grpc): add WithErrorCheck option by @rarguelloF in #5057
- perf(contrib/pgx.v5): cache connection config to avoid deep-copying on every span by @eric-weaver in #4747
- refactor(contrib/aerospike): use orchestrion method-call aspect for context propagation by @rarguelloF in #5061
- feat(contrib/net/http): add option to wrap a provided *http.ServeMux by @powerbill in #2690
- fix(ddtrace/tracer): harden baggage and tracestate propagation by @darccio in #5063
- fix(contrib): bound gzip-decompressed size in Elasticsearch body snippets by @darccio in #5159
- fix(contrib): use each body's own Content-Encoding in Elasticsearch peek() by @darccio in #5160
- fix(contrib/twmb/franz-go): release finished consume span pointers from activeSpans by @darccio in #5193
- fix(contrib/valyala/fasthttp): stop leaking response body into span error tag on 5xx by @darccio in #5305
- fix(contrib/valyala/fasthttp): obfuscate query string in http.url tag by @darccio in #5304
- perf(ddtrace/tracer): add WithTags to reduce per-tag closure allocations by @darccio in #5007
- perf(contrib/database/sql): cache static span tags via WithStartSpanConfig by @darccio in #5201
- perf(contrib): cache static span tags via WithStartSpanConfig (Kafka contribs) by @darccio in #5198
- perf(contrib): cache static span tags via WithStartSpanConfig (Mongo contribs) by @darccio in #5200
- perf(contrib): cache static span tags via WithStartSpanConfig (Redis + Valkey contribs) by @darccio in #5199
- feat(telemetry): dogfood the SDK error-reporting API from #4997 by @darccio in #5251
- refactor(config): migrate tracingEnabled by @mtoffl01 in #4854
- fix: reduce heap allocations introduced by ETP by @hannahkm in #4960
- refactor(config): migrate remaining unmigrated tracer configs to internal/config by @mtoffl01 in #4873
- fix(contrib/confluentinc/confluent-kafka-go/kafka.v2): match existing behavior for Producers when instrumenting via Orchestrion by @sendqueery in #4876
- feat(tracer): add WithStatsdClient option for sharing statsd clients by @jessefeinman in #4935
- refactor(config): migrate traceSamplingRules to internal/config by @mtoffl01 in #4885
- feat(ddtrace/tracer): adaptive pre-grow for
payloadV04 and payloadV1 buffers by @darccio in #4981
- refactor(config): migrate DD_SITE to internal/config by @brianwardwell in #4828
- perf(ddtrace/tracer): avoid per-header ToLower allocs during context extraction by @darccio in #5005
- perf(ddtrace/tracer): drop per-extract ToLower in parseTraceparent by @darccio in #5010
- fix(orchestrion): fix flaky TestCmdi server startup race by @darccio in #5018
- feat(opentelemetry): implement RecordError on the OTel bridge span by @link04 in #4892
- feat(tracer): add DD_TRACE_OTEL_SEMANTICS_ENABLED for OTLP export by @link04 in #4889
- feat: ability to externally register instrumentations by @RomainMuller in #5023
- fix(tracer): export span events natively over OTLP by @link04 in #4893
- refactor(config): migrate DD_TRACE_PROPAGATION_* configs to internal/config by @mtoffl01 in #4872
- feat: expose
TelemetryClient from the instrumentation package by @RomainMuller in #5030
- feat(config): add OTLP span metrics config by @rachelyangdog in #4895
- fix(ddtrace/tracer): make SpanAttributes layout assertion architecture-aware (#4984) by @kakkoyun in #5008
- perf(httptrace): reduce per-request allocations on the HTTP hot path by @kakkoyun in #5040
- perf(ddtrace/tracer): avoid heap-allocating SpanContext on failed extraction by @darccio in #5015
- perf(ddtrace/tracer): avoid Msgsize() walk when pre-sizing v0.4/v1.0 payload buffer by @darccio in #5016
- feat(tracer): stats to OTLP histogram conversion for OTLP span metrics by @rachelyangdog in #4897
- feat(tracer): implement cardinality limits for client-side stats by @Sam-Maya in #5032
- feat(instrumentation): expose telemetry.Product* functions by @RomainMuller in #5058
- feat(ddtrace/tracer): expose Span.SetMetaStruct by @RomainMuller in #5050
- feat(tracer): apply agent-advertised trace filters for client-side stats by @ichinaski in #5035
- perf(ddtrace/tracer): make propagatingTags reads lock-free by @darccio in #5059
- fix(ddtrace/tracer): don't let an inferred parent override an explicit ChildOf by @kakkoyun in #5078
- test(ddtrace/tracer): cover span pool activation surface by @darccio in #5085
- feat(tracer): add OTLP metrics HTTP exporter for span metrics by @rachelyangdog in #4899
- fix(internal/orchestrion): exit GLS scopes by token, not by stack position by @kakkoyun in #5082
- fix(internal/orchestrion): move the GLS liveness bit from the span to the stack entry by @kakkoyun in #5086
- feat(tracer): wire OTLP span metrics into tracer flush path by @rachelyangdog in #4900
- feat(instrumentation): expose stack trace recording by @RomainMuller in #5089
- feat(tracer): report OTLP export status in startup log by @bm1549 in #5062
- test(net/http): add cross-goroutine client integration test by @rarguelloF in #5096
- perf(ddtrace/tracer): avoid heap allocations in propagatorBaggage.extractTextMap by @darccio in #5119
- fix(stacktrace): parse value receivers by @RomainMuller in #5127
- feat(ddtrace/tracer): emit and propagate OpenTelemetry ot.th/ot.rv sampling by @genesor in #5060
- feat(ddtrace/tracer): carry ot.th/ot.rv and sampled flag on OTLP export by @genesor in #5093
- test(ddtrace/tracer): de-flake TestPartialFlushSpanLockOrderingCycle via deterministic acquire hook by @darccio in #4929
- test(ddtrace/x/agenttest): make the mock agent's /info response configurable by @darccio in #5146
- refactor(ddtrace/tracer): derive the trace-intake URL from the payload's protocol by @darccio in #5147
- refactor(internal/config): rename TraceProtocol to RequestedTraceProtocol by @darccio in #5148
- fix(ddtrace/tracer): propagate baggage when extract-first is enabled by @darccio in #5120
- fix(ddtrace/tracer): align OTLP trace-metrics export with span metrics connector conventions by @mabdinur in #5130
- fix(ddtrace/tracer): make OTLP metric attrs mode-independent by @mabdinur in #5155
- fix(internal/config): report derived stats-computation to config telemetry by @darccio in #5141
- fix(internal/config): stop forcing v0.4 when OTLP span metrics are enabled by @darccio in #5143
- fix(ddtrace/tracer): instrument payload queue-full trace drops by @darccio in #5136
- fix(ddtrace/tracer): decouple trace protocol v1.0 from client-side stats by @darccio in #5161
- fix(ddtrace/tracer): clear span-context snapshot on ContextWithSpan(ctx, nil) by @darccio in #5162
- fix(ddtrace/tracer): return after nil guard in Span.Format by @darccio in #5170
- fix(otel): correct telemetry namespace for otel metrics/log_records by @mtoffl01 in #5188
- perf(ddtrace/tracer): collapse ContextWithSpan's two context nodes into one by @darccio in #5164
- perf(ddtrace/opentelemetry): batch baggage in oteltracer.Start by @mtoffl01 in #5190
- chore(telemetry): fix known-metrics generator and regenerate from ddoghq/dd-go by @Sam-Maya in #5191
- fix(ddtrace/tracer): re-evaluate agent v1.0 support at runtime by @darccio in #5167
- fix(ddtrace/tracer): force client-side stats on 7.77/7.78 agents using v1.0 protocol by @darccio in #5179
- fix(ddtrace/opentelemetry): preserve pprof labels by @rarguelloF in #5215
- fix(opentelemetry): preserve error.type under OTel semantics by @link04 in #5037
- fix(opentelemetry): set OTel sampling state (ot.th/ot.rv) in AlwaysOn sampler by @vpellan in #5224
- feat(crashtracker): add Go crashtracker API by @kakkoyun in #5026
- fix(ddtrace/tracer): settle writer payload before checking it in downgrade race test by @darccio in #5252
- test(ddtrace/tracer): drain response body before close in TestConcurrentTraceFlushOverUDS by @darccio in #5248
- test(ddtrace/tracer): add BenchmarkStartSpanManyTags by @darccio in #5196
- feat(config): add cross-product Start telemetry (data-gathering) by @mtoffl01 in #5033
- feat(internal/log/telemetry): add SDK error-reporting API for Error Tracking by @darccio in #4997
- fix(contrib/jackc/pgx.v5): forward TraceRelease to the wrapped tracer by @VashDD in #5317
- chore: upgrade Orchestrion to v1.13.0 by @kakkoyun in #5326
- feat(crashtracker): orchestrion auto-injection aspect by @kakkoyun in #5027
- test(ddtrace/tracer): retry sends in span pool concurrent correctness test by @darccio in #5342
- fix(internal/telemetry): capture stacktraces at the call site, not at replay by @darccio in #5099
- refactor(ddtrace/tracer): remove dead Dynamic Instrumentation uprobe relay by @darccio in #5257
Application and API Protection (AAP)
- fix(appsec): keep first redirect hop for redirection derivatives by @eliottness in #4972
- feat(appsec): add command injection (CMDi) RASP via os.StartProcess by @eliottness in #4978
- chore(appsec): upgrade go-libddwaf to v5.0.0 across all modules by @eliottness in #4985
- fix(appsec): resolve the client IP once at the instrumentation boundary, and trust GCP's reported peer by @eliottness in #5118
- perf(appsec): replace EqualFold header scan with O(1) canonical map lookup by @kakkoyun in #5039
- fix(appsec): acknowledge streamed ext_proc response bodies by @eliottness in #5183
- fix(appsec): bound service extension memory to the container limit by @eliottness in #5228
- fix(appsec): only drain unused ext_proc body chunks for GCP by @eliottness in #5210<
Note truncated.