github.com/stripe/veneur

Improvements

• Veneur now tracks statsd metrics for SSF spans concerning its own operation. This means that the veneur.ssf.spans.received_total counter and the veneur.ssf.packet_size histogram again reflect trace spans routed internally. Thanks, antifuchs!

Added

• New 'blackhole' sink for testing and benchmark purposes. Thanks gphat!

Improvements

• Veneur no longer requires the use of Datadog as a target for flushes. Veneur can now use one or more of any of its supported sinks as a backend. This realizes our desire for Veneur to be fully vendor agnostic. Thanks gphat!
• The package github.com/stripe/veneur/trace now depends on fewer other packages across veneur, making it easier to pull in trace as a dependency. Thanks antifuchs!
• A Veneur server with tracing enabled now submits traces and spans concerning its own operation to itself internally without sending them over UDP. See the "Upgrade Notes" section below for metrics affected by this change. Thanks antifuchs!
• veneur-prometheus and veneur-proxy executables are now included in the docker images. Thanks jac
• All Veneur executables are now in $PATH in the docker images. Thanks jac
• When using Lightstep as a tracing sink, spans can be load-balanced more evenly across collectors by configuring the trace_lightstep_num_clients option to multiplex across multiple clients. Thanks aditya!
• sync_with_interval is a new configuration option! If enabled, when starting, Veneur will now delay its first metric to be aligned with an interval boundary on the local clock. This will effectively "synchronize" Veneur instances across your deployment assuming reasonable clock behavior. The result is a metric timestamps in your TSDB that mostly line up improving bucketing behavior. Thanks gphat!
• Cleaned up some linter warnings. Thanks gphat!
• Tests no longer depend on implicit presence of a Datadog metric or span sink. Thanks gphat!
• Refactor internal HTTP helper into its own package fix up possible circular deps. Thanks gphat!

Bugfixes

• Fix a panic when using veneur-emit to emit metrics via -ssf when no tags are specified. Thanks myndzi
• Remove spurious warnings about unset configuration settings. Thanks antifuchs

Removed

• Removed the InfluxDB plugin as it was experimental and wasn't working. We can make a sink for it in the future if desired. Thanks gphat!

Notes for upgrading from previous versions

• The set data structure serialization format for communiation with a global Veneur server has changed in an incompatible way. If your infrastructure relies on a global Veneur installation, they will drop set data from non-matching versions until the entire fleet and the global Veneur are all at the same version.
• The metrics for SSF packets (and spans) received have changed names: They used to be veneur.packet.received_total and veneur.packet.spans.received_total, respectively, and they are now named veneur.ssf.received_total and veneur.ssf.spans.received_total.

Added

• New configuration option statsd_listen_addresses, a list of URIs indicating on which ports (and protocols) Veneur should listen on for statsd metrics. This deprecates both the udp_address and tcp_address settings. Thanks antifuchs!
• New package github.com/stripe/veneur/protocol, containing a wire protocol for sending/reading SSF over a streaming connection. Thanks antifuchs!
• github.com/veneur/trace now contains customizable Clients that support streaming connections.
• veneur-prometheus now has a -p option for specifying a prefix for all metrics. Thanks gphat!
• New metrics ssf.spans.tags_per_span and ssf.packet_size track the distribution of tags per span and total packet sizes, respectively.
• Our super spiffy logo, designed by mercedes, is now included at the top of the README!
• Refactor internals to use a new intermediary metric struct to facilitate new plugins. Thanks gphat!
• The BUILD_DATE and VERSION variables can be set at link-time and are now exposed by the /builddate and /version endpoints.

Improvements

• A new HyperLogLog implementation means sets are faster and allocate less memory. Thanks, martinpinto and seiflotfy!
• Introduced a new metricSink which provides a common interface for metric backends. In an upcoming release all plugins will be converted to this interface. Thanks gphat!

Deprecations

• veneur-emit no longer supports the -config argument, as it's no longer possible to reliably detect which statsd host/port to connect to. The -hostport option now takes a URL of the same form statsd_listen_addresses takes to explicitly tell it what address it should send to.
• SSFSpanCollection got removed, as it is superseded by the wire protocol. If you need to send multiple spans in bulk, we recommend setting up a buffered trace.Client!

Added

• Veneur-emit can now time any shell command and emit its duration as a Timing metric. Thanks redsn0w422!
• Config options can now be provided via environment variables using envconfig for Veneur and veneur-proxy. Thanks gphat!
• SSF now includes a boolean indicator field for signaling that this span is useful as a Service Level Indicator for its service.
• A type SSFSpanCollection has been added but is not yet used.
• The veneur-prometheus command can be used to scrape prometheus endpoints and emit those metrics to Veneur. Thanks gphat and jvns

Improvements

As a result of the efficiency improvements in this release, we've seen ~50% reduction in memory usage by way of measuring the allocated heap.

Secondly, the shift in not buffering spans on their way to LightStep should be noted. This changes behavior in Veneur which has traditionally done everything in 10s increments.

• If possible, initialization errors when starting Veneur will now be reported to Sentry. Thanks chimeracoder!
• Check return value of LightStep flush. Thanks chimeracoder!
• No longer using a fork of Logrus that fixed a race condition. Thanks chimeracoder!
• Updated to latest version of LightStep's tracing library, which drastically improves the success of spans on hosts with high span rates. Thanks gphat!
• No longer buffers spans for LightStep, they are dispatched directly to the LightStep client. Thanks gphat!
• Reuse an existing buffer when parsing incoming spans, reducing allocations. Thanks gphat!
• Use gogo protobuf for code generation of SSF's protobuf, resulting in faster and less memory span ingestion. Thanks gphat!

Bugfixes

• veneur-proxy no longer balks at using static hosts for tracing and metrics. Thanks gphat!

Deprecations

• SSF's operation field has been deprecated in favor of the field name.
• SSF spans with a tag name will have that name placed into the SSF span name field until 2.0 is released.

Bugfixes

• Correctly parse Set metrics if sent via SSF. Thanks redsn0w422!
• Return correct array of tags after parsing an SSF metric. Thanks redsn0w422!
• Don't panic if a packet doesn't have tags. Thanks redsn0w422!
• Fix a typo in the link to veneur-emit in the readme. Thanks vasi!

Added

• Adds events and service checks to veneur-emit. Thanks redsn0w422!
• Switch to dep for managing the vendor directory. Thanks chimeracoder!
• Remove support for govendor. Thanks chimeracoder!
• Emit GC metrics at flush time. This feature is best used with Go 1.9 as previous versions cause GC pauses when collecting this information from go. Thanks gphat!
• Allow configuration of LightStep's reconnect period via trace_lightstep_reconnect_period and the maximum number of spans it will buffer via trace_lightstep_maximum_spans. Thanks gphat!
• Switch to dep for managing the vendor directory. Thanks chimeracoder!
• Remove support for govendor. Thanks chimeracoder!
• Added harmonic mean as an optional aggregate type. Use hmean as an option to the aggregates config option to enable. Thanks non!

Improvements

• Added tests for parseMetricSSF. Thanks redsn0w422!
• Refactored veneur-emit flag usage to make testing easier. Thanks redsn0w422!
• Minor text fixes in the README. Thanks an-stripe!
• Restructured SSF parsing logic and added more tests. Thanks redsn0w422!
• Tag packet.spans.received_total with service from the span. Thanks chimeracoder!

Bugfixes

• Flush the lightstep tracer before closing it. Thanks gphat with assist from stangles!

Improvements

• Better document how to configure Veneur as a DogStatsD replacement. Thanks gphat with assist from stangles!

Bugfixes

• Fixed an error in graceful shutdown of the TCP listener. Thanks evanj!
• Don't hang if we call log.Fatal and we aren't hooked up to a Sentry. Thanks evanj!
• Fix flusher_test being called more than once resulting in flappy failure. Thanks evanj!
• Improve flusher test to not start Veneur, fixing flapping test. Thanks evanj!

Added

• veneur-emit can now emit metrics using the SSF protocol. Thanks redsn0w422!
• Documentation for SSF. Thanks gphat!

Improvements

• It is no longer required to emit a sum to get an avg when configuring what aggregations to emit for a histogram. Thanks cgilling!
• Tags added in the tags config key are now applied to trace spans. Thanks chimeracoder!
• Additional documentation for veneur-proxy. Thanks gphat!
• Revamped configuration file organization and comments. Thanks gphat!
• Changed some config keys to have more specific names to facilitate future refactoring. Thanks gphat!
• Adjust the flush loop to listen for server shutdown to improve test consistency. Thanks evanj!
• Veneur can now, experimentally, ingest metrics using the SSF protocol. Thanks redsn0w422!
• Reresolve the LightStep trace flusher on each flush, accomodating Consul-based DNS use and preventing stale sinks. Thanks chimeracoder!

Deprecations

• The following configuration keys are deprecated and will be removed in version 2.0 of Veneur:
  • datadog_api_key replaces key
  • datadog_api_hostname replaces api_hostname
  • datadog_trace_api_address replaces trace_api_address
  • ssf_address replaces trace_address

Bugfixes

• Decrease logging level for proxy's "forwarded" messages. Thanks gphat!
• Failed discovery refreshes now log the service name. Thanks gphat!

Improvements

• Proxy no longer requires a trace service name, since it's not wired up. Thanks gphat!

Bugfixes

• Fix flusher_test to properly shutdown HTTP after handling. Thanks evanj!
• Verify that trace_max_length_bytes is properly set. Thanks evanj!
• Fix some race conditions in testing.

Improvements

• Document performance cost of TLS with RSA and ECDH keys. Thanks evanj!
• Reduce logging of tracing information to debug level to decrease unnecessary logging.
• Reduce common TCP error logs to info level. Thanks evanj!
• Deal with server shutdown without inspecting errors strings. Thanks evanj!
• Decrease the number of things we send to Sentry as "errors".
• Detect and emit a metric veneur.packet.error_total tagged reason:toolong for metrics that exceed the metric max length.
• Emit a metric veneur.packet.error_total tagged reason:zerolength for metrics have no contents.
• Correct unnecessary allocation / goroutine in TCP connections that was leaking memory. Thanks evanj!
• Close idle TCP connections after 10 minutes. Thanks evanj!
• Fixed a lot of go lint errors.

Added

• Add a metric veneur.sentry.errors_total for number of errors we send to Sentry.
• New plugin flush_file for writing metrics to a flat file.
• New /healthcheck/tracing endpoint that returns 200 if this Veneur instance is accepting traces.

Changes

• Refactor tests to use a more shareable test fixture. Thanks evanj!
• Refactor Server's constructor to not start any goroutines and add a Start() that takes care of that, making for easier tests.

Bugfixes

• Hostname and device name tags are now omitted from JSON generated for transmission to Datadog at flush time. Thanks evanj!
• Fix panic when an error is generated and Sentry is not configured. Thanks evanj!
• Fix typos in README

Improvements

• Add omit_empty_hostname option. If true and hostname tag is set to empty, Veneur will not add a host tag to its own metrics. Thanks evanj!
• Support "all interfaces" addresses (:1234) for listening configuration. Thanks evanj!
• Add support for receiving statsd packets over authenticated TLS connections. Thanks evanj!
• [EXPERIMENTAL] Add InfluxDB support.
• [EXPERIMENTAL] Add support for ingesting traces and sending to Datadog's APM agent.