NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
crates.io · #428 most downloaded on crates.io
Criterion's plotting library
Last release 8 months ago
04 Feb 2026
Release timing varies
gaps range from 2 months to 2.9 years
Most releases are documented
notes for 19 of 25 stable releases
1 version withdrawn
withdrawn after publishing
11 years old
26 releases · first in 2016
Other Update Readme
Fixed Typo
One column per quarter.
Bump MSRV to 1.86, stable to 1.91.1
Throughput::ElementsAndBytes allows the text summary to report throughput in both units simultaneously.Bump version of criterion-plot to align dependencies.
The real_blackbox feature no longer has any impact. Criterion always uses std::black_box() now. Users of criterion::black_box() should switch to std::
real_blackbox feature no longer has any impact. Criterion always uses std::hint::black_box() now.criterion::black_box() should switch to std::hint::black_box().clap dependency unpinned.tokio::runtime::Handle, not only a tokio::runtime::RuntimeReplaced lazy_static dependency with once_cell
html_reports feature--quick) no longer outputs 1ms for measured times over 5 secondsNothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
The Criterion::can_plot function has been removed.
Criterion::can_plot function has been removed.Criterion::bench_function_over_inputs function has been removed.Criterion::bench_functions function has been removed.Criterion::bench function has been removed.rayon and plotters are optional (and default) dependencies.--warm-up-time, --measurement-time and --profile-time.--discard-baseline flag for discarding rather than saving benchmark results.--quiet flag for printing just a single line per benchmark.Throughput::BytesDecimal option for measuring throughput in bytes but printing them using
decimal units like kilobytes instead of binary units like kibibytes.bench_with_input, the input parameter will now be passed through black_box before
passing it to the benchmark.Added new plotting backend using the plotters crate. Implementation generously provided by Hao Hou, author of the plotters crate.
plotters crate. Implementation generously provided by Hao
Hou, author of the plotters crate.--plotting-backend command-line option to select the plotting backend. The existing
gnuplot backend will be used by default when available, and the plotters backend will be used when
gnuplot is not available or when requested.Criterion::plotting_backend() function to configure the plotting backend in code.--load-baseline command-line option to load a baseline for comparison
rather than measuring the current codefibonacci functions.#[criterion] benchmarks ignoring the command-line options.nresamples is set too low. Also added a warning
against setting nresamples too low.All of these types/functions are now soft-deprecated (meaning they're hidden from the documentation and should not be used in new code). They will be…
cpuprofiler which must be explicitly started and stopped within the profiled
process.BenchmarkGroup type, which supersedes ParameterizedBenchmark, Benchmark,
Criterion::bench_functions, Criterion::bench_function_over_inputs, and Criterion::bench.
BenchmarkGroup performs the same function as all of the above, but is cleaner to use and more
powerful and flexible. All of these types/functions are now soft-deprecated (meaning they're
hidden from the documentation and should not be used in new code). They will be fully deprecated
at some point in the 0.3.* series and removed in 0.4.0.iter_custom - a "timing loop" that allows the caller to perform their own measurements. This is
useful for complex measurements that don't fit into the usual mode of calling a lambda in a loop.throughput_num and throughput_type have been added to the raw.csv file.raw.csv file format has been changed slightly. The sample_time_nanos field has been split
into sample_measured_value and unit fields to accommodate custom measurements.--measure-only command-line argument; it was deprecated in favor of --profile-time
in 0.2.6.iter_custom timing loop can be used as a substitute; see benches/external_process.rs for an
example of this.--test argument is now deprecated. To test benchmarks, use cargo test --benches.Fixed version numbers to prevent incompatibilities between criterion and criterion-stats crates.
criterion and criterion-stats
crates.This is necessary to prevent the compiler from optimizing away the benchmark. This is technically a breaking change - that function requires a new typ…
--list command line option, which lists the benchmarks but does not run them, to match
cargo test -- --list.--profile-time command-line argument which disables reporting and analysis and instead
simply iterates each benchmark for approximately the given number of seconds. This supersedes the
(now-deprecated) --measure-only argument.Bencher::iter_with_large_setup can now return output. This is necessary to
prevent the compiler from optimizing away the benchmark. This is technically a breaking change -
that function requires a new type parameter. It's so unlikely to break existing code that I
decided not to delay this for a breaking-change release.iter_with_large_setup and iter_with_drop methods.criterion_group and criterion_main macros no longer require the Criterion struct to be
explicitly imported.gnuplot --version fails.use criterion::Criterion;gnuplot 5.2 patchlevel 5a (Gentoo revision r0)Criterion::without_plots, --noplot or disabling the
HTML reports feature)iter_with_large_setup to exclude time spent dropping values returned
by the routine. Time measurements taken with 0.2.6 using these methods may differ from those taken
with 0.2.5.--measure-only command-line-argument in favor of --profile-time. This will be
removed in 0.3.0.html_reports cargo feature is now deprecated. This feature will become non-optional in 0.3.0.Fixed links from generated report files to documentation.
cargo clean; cargo bench -- <filter> would cause Criterion.rs to log
an error message.CARGO_TARGET_DIR environment variable.Other parts of the code require this and will panic if the sample size is 1, so this is not considered to be a breaking change.
--save-baseline and --baseline, which change
how benchmark results are stored and compared. This is useful for
working against a fixed baseline(eg. comparing progress on an
optimization feature branch to the commit it forked from).
Default behavior of Criterion.rs is now --save-baseline base
which emulates the previous, user facing behavior.
--save-baseline saves the benchmark results under the provided name.--baseline compares the results to a saved baseline.
If the baseline does not exist for a benchmark, an error is given.--test, which causes Criterion to execute the benchmarks once
without measuring or reporting the results. This is useful for checking that the
benchmarks run successfully in a CI setting.raw.csv file to the output which contains a stable, machine-readable
representation of the measurements taken by benchmarks. This enables users to
perform their own analysis or keep historical information without depending on
private implementation details.sample_size method on the Criterion, Benchmark and
ParameterizedBenchmark structs has been changed to panic if the sample size
is less than 2. Other parts of the code require this and will panic if the
sample size is 1, so this is not considered to be a breaking change.Criterion.rs will now panic with a clear error message if the user attempts to run a benchmark which doesn't call the Bencher::iter function or a rela
Bencher::iter function or a related function,
rather than failing in an uncontrolled manner later.--measure-only argument which causes the benchmark executable to run the
warmup and measurement and then move on to the next benchmark without analyzing or
saving data. This is useful to prevent Criterion.rs' analysis code from appearing
in profile data when profiling benchmarks.Fixed broken links in some summary reports.
HTML reports are now a default Cargo feature. If you wish to disable HTML reports, disable Criterion.rs' default features. Doing so will allow compati
report folder.Added Criterion.bench function, which accepts either a Benchmark or ParameterizedBenchmark. These new structures allow for custom per-benchmark config
Criterion.bench function, which accepts either a Benchmark or
ParameterizedBenchmark. These new structures allow for custom per-benchmark
configuration as well as more complex benchmark grouping (eg. comparing a Rust
function against an external program over a range of inputs) which was not
possible previously.Benchmark.throughput and
ParameterizedBenchmark.throughput functions for further details.--noplot command line option to disable plot generation.Benchmark or ParameterizedBenchmark types to do per-benchmark configuration
instead.Criterion.bench_* must now have a 'static lifetime.
This means that you may need to change your closures from |bencher| {...}
to move |bencher| {...}.Criterion.bench_functions now takes I as an input parameter, not &I.Debug rather than Display.target/criterion rather than .criterion.criterion::ConfidenceInterval andcriterion::Estimate types are
no longer publicly accessible.Criterion.summarize function has been removed.Nothing published for this version
Criterion.rs is now stable-compatible!
black_box function.
Some benchmarks may now be affected by dead-code-elimination where they
previously weren't and may have to be updated.serde to save results. Existing results files will
be automatically removed when benchmarks are run.Added a chapter to the book on how Criterion.rs collects and analyzes data.
cargo bench.
Benchmarks defined without these macros should continue to work.[Unreleased]: https://github.com/bheisler/criterion.rs/compare/0.4.0...HEAD [0.1.1]: https://github.com/bheisler/criterion.rs/compare/0.1.0...0.1.1 [0
Your coding agent can read these notes before it upgrades. Set up the MCP server →