NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
crates.io · #404 most downloaded on crates.io
Statistics-driven micro-benchmarking library
Last release 8 months ago
04 Feb 2026
Release timing varies
gaps range from 2 months to 2.0 years
Nearly every release is documented
notes for 29 of 29 stable releases
1 version withdrawn
withdrawn after publishing
9 years old
30 releases · first in 2017
don't build alloca on unsupported targets
(deps) bump crate-ci/typos from 1.23.5 to 1.40.0
One column per quarter.
Bump MSRV to 1.86, stable to 1.91.1
Throughput::ElementsAndBytes allows the text summary to report throughput in both units simultaneously.Bump version of criterion-plot to align dependencies.
The real_blackbox feature no longer has any impact. Criterion always uses std::black_box() now. Users of criterion::black_box() should switch to std::
real_blackbox feature no longer has any impact. Criterion always uses std::hint::black_box() now.criterion::black_box() should switch to std::hint::black_box().clap dependency unpinned.tokio::runtime::Handle, not only a tokio::runtime::RuntimeQuick mode (--quick) no longer crashes with measured times over 5 seconds when --noplot is not active
Replaced lazy_static dependency with once_cell
html_reports feature--quick) no longer outputs 1ms for measured times over 5 secondsThe Criterion::can_plot function has been removed.
Criterion::can_plot function has been removed.Criterion::bench_function_over_inputs function has been removed.Criterion::bench_functions function has been removed.Criterion::bench function has been removed.rayon and plotters are optional (and default) dependencies.--warm-up-time, --measurement-time and --profile-time.--discard-baseline flag for discarding rather than saving benchmark results.--quiet flag for printing just a single line per benchmark.Throughput::BytesDecimal option for measuring throughput in bytes but printing them using
decimal units like kilobytes instead of binary units like kibibytes.bench_with_input, the input parameter will now be passed through black_box before
passing it to the benchmark.Symbol for microseconds changed from ASCII 'us' to unicode 'µs'
Corrected Criterion.toml in the book.
Criterion.toml in the book.Criterion::can_plot is no longer useful and is deprecated pending deletion in 0.4.0.
with_output_color for enabling or disabling CLI output coloring programmatically.--help output regarding "unrecognized option" errors.Criterion::can_plot is no longer useful and is deprecated pending deletion in 0.4.0.Benchmark and ParameterizedBenchmark were already hidden from documentation, but are now
formally deprecated pending deletion in 0.4.0. Callers should use BenchmarkGroup instead.Criterion::bench_function_over_inputs, Criterion::bench_functions, and Criterion::bench were
already hidden from documentation, but are now formally deprecated pending deletion in 0.4.0.
Callers should use BenchmarkGroup instead.--message-format=json option for machine-readable output instead. A warning message will be
printed at the start of benchmark runs which do not have "html_reports" or "cargo_bench_support"
enabled, but because CSV output is not widely used it has no warning.Added CRITERION_HOME environment variable to set the directory for Criterion to store its results and charts in.
CRITERION_HOME environment variable to set the directory for Criterion to store
its results and charts in.cargo-criterion that are
difficult to implement in Criterion-rs.Added ?Sized bound to benchmark parameter types, which allows dynamically sized types like &str and &[T] to be used as benchmark parameters.
?Sized bound to benchmark parameter types, which allows dynamically sized types like
&str and &[T] to be used as benchmark parameters.--output-format <format> command-line option. If --output-format bencher is passed,
Criterion.rs will print its measurements in a format similar to that used by the bencher crate
or unstable libtest benchmarks, and using similar statistical measurements as well. Though this
provides less information than the default format, it may be useful for supporting tools which
parse this output format.--nocapture argument. This argument does nothing, but prevents Criterion.rs from exiting
when running tests or benchmarks and allowing stdout output from other tests.CRITERION_DEBUG or CRITERION_TARGET_DIR environment variables contain
non-UTF8 characters.Added new plotting backend using the plotters crate. Implementation generously provided by Hao Hou, author of the plotters crate.
plotters crate. Implementation generously provided by Hao
Hou, author of the plotters crate.--plotting-backend command-line option to select the plotting backend. The existing
gnuplot backend will be used by default when available, and the plotters backend will be used when
gnuplot is not available or when requested.Criterion::plotting_backend() function to configure the plotting backend in code.--load-baseline command-line option to load a baseline for comparison
rather than measuring the current codefibonacci functions.#[criterion] benchmarks ignoring the command-line options.nresamples is set too low. Also added a warning
against setting nresamples too low.All of these types/functions are now soft-deprecated (meaning they're hidden from the documentation and should not be used in new code). They will be…
cpuprofiler which must be explicitly started and stopped within the profiled
process.BenchmarkGroup type, which supersedes ParameterizedBenchmark, Benchmark,
Criterion::bench_functions, Criterion::bench_function_over_inputs, and Criterion::bench.
BenchmarkGroup performs the same function as all of the above, but is cleaner to use and more
powerful and flexible. All of these types/functions are now soft-deprecated (meaning they're
hidden from the documentation and should not be used in new code). They will be fully deprecated
at some point in the 0.3.* series and removed in 0.4.0.iter_custom - a "timing loop" that allows the caller to perform their own measurements. This is
useful for complex measurements that don't fit into the usual mode of calling a lambda in a loop.throughput_num and throughput_type have been added to the raw.csv file.raw.csv file format has been changed slightly. The sample_time_nanos field has been split
into sample_measured_value and unit fields to accommodate custom measurements.--measure-only command-line argument; it was deprecated in favor of --profile-time
in 0.2.6.iter_custom timing loop can be used as a substitute; see benches/external_process.rs for an
example of this.--test argument is now deprecated. To test benchmarks, use cargo test --benches.Enabled automatic text-coloring on Windows.
Criterion::iter_batched/iter_batched_ref.iter_with_setup/iter_with_large_setup are now deprecated in favor of iter_batched.
iter_batched/iter_batched_ref timing loops, which allow for setup (like
iter_with_setup/iter_with_large_setup) and exclude drop (like iter_with_large_drop) but
measure the runtime more accurately, use less memory and are more flexible.iter_with_setup/iter_with_large_setup are now deprecated in favor of iter_batched.Criterion.rs no longer depends on the default features of the rand-core crate. This fixes some downstream crates which use rand in a no_std context.
rand-core crate. This fixes some
downstream crates which use rand in a no_std context.Criterion.rs now uses rayon internally instead of manual unsafe code built with thread-scoped.
rayon internally instead of manual unsafe code built with thread-scoped.criterion-stats crate into criterion crate. criterion-stats will no longer receive
updates.Fixed version numbers to prevent incompatibilities between criterion and criterion-stats crates.
criterion and criterion-stats
crates.This is necessary to prevent the compiler from optimizing away the benchmark. This is technically a breaking change - that function requires a new typ…
--list command line option, which lists the benchmarks but does not run them, to match
cargo test -- --list.--profile-time command-line argument which disables reporting and analysis and instead
simply iterates each benchmark for approximately the given number of seconds. This supersedes the
(now-deprecated) --measure-only argument.Bencher::iter_with_large_setup can now return output. This is necessary to
prevent the compiler from optimizing away the benchmark. This is technically a breaking change -
that function requires a new type parameter. It's so unlikely to break existing code that I
decided not to delay this for a breaking-change release.iter_with_large_setup and iter_with_drop methods.criterion_group and criterion_main macros no longer require the Criterion struct to be
explicitly imported.gnuplot --version fails.use criterion::Criterion;gnuplot 5.2 patchlevel 5a (Gentoo revision r0)Criterion::without_plots, --noplot or disabling the
HTML reports feature)iter_with_large_setup to exclude time spent dropping values returned
by the routine. Time measurements taken with 0.2.6 using these methods may differ from those taken
with 0.2.5.--measure-only command-line-argument in favor of --profile-time. This will be
removed in 0.3.0.html_reports cargo feature is now deprecated. This feature will become non-optional in 0.3.0.Fixed links from generated report files to documentation.
cargo clean; cargo bench -- <filter> would cause Criterion.rs to log
an error message.CARGO_TARGET_DIR environment variable.Other parts of the code require this and will panic if the sample size is 1, so this is not considered to be a breaking change.
--save-baseline and --baseline, which change
how benchmark results are stored and compared. This is useful for
working against a fixed baseline(eg. comparing progress on an
optimization feature branch to the commit it forked from).
Default behavior of Criterion.rs is now --save-baseline base
which emulates the previous, user facing behavior.
--save-baseline saves the benchmark results under the provided name.--baseline compares the results to a saved baseline.
If the baseline does not exist for a benchmark, an error is given.--test, which causes Criterion to execute the benchmarks once
without measuring or reporting the results. This is useful for checking that the
benchmarks run successfully in a CI setting.raw.csv file to the output which contains a stable, machine-readable
representation of the measurements taken by benchmarks. This enables users to
perform their own analysis or keep historical information without depending on
private implementation details.sample_size method on the Criterion, Benchmark and
ParameterizedBenchmark structs has been changed to panic if the sample size
is less than 2. Other parts of the code require this and will panic if the
sample size is 1, so this is not considered to be a breaking change.Criterion.rs will now panic with a clear error message if the user attempts to run a benchmark which doesn't call the Bencher::iter function or a rela
Bencher::iter function or a related function,
rather than failing in an uncontrolled manner later.--measure-only argument which causes the benchmark executable to run the
warmup and measurement and then move on to the next benchmark without analyzing or
saving data. This is useful to prevent Criterion.rs' analysis code from appearing
in profile data when profiling benchmarks.Fixed broken links in some summary reports.
HTML reports are now a default Cargo feature. If you wish to disable HTML reports, disable Criterion.rs' default features. Doing so will allow compati
report folder.Added Criterion.bench function, which accepts either a Benchmark or ParameterizedBenchmark. These new structures allow for custom per-benchmark config
Criterion.bench function, which accepts either a Benchmark or
ParameterizedBenchmark. These new structures allow for custom per-benchmark
configuration as well as more complex benchmark grouping (eg. comparing a Rust
function against an external program over a range of inputs) which was not
possible previously.Benchmark.throughput and
ParameterizedBenchmark.throughput functions for further details.--noplot command line option to disable plot generation.Benchmark or ParameterizedBenchmark types to do per-benchmark configuration
instead.Criterion.bench_* must now have a 'static lifetime.
This means that you may need to change your closures from |bencher| {...}
to move |bencher| {...}.Criterion.bench_functions now takes I as an input parameter, not &I.Debug rather than Display.target/criterion rather than .criterion.criterion::ConfidenceInterval andcriterion::Estimate types are
no longer publicly accessible.Criterion.summarize function has been removed.Criterion.rs is now stable-compatible!
black_box function.
Some benchmarks may now be affected by dead-code-elimination where they
previously weren't and may have to be updated.serde to save results. Existing results files will
be automatically removed when benchmarks are run.Added a chapter to the book on how Criterion.rs collects and analyzes data.
cargo bench.
Benchmarks defined without these macros should continue to work.[Unreleased]: https://github.com/bheisler/criterion.rs/compare/0.4.0...HEAD [0.1.1]: https://github.com/bheisler/criterion.rs/compare/0.1.0...0.1.1 [0
Your coding agent can read these notes before it upgrades. Set up the MCP server →