minreq

This changelog is identical to 3.0.0-rc.0, pasted here for convenience.

This release includes many breaking changes as implied by the major version bump. See especially the Changed and Deleted sections below.

This release includes all the planned changes that have been accumulating over the years for 3.0, renames some features, but otherwise tries to keep everything the same as 2.0.

Changed

• MSRV from 1.48 to 1.63, and updated the policy to be looser (now tracking Debian oldstable's Rust version instead of settling on a specific version until eternity, and the MSRV no longer applies to any optional features).
• The minreq::Error type to be non-exhaustive, to make adding new errors later possible, and added new variants: NativeTlsCreateConnection, OpenSslCreateConnection, InvalidProtocol, and InvalidProtocolInRedirect.
• The default proxy port to 1080, to match curl.
• The default maximum sizes for the response status line and headers, both 8KiB.
• The type of Response::headers and ResponseLazy::headers from HashMap<String, String> to Vec<(String, String)>, and removed lowercase normalization of the field names.
  • To help the usual case, there's two new functions, Response::header and Response::headers, to get the header value (or all values, if there are multiple of the same header) by field name. In other cases, iterate through and use str::eq_ignore_ascii_case to find your header(s).
• The query parameter function to take AsRef<str> instead of Into<String> to avoid unnecessary allocations. Thanks for the PR, @alpha-tango-kilo! (#69)
• Response::status_code and ResponseLazy::status_code type from i32 to u16. Thanks for the suggestion, @ModProg!
  (#118)
• Renamed features https-bundled to https-openssl, https-bundled-probe to https-openssl-probe, and https-native to https-native-tls to make them a little more obvious in what they bring in.

Removed

• Iterator impl for ResponseLazy, see the addition section for more on the new Read impl. Thanks for the PR, @mrkline! (#104)

Added

• Response::header and Response::headers for ergonomically sorting through the headers of a response, now that they aren't in a convenient HashMap anymore.
• Read impl for ResponseLazy. This improves performance, and the Read trait generally matches the functionality better than Iterator. Thanks for the PR, @mrkline! (#104)

Changed

• MSRV from 1.48 to 1.63, and updated the policy to be looser (now tracking Debian oldstable's Rust version instead of settling on a specific version until eternity, and the MSRV no longer applies to any optional features).
• The minreq::Error type to be non-exhaustive, to make adding new errors later possible, and added new variants: NativeTlsCreateConnection, OpenSslCreateConnection, InvalidProtocol, and InvalidProtocolInRedirect.
• The default proxy port to 1080, to match curl.
• The default maximum sizes for the response status line and headers, both 8KiB.
• The type of Response::headers and ResponseLazy::headers from HashMap<String, String> to Vec<(String, String)>, and removed lowercase normalization of the field names.
  • To help the usual case, there's two new functions, Response::header and Response::headers, to get the header value (or all values, if there are multiple of the same header) by field name. In other cases, iterate through and use str::eq_ignore_ascii_case to find your header(s).
• The query parameter function to take AsRef<str> instead of Into<String> to avoid unnecessary allocations. Thanks for the PR, @alpha-tango-kilo! (#69)
• Response::status_code and ResponseLazy::status_code type from i32 to u16. Thanks for the suggestion, @ModProg! (#118)
• Renamed features https-bundled to https-openssl, https-bundled-probe to https-openssl-probe, and https-native to https-native-tls to make them a little more obvious in what they bring in.

Removed

• Iterator impl for ResponseLazy, see the addition section for more on the new Read impl. Thanks for the PR, @mrkline! (#104)

Added

• Response::header and Response::headers for ergonomically sorting through the headers of a response, now that they aren't in a convenient HashMap anymore.
• Read impl for ResponseLazy. This improves performance, and the Read trait generally matches the functionality better than Iterator. Thanks for the PR, @mrkline! (#104)

Nothing to see here, added some excludes in the Cargo.toml and cleaned up the rustls cert provider dependency feature gates. Just releasing these changes as an RC so that the final RC would be identical to the actual 3.0.0 release.

This is the first release candidate for 3.0. If no issues are found, I'll release 3.0 in a few weeks.

Includes all the planned changes that have been accumulating over the years for 3.0, renames some features, but otherwise tries to keep everything the same as 2.0. Note that everything under Changed and Removed here is a breaking change.

Changed

• MSRV from 1.48 to 1.63, and updated the policy to be looser (now tracking Debian oldstable's Rust version instead of settling on a specific version until eternity, and the MSRV no longer applies to any optional features).
• The minreq::Error type to be non-exhaustive, to make adding new errors later possible, and added new variants: NativeTlsCreateConnection, OpenSslCreateConnection, InvalidProtocol, and InvalidProtocolInRedirect.
• The default proxy port to 1080, to match curl.
• The default maximum sizes for the response status line and headers, both 8KiB.
• The type of Response::headers and ResponseLazy::headers from HashMap<String, String> to Vec<(String, String)>, and removed lowercase normalization of the field names.
  • To help the usual case, there's two new functions, Response::header and Response::headers, to get the header value (or all values, if there are multiple of the same header) by field name. In other cases, iterate through and use str::eq_ignore_ascii_case to find your header(s).
• The query parameter function to take AsRef<str> instead of Into<String> to avoid unnecessary allocations. Thanks for the PR, @alpha-tango-kilo! (#69)
• Response::status_code and ResponseLazy::status_code type from i32 to u16. Thanks for the suggestion, @ModProg!
  (#118)
• Renamed features https-bundled to https-openssl, https-bundled-probe to https-openssl-probe, and https-native to https-native-tls to make them a little more obvious in what they bring in.

Removed

• Iterator impl for ResponseLazy, see the addition section for more on the new Read impl. Thanks for the PR, @mrkline! (#104)

Added

• Response::header and Response::headers for ergonomically sorting through the headers of a response, now that they aren't in a convenient HashMap anymore.
• Read impl for ResponseLazy. This improves performance, and the Read trait generally matches the functionality better than Iterator. Thanks for the PR, @mrkline! (#104)

Should be a pretty transparent update, pruning the log dependency for dependents which do not use it, and keeping everything the same for the ones that do.

Fixed

• Made the log crate optional. Thanks for the contribution, @Zodey-hub! (#122)

Fixed

• Made the log crate optional. Thanks for the contribution, @Zodey-hub! (#122)

Cleaned up the https-bundled and https-bundled-probe code quite a bit. Now it's more "openssl backend" rather than "native-tls backend but vendored", so the features might change their names to https-openssl in the next major version, whenever that ends up happening.

Also, added a new option for requests, Request::with_follow_redirects, which can be used to disable redirection, for cases where you want to read the redirection HTTP responses themselves.

Changed

• https-bundled, https-bundled-probe: Removed almost all of the bundled native-tls code (~1k LoC), only keeping the relevant part (~30 LoC). There should be no change to the actual code that ends up being ran, but if you're using these features, make sure to test that everything works as you expect, something might have slipped.

Fixed

• https-*: Refactored the TLS handling code a bit. This should have no visible effect downstream, src/connection.rs is just a little bit more readable now.
• Removed build.rs, which turned out to be dead code. This should have no effect, but if it does, it should also only affect the https-bundled and https-bundled-probe features.

Added

• Request::with_follow_redirects for disabling redirection handling. Thanks for the suggestion, @tachibanayui! (#120)

Changed

• https-bundled, https-bundled-probe: Removed almost all of the bundled native-tls code (~1k LoC), only keeping the relevant part (~30 LoC). There should be no change to the actual code that ends up being ran, but if you're using these features, make sure to test that everything works as you expect, something might have slipped.

Fixed

• https-*: Refactored the TLS handling code a bit. This should have no visible effect downstream, src/connection.rs is just a little bit more readable now.
• Removed build.rs, which turned out to be dead code. This should have no effect, but if it does, it should also only affect the https-bundled and https-bundled-probe features.

Added

• Request::with_follow_redirects for disabling redirection handling. Thanks for the suggestion, @tachibanayui! (#120)

Fixed

• Updated the base64 dependency, only used by the proxy feature, to its newest minor version. Thanks for the report, @Jackhr-arch! (#119)

Fixed

• Updated the base64 dependency, only used by the proxy feature, to its newest minor version. Thanks for the report, @Jackhr-arch! (#119)

A bit of dependency cleanup, and updated the list of crates that need to be pinned to build with MSRV.

Fixed

• Removed the once_cell dependency by making use of the new std::sync::OnceLock type. This change only affects the rustls-based https features. Thanks for the PR, @LyonSyonII! (#115)
• MSRV builds that got broken due to a rustix update. Now tempfile is pinned as well.

Fixed

• Removed the once_cell dependency by making use of the new std::sync::OnceLock type. This change only affects the rustls-based https features. Thanks for the PR, @LyonSyonII! (#115)
• MSRV builds that got broken due to a rustix update. Now tempfile is pinned as well.

"Cleaned up" some code in 2.13.1, thinking that minreq doesn't have a build.rs, but it does, and the code could actually be enabled via an environment variable. Reverted that bit here.

Fixed

• Reverted a part of 2.13.1, accidentally removed some code that wasn't actually dead code.

Fixed

• Reverted a part of 2.13.1, accidentally removed some code that wasn't actually dead code.

Only affects builds using the https-bundled-probe feature.

This is a tiny update to the copied-over native-tls cert loading code when using openssl-probe without native-tls, to avoid an apparently unsafe function in openssl-probe.

Fixed

• Usage of an openssl-probe function that's deprecated due to safety issues. See rustsec/advisory-db#2209 for further info.

Fixed

• Usage of an openssl-probe function that's deprecated due to safety issues. See rustsec/advisory-db#2209 for further info.

Small fixups all around, mostly code size improvements. In case your application depends specifically on webpki-roots certs being used, but still uses the https-rustls-probe feature, I'm afraid you'll have to stay on 2.12. I hope this isn't the case for anyone.

Also: periodic MSRV CI did its job! It took me 5 days to notice the notification, but still a big improvement in issue-appearance-to-fix-latency.

Changed

• The https-rustls-probe feature no longer brings in the webpki-roots and rustls-webpki crates. Thanks for the report, @polarathene! (#111)

Fixed

• Cleaned up an unnecessary format!() in Connection::connect. Thanks for the PR, @melotic! (#112)
• Fixed some msrv and lint issues introduced by libc and clippy updates respectively.

Changed

• The https-rustls-probe feature no longer brings in the webpki-roots and rustls-webpki crates. Thanks for the report, @polarathene! (#111)

Fixed

• Cleaned up an unnecessary format!() in Connection::connect. Thanks for the PR, @melotic! (#112)
• Fixed some msrv and lint issues introduced by libc and clippy updates respectively.

Added

• Request::with_headers, to allow passing in many headers at a time. Thanks for the idea and PR, @rawhuul! (#110)

Fixed

• The dev dependency tiny_http's version up to 0.12. Thanks for the PR, @davide125! (#107)

Fixed

• Unnecessary buffering causing performance problems. Thanks for the PRs, @mrkline! (#102, #103)
• Connections failing if the first resolved address fails to connect (even if there's more to try). Thanks for the PR, @darosior! (#106)

Changed

• Removed upper bounds on the serde_json, log and chrono dependencies (dev-dependency in the case of chrono). If you were depending on minreq compiling with the MSRV compiler without any issues, check out the MSRV section in the readme, it's been updated with additional instructions. Thanks for the report, @RCasatta! (#99)

Fixed

• Fragment handling, once again. Turns out you're not supposed to include fragments in the request. This may break usage with servers that are written with the wrong assumptions. Thanks for the report, @rawhuul! (#100)

Added

• Response::url and ResponseLazy::url fields, to contain the final URL after redirects and fragment replacement semantics.

Changed

• Loosened the rustls version requirement from 0.21.6 to 0.21.1.

Changed

• From webpki to rustls-webpki. Thanks for the heads-up about webpki not being maintained, @RCasatta! (#98)
• Updated rustls and webpki-roots to their most recent versions.
• Maximum versions for the following dependencies to keep minreq compiling on Rust 1.48:
  • serde_json (>=1.0.0, <1.0.101)
  • log (>=0.4.0, <0.4.19)
  • chrono (dev-dependency, >=0.4.0, <0.4.24)

Fixed

• Errors when using an IP address as the host with HTTPS (tested with https://8.8.8.8). (#34)

Fixed

• Proxy strings with the protocol included not working. Thanks for the report, @tkkcc! (#95)

Added

• Default proxy from environment variables when the proxy feature is enabled, based on what curl does. Thanks for the PR, @krypt0nn! (#94)

Changed

• From lazy_static to once_cell for library internals. Thanks for the PR, @alpha-tango-kilo! (#80)

Added

• A Read impl for ResponseLazy. Thanks for the PR, @Luro02! (#81)
• Building with --all-features, with the send_https function defaulting to the rustls-based implementation. Thanks for the PR, @tcharding! (#89)
• An explicit minimum supported rust version policy. The MSRV for versions 2.x is 1.48. Thanks for the suggestion and PR, @tcharding! (#90)
• Performance improvements, test fixes, CI updates.

Changed

• The error returned when the request url does not start with https:// or http:// now is now a slightly different IoError, with a clearer message. This will be changed to a proper minreq-specific error in 3.0, but for now it's an IoError to avoid breaking the Error type.

Added

• The urlencoding feature for automatically percent-encoding urls. Thanks for the idea and PR, @alpha-tango-kilo! (#67, #68)

Fixed

• GitHub API requests without User-Agent returning an IoError. Thanks for the report, @tech-ticks! (#66)

Fixed

• Returning the wrong status code when the response was missing a status phrase. Thanks for the PR, @richarddd! (#64)
• Non-lazy requests crashing if the request had a very big Content-Length header. Thanks for the report, @Shnatsel! (#63)

Fixed

• A regression in 2.4.1 where the port is no longer included in the Host, even if it's a non-standard port. Now the port is always included if it's in the request URL, and omitted if the port is implied. Thanks for the report, @ollpu! (#61)

Fixed

• The port is no longer included in the Host header when sending requests, and port handling was cleaned up overall. This fixes issues with infinite redirections and https handshakes for some websites. Thanks to @Shnatsel for reporting the issues, and @joeried for debugging and figuring out the root cause of these problems! (#48, #49)

Added

• Request::with_param for more ergonomic query parameter usage. Thanks for the PR, @sjvignesh! (#54)
• Request::with_max_headers_size and Request::with_max_status_line_length for avoiding DoS when the server sends large headers or status lines. Thanks for the report, @Shnatsel! (#55)
• Support for the rustls-native-certs crate via a new https-rustls-probe feature. Thanks for the PR, @joeried! (#59)

Fixed

• Chunk length handling for some servers with slightly off-spec chunk lengths. Thanks for the report, @Shnatsel! (#50)
• Timeouts not always being properly enforced. Thanks for the report, @Shnatsel! (#52)

Fixed

• Removed some leftover printlns from the redirection update in 2.3.0 and ensured there's no printlns in the library anymore. Thanks for reporting the issue @Shnatsel! #45
• Fixed the timeout not being respected during the initial TCP connect. Thanks for the report and fix @KarthikNedunchezhiyan! #46, #47

Changed

• Breaking (sort of): the redirection code was improved to match RFC 7231 section 7.1.2, which could subtly break some programs relying on very specific redirects, which is why this should be investigated if you come across weird behaviour after updating. No API changes though, so only a minor version bump. The following two points are now fixed when redirecting:
  • Fragments, the bit after a #-character in the url. If the redirecting url has a fragment, and the one in Location does not, the original fragment should be included in the new url. If Location does have a fragment, it should override the one in the redirecting url.
  • Relative urls. Minreq now properly redirects when Location is relative, e.g. /Foo.html instead of https://example.com/Foo.html. Thanks, @fjt523!

Fixed

• The Content-Length: 0 header is now inserted into requests that should have it. Thanks, @KarthikNedunchezhiyan!
• Status line parsing is now fixed, so "400 Bad Request" is not parsed as "400 Bad". Thanks, @KarthikNedunchezhiyan!

Added

• M1 Mac support by bumping the ring dependency. Thanks, @ryanmcgrath!

Fixed

• Some documentation which has been long due for an update. I just always forget when writing an actual update. No code changes!

Added

• Support for native-tls and openssl-sys via new features, in addition to rustls. Thanks to @dubiousjim!

Fixed

• Handling of status codes 204 and 304. Thanks to @Mubelotix!

Added

• Proxy support via the proxy feature. Thanks to @rustysec!

Fixed

• Fixed regression in header parsing caused by 2.0.2, which was yanked.

Fixed

• Fixed a panic when sending a request to an invalid domain via https.
• Fixed a panic when parsing headers that have >1 byte unicode characters right after the ":" in the response.

Fixed

• Made timeouts work as described in the documentation. Fixed issue #22.

Added

• API for loading the HTTP response body through an iterator, allowing for processing of the data during the download.
  • See the ResponseLazy documentation for more information.
• Error type for all the errors that this crate can run into for easier ? usage and better debuggability.
• Punycode support for non-ascii hostnames via the punycode feature.
• Trailer header support.
• Examples hello, iterator, and json.

Changed

• Breaking, will cause problems not detectable by the compiler: Response headers' field names are now in lowercase, as they are case-insensitive and this makes getting header values easier. The values are unaffected. So if your code has response.headers.get("Content-Type"), you have to change it to response.headers.get("content-type"), or it will not return what you want.
• Breaking: Restructure the Response struct:
  • Removed bytes and body_bytes.
  • Added as_bytes(), into_bytes(), and as_str() in their place.
• Breaking: Changed the with_body parameter type to Into<Vec<u8>> from Into<String>.
  • Strings implement Into<Vec<u8>>, so this shouldn't cause any problems, unless you're using some interesting types that implement Into<String> but not Into<Vec<u8>>.
• Clean up the crate internals overall. Note: This might cause instability, if you're very concerned about stability, please hold off upgrading for a while.
• Remove panic! when trying to make an https:// request without the https feature. The request will now return an error instead. The library should not panic anymore.
• Audit the remaining unwrap()s from library code, none of them should actually ever cause a panic now.

Removed

• create_request in favor of just using Response::new.

Changed

• Updated dependencies.

Fixed

• Tests on Windows by changing the ip in tests from 0.0.0.0 to localhost.
• Reuse rustls::ClientConfig between requests.
• Content-Length and Transfer-Encoding detection failing because of case-sensitiveness.

Added

• json-using-serde feature.

Added

• The body_bytes field to Response, containing the body in raw bytes.

Fixed

• Some clippy warnings.
• Panic when getting a non-UTF-8 response, instead setting the body string to an empty string, for now.

Fixed

• HTTP response body handling.

Added

• Support for the HTTP status codes 301, 302, 303, and 307.

Fixed

• Less .clones()s.

Fixed

• Fix response handling when Transfer-Encoding is chunked.

Changed

• Moved to 2018 edition.

Fixed

• HEAD requests and ones that receive a 1xx, 204, or 304 status code as a response.

Changed

• Timeout made optional.
• Updated dependencies.

Fixed

• Improved performance for HTTP (not HTTPS) requests.