NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
crates.io, #2441 most downloaded on crates.io
Apache OpenDAL™: One Layer, All Storage.
Latest stable
0.59.5
Last release 1 months ago
18 Oct 2026
Release timing varies
gaps range from 1 weeks to 5 months
Nearly every release is documented
notes for 60 of the last 60 stable releases
3 versions withdrawn
withdrawn after publishing
5 years old
161 releases · first in 2022
Drag across the ruler to compare, or tap a release to open it.
The following APIs have been deprecated and will be removed in the future releases:
OpenDAL has refined its vision to One Layer, All Storage, driven by the following core principles: Open Community, Solid Foundation, Fast Access, Object Storage First, and Extensible Architecture.
Explore the detailed vision at OpenDAL Vision.
OpenDAL v0.51 implements RFC-5313, which removes the concept of metakey.
The following structs have been removed:
MetakeyThe following APIs have been removed:
list_with(path).metakey()Users no longer need to pass the metakey into the list. Instead, services will make their best effort to return as much metadata as possible. Users can check items like Capability::list_has_etag before making a call.
write_multi_align_sizeThe capability write_multi_align_size is not utilized by any services, and we have no plans to support it in the future; therefore, we have removed it.
OpenDAL used to perform capability checks for all services, but since v0.51, it only conducts checks that impact data correctness like write_with_if_not_exists or delete_with_version by default in the CorrectnessCheckLayer. If users wish to verify other non-critical capabilities like write_with_content_type or write_with_cache_control, they should manually enable the CapabilityCheckLayer.
OpenDAL v0.51 implements RFC-3911, which adds Deleter in OpenDAL to replace batch operation.
The following new APIs have been added:
Operator::delete_iter]Operator::delete_try_iter]Operator::delete_stream]Operator::delete_try_stream]Operator::deleter]Deleter::delete]Deleter::delete_iter]Deleter::delete_try_iter]Deleter::delete_stream]Deleter::delete_try_stream]Deleter::flush]Deleter::close]Deleter::into_sink]DeleteInput]IntoDeleteInput]FuturesDeleteSink]The following APIs have been deprecated and will be removed in the future releases:
Operator::remove (replace with [Operator::delete_iter])Operator::remove_via (replace with [Operator::delete_stream])As a result of this change, the limit and with_limit APIs on Operator have also been deprecated; they are currently no-ops.
adapter::kv now returns Scanner instead of Vec<String>To support returning key-value entries in a streaming manner instead of loading them all into memory, OpenDAL updated its adapter API to return a Scanner instead of a Vec<String>.
- async fn scan(&self, path: &str) -> Result<Vec<String>>
+ async fn scan(&self, path: &str) -> Result<Self::Scanner>
All services intending to implement kv::Adapter should adhere to this API change.
metadata API to infoOpenDAL changes it's old metadata API to info to align with the new AccessorInfo struct.
- fn metadata(&self) -> Arc<AccessorInfo>
+ fn info(&self) -> Arc<AccessorInfo>
RangeWriterThe struct RangeWriter is not utilized by any services, and we have no plans to support it in the future; therefore, we have removed it.
<!-- Release notes generated using configuration in .github/release.yml at v0.51.0 -->
if_not_exist in OpWrite by @kemingy in https://github.com/apache/opendal/pull/5305to_bytes when NonContiguous contains a single Bytes by @ever0de in https://github.com/apache/opendal/pull/5388OpWrite by @Frank-III in https://github.com/apache/opendal/pull/5360content_encoding to MetaData by @Frank-III in https://github.com/apache/opendal/pull/5400content encoding to Opwrite by @Frank-III in https://github.com/apache/opendal/pull/5390/ when not presented by @Frank-III in https://github.com/apache/opendal/pull/5382stat instead of Entry::metadata in fetch_placeholders by @ho-229 in https://github.com/apache/opendal/pull/5416gh-pages by @Xuanwo in https://github.com/apache/opendal/pull/5290info method of trait Access and trait LayeredAccess by @koushiro in https://github.com/apache/opendal/pull/5258info method of kv::Adapter and typed_kv::Adapter by @koushiro in https://github.com/apache/opendal/pull/5259Full Changelog: https://github.com/apache/opendal/compare/v0.50.2...v0.51.0
Your coding agent can read these notes before it upgrades. Set up the MCP server →
chore(core/blocking_operator): deduplicate deprecated is_exist logic by @simonsan in https://github.com/apache/opendal/pull/5261
<!-- Release notes generated using configuration in .github/release.yml at v0.50.2 -->
| Name | Version | Next |
|---|---|---|
| core | 0.50.1 | 0.50.2 |
| integrations/cloud_filter | 0.0.2 | 0.0.3 |
| integrations/compat | 1.0.0 | 1.0.1 |
| integrations/dav-server | 0.2.1 | 0.2.2 |
| integrations/fuse3 | 0.0.8 | 0.0.9 |
| integrations/object_store | 0.48.1 | 0.48.2 |
| integrations/parquet | 0.2.1 | 0.2.2 |
| integrations/spring | - | - |
| integrations/unftp-sbe | 0.0.8 | 0.0.9 |
| integrations/virtiofs | - | - |
| bin/oay | 0.41.12 | 0.41.13 |
| bin/ofs | 0.0.13 | 0.0.14 |
| bin/oli | 0.41.12 | 0.41.13 |
| bindings/c | 0.45.0 | 0.45.1 |
| bindings/cpp | 0.45.12 | 0.45.13 |
| bindings/dotnet | 0.1.10 | 0.1.11 |
| bindings/go | 0.1.4 | 0.1.5 |
| bindings/haskell | 0.44.12 | 0.44.13 |
| bindings/java | 0.47.4 | 0.47.5 |
| bindings/lua | 0.1.10 | 0.1.11 |
| bindings/nodejs | 0.47.6 | 0.47.7 |
| bindings/ocaml | - | - |
| bindings/php | 0.1.10 | 0.1.11 |
| bindings/python | 0.45.11 | 0.45.12 |
| bindings/ruby | 0.1.10 | 0.1.11 |
| bindings/swift | - | - |
| bindings/zig | - | - |
File.readline by @TennyZhuang in https://github.com/apache/opendal/pull/5271clap_derive to reduce boilerplate code by @koushiro in https://github.com/apache/opendal/pull/5233[DISCUSS] prefix for discussion by @Xuanwo in https://github.com/apache/opendal/pull/5210is_exist logic by @simonsan in https://github.com/apache/opendal/pull/5261Full Changelog: https://github.com/apache/opendal/compare/v0.50.1...v0.50.2
feat(bindings/c): add opendal_operator_exists and mark is_exist deprecated by @PragmaTwice in https://github.com/apache/opendal/pull/5199
Change to use Cmake to build the C binding.
| Name | Version | Next |
|---|---|---|
| core | 0.50.0 | 0.50.1 |
| integrations/cloud_filter | 0.0.1 | 0.0.2 |
| integrations/compat | - | 1.0.0 |
| integrations/dav-server | 0.2.0 | 0.2.1 |
| integrations/fuse3 | 0.0.7 | 0.0.8 |
| integrations/object_store | 0.48.0 | 0.48.1 |
| integrations/parquet | 0.2.0 | 0.2.1 |
| integrations/spring | - | - |
| integrations/unftp-sbe | 0.0.7 | 0.0.8 |
| integrations/virtiofs | - | - |
| bin/oay | 0.41.11 | 0.41.12 |
| bin/ofs | 0.0.12 | 0.0.13 |
| bin/oli | 0.41.11 | 0.41.12 |
| bindings/c | 0.44.13 | 0.45.0 |
| bindings/cpp | 0.45.11 | 0.45.12 |
| bindings/dotnet | 0.1.9 | 0.1.10 |
| bindings/go | 0.1.3 | 0.1.4 |
| bindings/haskell | 0.44.11 | 0.44.12 |
| bindings/java | 0.47.3 | 0.47.4 |
| bindings/lua | 0.1.9 | 0.1.10 |
| bindings/nodejs | 0.47.5 | 0.47.6 |
| bindings/ocaml | - | - |
| bindings/php | 0.1.9 | 0.1.10 |
| bindings/python | 0.45.10 | 0.45.11 |
| bindings/ruby | 0.1.9 | 0.1.10 |
| bindings/swift | - | - |
| bindings/zig | - | - |
ManuallyDrop instead of forget to make sure pointer is valid by @ethe in https://github.com/apache/opendal/pull/5166Full Changelog: https://github.com/apache/opendal/compare/v0.50.0...v0.50.1
| integrations/dav-server | 0.1.0 | 0.1.1 |
| Name | Version | Next |
|---|---|---|
| core | 0.49.2 | 0.50.0 |
| integrations/dav-server | 0.1.0 | 0.1.1 |
| integrations/fuse3 | 0.0.6 | 0.0.7 |
| integrations/object_store | 0.47.0 | 0.47.1 |
| integrations/parquet | 0.1.2 | 0.2.0 |
| integrations/unftp-sbe | 0.0.6 | 0.0.7 |
| bin/oay | 0.41.10 | 0.41.11 |
| bin/ofs | 0.0.11 | 0.0.12 |
| bin/oli | 0.41.10 | 0.41.11 |
| bindings/c | 0.44.12 | 0.44.13 |
| bindings/cpp | 0.45.10 | 0.45.11 |
| bindings/dotnet | 0.1.8 | 0.1.9 |
| bindings/go | 0.1.2 | 0.1.3 |
| bindings/haskell | 0.44.10 | 0.44.11 |
| bindings/java | 0.47.2 | 0.47.3 |
| bindings/lua | 0.1.8 | 0.1.9 |
| bindings/nodejs | 0.47.4 | 0.47.5 |
| bindings/php | 0.1.8 | 0.1.9 |
| bindings/python | 0.45.9 | 0.45.10 |
| bindings/ruby | 0.1.8 | 0.1.9 |
services-postgresql's connect string now supports only URL formatPreviously, it supports both URL format and key-value format. After switching the implementation from tokio-postgres to sqlx, the service now supports only the URL format.
list now returns path itselfPreviously, list("a/b") would not return a/b even if it does exist. Since v0.50.0, this behavior has been changed. OpenDAL will now return the path itself if it exists. This change applies to all cases, whether the path is a directory or a file.
In OpenDAL v0.50.0, we did a refactor on all metrics-related layers. They are now sharing the same underlying implemenationts. PrometheusLayer, PrometheusClientLayer and MetricsLayer are now have similar public APIs and exactly the same metrics value.
<!-- Release notes generated using configuration in .github/release.yml at v0.50.0 -->
num_cpus with std::thread::available_parallelism by @miroim in https://github.com/apache/opendal/pull/5080PrometheusClientLayerBuilder type by @koushiro in https://github.com/apache/opendal/pull/5093Full Changelog: https://github.com/apache/opendal/compare/v0.49.2...v0.50.0
| integrations/dav-server | 0.1.0 |
| Name | Version |
|---|---|
| core | 0.49.2 |
| integrations/dav-server | 0.1.0 |
| integrations/fuse3 | 0.0.6 |
| integrations/object_store | 0.47.0 |
| integrations/parquet | 0.1.2 |
| integrations/unftp-sbe | 0.0.6 |
| bin/oay | 0.41.10 |
| bin/ofs | 0.0.11 |
| bin/oli | 0.41.10 |
| bindings/c | 0.44.12 |
| bindings/cpp | 0.45.10 |
| bindings/dotnet | 0.1.8 |
| bindings/go | 0.1.2 |
| bindings/haskell | 0.44.10 |
| bindings/java | 0.47.2 |
| bindings/lua | 0.1.8 |
| bindings/nodejs | 0.47.4 |
| bindings/php | 0.1.8 |
| bindings/python | 0.45.9 |
| bindings/ruby | 0.1.8 |
<!-- Release notes generated using configuration in .github/release.yml at v0.49.2 -->
integrations/cloudfilter for ofs by @ho-229 in https://github.com/apache/opendal/pull/4935fn root semantics; fix missing root for some services; rm duplicated normalize ops by @yjhmelody in https://github.com/apache/opendal/pull/5035scheme field from PrometheusAccessor and PrometheusMetricWrapper type by @koushiro in https://github.com/apache/opendal/pull/5026starts_with by @haoqixu in https://github.com/apache/opendal/pull/5036increment_request_total of write operation by @koushiro in https://github.com/apache/opendal/pull/5023token method semantics by @yjhmelody in https://github.com/apache/opendal/pull/5045Full Changelog: https://github.com/apache/opendal/compare/v0.49.1...v0.49.2
| integrations/dav-server | 0.0.8 |
| Name | Next |
|---|---|
| core | 0.49.1 |
| integrations/dav-server | 0.0.8 |
| integrations/fuse3 | 0.0.5 |
| integrations/object_store | 0.46.1 |
| integrations/parquet | 0.1.1 |
| integrations/unftp-sbe | 0.0.5 |
| bin/oay | 0.41.9 |
| bin/ofs | 0.0.10 |
| bin/oli | 0.41.9 |
| bindings/c | 0.44.11 |
| bindings/cpp | 0.45.9 |
| bindings/dotnet | 0.1.7 |
| bindings/go | 0.1.1 |
| bindings/haskell | 0.44.9 |
| bindings/java | 0.47.1 |
| bindings/lua | 0.1.7 |
| bindings/nodejs | 0.47.3 |
| bindings/php | 0.1.7 |
| bindings/python | 0.45.8 |
| bindings/ruby | 0.1.7 |
<!-- Release notes generated using configuration in .github/release.yml at v0.49.1 -->
Full Changelog: https://github.com/apache/opendal/compare/v0.49.0...v0.49.1
| integrations/dav-server | 0.0.7 |
| Name | Next |
|---|---|
| core | 0.49.0 |
| integrations/dav-server | 0.0.7 |
| integrations/fuse3 | 0.0.4 |
| integrations/object_store | 0.46.0 |
| integrations/parquet | 0.1.0 |
| integrations/unftp-sbe | 0.0.4 |
| bin/oay | 0.41.8 |
| bin/ofs | 0.0.9 |
| bin/oli | 0.41.8 |
| bindings/c | 0.44.10 |
| bindings/cpp | 0.45.8 |
| bindings/dotnet | 0.1.6 |
| bindings/go | 0.1.0 |
| bindings/haskell | 0.44.8 |
| bindings/java | 0.47.0 |
| bindings/lua | 0.1.6 |
| bindings/nodejs | 0.47.2 |
| bindings/php | 0.1.6 |
| bindings/python (*) | 0.45.8 |
| bindings/ruby | 0.1.6 |
0.45.8, but due to https://github.com/apache/opendal/issues/5000, we were unable to. We will release it next time.Configurator now returns associated builder insteadConfigurator used to return impl Builder, but now it returns associated builder type directly. This will allow users to use the builder in a more flexible way.
impl Configurator for MemoryConfig {
- fn into_builder(self) -> impl Builder {
+ type Builder = MemoryBuilder;
+ fn into_builder(self) -> Self::Builder {
MemoryBuilder { config: self }
}
}
LoggingLayer now accepts LoggingInterceptorLoggingLayer now accepts LoggingInterceptor trait instead of configuration. This change will allow users to customize the logging behavior more flexibly.
pub trait LoggingInterceptor: Debug + Clone + Send + Sync + Unpin + 'static {
fn log(
&self,
info: &AccessorInfo,
operation: Operation,
context: &[(&str, &str)],
message: &str,
err: Option<&Error>,
);
}
Users can now implement the log in the way they want.
artifactId of the opendal-java has changed from to opendal to align with the convention of entire OpenDAL project.
<dependencies>
<dependency>
<groupId>org.apache.opendal</groupId>
- <artifactId>opendal-java</artifactId>
+ <artifactId>opendal</artifactId>
<version>${opendal.version}</version>
</dependency>
<dependency>
<groupId>org.apache.opendal</groupId>
- <artifactId>opendal-java</artifactId>
+ <artifactId>opendal</artifactId>
<version>${opendal.version}</version>
<classifier>${os.detected.classifier}</classifier>
</dependency>
</dependencies>
<!-- Release notes generated using configuration in .github/release.yml at v0.49.0 -->
put_multipart in object_store by @Rachelint in https://github.com/apache/opendal/pull/4793AsyncWriter for parquet integrations by @WenyXu in https://github.com/apache/opendal/pull/4958AsyncReader for parquet integrations by @WenyXu in https://github.com/apache/opendal/pull/4972allow_anonymous by @jdockerty in https://github.com/apache/opendal/pull/4979Full Changelog: https://github.com/apache/opendal/compare/v0.48.0...v0.49.0
Since v0.48, Operator's new APIs from_iter and via_iter methods have deprecated the from_map and via_map methods.
| Name | Version |
|---|---|
| core | 0.48.0 |
| integrations/cloudfilter | 0.0.0 |
| integrations/dav-server | 0.0.6 |
| integrations/fuse3 | 0.0.3 |
| integrations/object_store | 0.45.0 |
| integrations/unftp-sbe | 0.0.3 |
| bin/oay | 0.41.7 |
| bin/ofs | 0.0.8 |
| bin/oli | 0.41.7 |
| bindings/c | 0.44.9 |
| bindings/cpp | 0.45.7 |
| bindings/dotnet | 0.1.5 |
| bindings/haskell | 0.44.7 |
| bindings/java | 0.46.4 |
| bindings/lua | 0.1.5 |
| bindings/nodejs | 0.47.1 |
| bindings/php | 0.1.5 |
| bindings/python | 0.45.7 |
| bindings/ruby | 0.1.5 |
customized_credential_loadSince v0.48, the customed_credential_load function has been renamed to customized_credential_load to fix the typo of customized.
- builder.customed_credential_load(v);
+ builder.customized_credential_load(v);
security_token to session_tokenIn 2014 Amazon switched from AWS_SECURITY_TOKEN to AWS_SESSION_TOKEN. To be consistent with the naming of AWS STS, we have renamed the security_token field to session_token in the S3 service.
- builder.security_token(v);
+ builder.session_token(v);
from_iter and via_iter replaces from_map and via_mapSince v0.48, Operator's new APIs from_iter and via_iter methods have deprecated the from_map and via_map methods.
- Operator::from_map::<Fs>(map)?.finish();
+ Operator::from_iter::<Fs>(map)?.finish();
New API from_iter and via_iter should cover all use cases of from_map and via_map.
Since v0.48, all service builder now takes ownership self instead of &mut self. This change will allow users to configure the service in a more flexible way.
- let mut builder = S3::default();
- builder.bucket("test");
- builder.root("/path/to/root");
+ let builder = S3::default().bucket("test").root("/path/to/root");
let op = Operator::new(builder)?.finish();
oio::Write::write will write the whole bufferStarting from version 0.48, oio::Write::write now writes the entire buffer. This update aligns the API more closely with oio::Read::read and simplifies the implementation of concurrent writing.
trait Write {
- fn write(&mut self, bs: Buffer) -> impl Future<Output = Result<usize>>;
+ fn write(&mut self, bs: Buffer) -> impl Future<Output = Result<()>>;
}
write will now return Result<()> instead of Result<usize>. The number of bytes written can be obtained from the buffer's length.
Access::metadata() will return Arc<AccessInfo>Starting from version 0.48, Access::metadata() will return Arc<AccessInfo> instead of AccessInfo. This change is intended to improve performance and reduce memory usage.
trait Access {
- fn metadata(&self) -> AccessInfo;
+ fn metadata(&self) -> Arc<AccessInfo>;
}
MinitraceLayer renamed to FastraceLayerThe MinitraceLayer has been renamed to FastraceLayer to respond to the transition from minitrace to fastrace.
- use opendal::layers::MinitraceLayer;
+ use opendal::layers::FastraceLayer;
Configurator to replace Builder::from_configSince v0.48, the Builder::from_config and Builder::from_map method has been replaced by the Configurator trait. The Configurator trait provides a more flexible and extensible way to configure OpenDAL.
Service implementers should update their code to use the Configurator trait instead:
impl Configurator for MemoryConfig {
fn into_builder(self) -> impl Builder {
MemoryBuilder { config: self }
}
}
impl Builder for MemoryBuilder {
const SCHEME: Scheme = Scheme::Memory;
type Config = MemoryConfig;
fn build(self) -> Result<impl Access> {
...
}
}
<!-- Release notes generated using configuration in .github/release.yml at v0.48.0 -->
Buffer::to_bytes() by cloning contiguous bytes by @LDeakin in https://github.com/apache/opendal/pull/4858root on using PrometheusClientLayer by @flaneur2020 in https://github.com/apache/opendal/pull/4907Content-Type based on the extension in the path. by @czy-29 in https://github.com/apache/opendal/pull/4912Arc<AccessInfo> for metadata by @Lzzzzzt in https://github.com/apache/opendal/pull/4883MimeGuessLayer. by @czy-29 in https://github.com/apache/opendal/pull/4916Full Changelog: https://github.com/apache/opendal/compare/v0.47.3...v0.48.0
| integrations/dav-server | 0.0.4 | 0.0.5 |
| Name | Version | Next |
|---|---|---|
| core | 0.47.2 | 0.47.3 |
| integrations/dav-server | 0.0.4 | 0.0.5 |
| integrations/fuse3 | 0.0.1 | 0.0.2 |
| integrations/object_store | 0.44.2 | 0.44.3 |
| integrations/unftp-sbe | 0.0.1 | 0.0.2 |
| bin/oay | 0.41.5 | 0.41.6 |
| bin/ofs | 0.0.6 | 0.0.7 |
| bin/oli | 0.41.5 | 0.41.6 |
| bindings/c | 0.44.7 | 0.44.8 |
| bindings/cpp | 0.45.5 | 0.45.6 |
| bindings/dotnet | 0.1.3 | 0.1.4 |
| bindings/haskell | 0.44.5 | 0.44.6 |
| bindings/java | 0.46.2 | 0.46.3 |
| bindings/lua | 0.1.3 | 0.1.4 |
| bindings/nodejs | 0.47.0 | 0.47.0 |
| bindings/php | 0.1.3 | 0.1.4 |
| bindings/python | 0.45.5 | 0.45.6 |
| bindings/ruby | 0.1.3 | 0.1.4 |
<!-- Release notes generated using configuration in .github/release.yml at v0.47.3 -->
oio::Write. by @yuchanns in https://github.com/apache/opendal/pull/4821Full Changelog: https://github.com/apache/opendal/compare/v0.47.2...v0.47.3
| integrations/cloudfilter | 0.0.0 | 0.0.0 |
| Name | Version | Next |
|---|---|---|
| core | 0.47.1 | 0.47.2 |
| integrations/cloudfilter | 0.0.0 | 0.0.0 |
| integrations/dav-server | 0.0.3 | 0.0.4 |
| integrations/fuse3 | 0.0.0 | 0.0.1 |
| integrations/object_store | 0.44.1 | 0.44.2 |
| integrations/unftp-sbe | 0.0.0 | 0.0.1 |
| integrations/virtiofs | 0.0.0 | 0.0.0 |
| bin/oay | 0.41.4 | 0.41.5 |
| bin/ofs | 0.0.5 | 0.0.6 |
| bin/oli | 0.41.4 | 0.41.5 |
| bindings/c | 0.44.6 | 0.44.7 |
| bindings/cpp | 0.45.4 | 0.45.5 |
| bindings/dotnet | 0.1.2 | 0.1.3 |
| bindings/go | 0.0.0 | 0.0.0 |
| bindings/haskell | 0.44.4 | 0.44.5 |
| bindings/java | 0.46.1 | 0.46.2 |
| bindings/lua | 0.1.2 | 0.1.3 |
| bindings/nodejs | 0.46.2 | 0.47.0 |
| bindings/ocaml | 0.0.0 | 0.0.0 |
| bindings/php | 0.1.2 | 0.1.3 |
| bindings/python | 0.45.4 | 0.45.5 |
| bindings/ruby | 0.1.2 | 0.1.3 |
| bindings/swift | 0.0.0 | 0.0.0 |
| bindings/zig | 0.0.0 | 0.0.0 |
Now, the append operation has been removed. You can use below code instead.
op.write("path/to/file", Buffer.from("hello world"), { append: true });
Access impl by @George-Miao in https://github.com/apache/opendal/pull/4693OpendalStorage by @George-Miao in https://github.com/apache/opendal/pull/4765WriteOptions for write methods by @bxb100 in https://github.com/apache/opendal/pull/4785OpendalMetadata by @George-Miao in https://github.com/apache/opendal/pull/4819MadsimLayer by @zzzk1 in https://github.com/apache/opendal/pull/4788Full Changelog: https://github.com/apache/opendal/compare/v0.47.1...v0.47.2
<!-- Release notes generated using configuration in .github/release.yml at v0.47.1 -->
<!-- Release notes generated using configuration in .github/release.yml at v0.47.1 -->
size for RangeWrite. by @reswqa in https://github.com/apache/opendal/pull/4755/ by @yuchanns in https://github.com/apache/opendal/pull/4754Full Changelog: https://github.com/apache/opendal/compare/v0.47.0...v0.47.1
Since v0.47, Reader's into_xxx APIs requires async and returns Result instead.
into_xxx APIsSince v0.47, Reader's into_xxx APIs requires async and returns Result instead.
- let r = op.reader("test.txt").await?.into_futures_async_read(1024..2048);
+ let r = op.reader("test.txt").await?.into_futures_async_read(1024..2048).await?;
Affected API includes:
Reader::into_futures_async_readReader::into_bytes_streamBlockingReader::into_std_readBlockingReader::into_bytes_iteratorAs explained in core: Bring Streaming Read Back, we do need read streaming back for better performance and low memory usage.
So our oio::Read changed back to streaming read instead:
trait Read {
- async fn read(&self, offset: u64, size: usize) -> Result<Buffer>;
+ async fn read(&mut self) -> Result<Buffer>;
}
All services and layers should be updated to meet this change.
PR-4641 renames async Operator to AsyncOperator and BlockingOperator to Operator.
PR-4626 implements OperatorInputStream and OperatorOutputStream which implements Java's core IO abstractions InputStream and OutputStream. Users can now read/write bytes streamlined without loading/preparing the bytes fully in memory.
<!-- Release notes generated using configuration in .github/release.yml at v0.47.0 -->
AccessorDyn by @morristai in https://github.com/apache/opendal/pull/4574Full Changelog: https://github.com/apache/opendal/compare/v0.46.0...v0.47.0
ci: Address Node.js 16 actions are deprecated by @Xuanwo in https://github.com/apache/opendal/pull/4293
Since 0.46, OpenDAL requires Rust 1.75.0 or later to use features like RPITIT and AFIT.
Starting with version 0.46, OpenDAL only includes the memory service by default to prevent compiling unnecessary service code. To use other services, please activate their respective feature flags.
Additionally, we have removed all reqwest-related feature flags:
reqwest's feature flags for options like rustls, native-tls, etc.rustls feature is no longer enabled by default; it must be activated manually.trust-dns option; users should configure the client builder directly.Since v0.46, OpenDAL transformed it's Read IO trait to range based instead of stateful poll based IO. This change will make the IO more efficient, easier for concurrency and ready for completion based IO.
opendal::Reader now have APIs like:
let r = op.reader("test.txt").await?;
let buf = r.read(1024..2048).await?;
Since version 0.46, OpenDAL features a native Buffer struct that supports both contiguous and non-contiguous buffers. This update enhances IO efficiency by minimizing unnecessary byte copying and enabling vectored IO.
OpenDAL's Reader will return Buffer and Writer will accept Buffer as input. Users who have implemented their own IO traits should update their code to use the new Buffer struct.
let r = op.reader("test.txt").await?;
// read returns `Buffer`
let buf: Buffer = r.read(1024..2048).await?;
let w = op.writer("test.txt").await?;
// Buffer can be created from continues bytes.
w.write("hello, world").await?;
// Buffer can also be created from non-continues bytes.
w.write(vec![Bytes::from("hello,"), Bytes::from("world!")]).await?;
// Make sure file has been written completely.
w.close().await?;
To enhance usability, we've integrated bridges into bytes::Buf and bytes::BufMut, allowing users to directly interact with the bytes API.
let r = op.reader("test.txt").await?;
let mut bs = vec![];
// read_into accepts bytes::BufMut
let buf: Buffer = r.read_into(&mut bs, 1024..2048).await?;
let w = op.writer("test.txt").await?;
// write_from accepts bytes::Buf
w.write_from("hello, world".as_bytes()).await?;
// Make sure file has been written completely.
w.close().await?;
OpenDAL's Reader and Writer previously implemented APIs such as AsyncRead and AsyncWrite directly. This design was not user-friendly, as it could lead to unexpected costs that users were unaware of in advance.
Since v0.46, OpenDAL provides bridge APIs for Reader and Writer instead.
let r = op.reader("test.txt").await?;
// Convert into futures AsyncRead + AsyncSeek.
let reader = r.into_futures_async_read(1024..2048);
// Convert into futures bytes stream.
let stream = r.into_bytes_stream(1024..2048);
let w = op.writer("test.txt").await?;
// Convert into futures AsyncWrite
let writer = w.into_futures_async_write();
// Convert into futures bytes sink;
let sink = w.into_bytes_sink();
Since version 0.46, OpenDAL has adopted Rust's native async_in_trait for our core IO traits, including oio::Read, oio::Write, and oio::List.
This update eliminates the need for manually written, poll-based state machines and simplifies the codebase. Consequently, OpenDAL now requires Rust version 1.75.0 or later.
Users who have implemented their own IO traits should update their code to use the new async trait syntax.
<!-- Release notes generated using configuration in .github/release.yml at v0.46.0 -->
Arc::from_iter by @Xuanwo in https://github.com/apache/opendal/pull/4440HttpClient::with() constructor by @waynexia in https://github.com/apache/opendal/pull/4447Buffer as cache in RangeWrite by @reswqa in https://github.com/apache/opendal/pull/4476Buffer as cache in OneshotWrite by @reswqa in https://github.com/apache/opendal/pull/4477Buffer by @George-Miao in https://github.com/apache/opendal/pull/4530IoBuf for Buffer by @George-Miao in https://github.com/apache/opendal/pull/4532oio::Read to async in trait by @Xuanwo in https://github.com/apache/opendal/pull/4336oio::List to async fn in trait by @Xuanwo in https://github.com/apache/opendal/pull/4352oio::Write to async in trait by @Xuanwo in https://github.com/apache/opendal/pull/4358File::read to PyResult<&PyBytes> by @reswqa in https://github.com/apache/opendal/pull/4360oio::WriteBuf and oio::ChunkedBytes after refactor by @Xuanwo in https://github.com/apache/opendal/pull/4361ofs API public by @ho-229 in https://github.com/apache/opendal/pull/4387Accessor to async fn in trait by @George-Miao in https://github.com/apache/opendal/pull/4562Accessor by @George-Miao in https://github.com/apache/opendal/pull/4564size argument for python File::read by @reswqa in https://github.com/apache/opendal/pull/4359File::write should return the written bytes by @reswqa in https://github.com/apache/opendal/pull/4367WebdavFile continuous reading and writing by @G-XD in https://github.com/apache/opendal/pull/4374service-* from default features by @xxchan in https://github.com/apache/opendal/pull/4311yaml file type by @jbampton in https://github.com/apache/opendal/pull/4339rdf file type as indent_size = 2 by @jbampton in https://github.com/apache/opendal/pull/4341indent_style = tab for Go by @jbampton in https://github.com/apache/opendal/pull/4342Full Changelog: https://github.com/apache/opendal/compare/v0.45.1...v0.46.0
<!-- Release notes generated using configuration in .github/release.yml at v0.45.1 -->
<!-- Release notes generated using configuration in .github/release.yml at v0.45.1 -->
| Name | Version |
|---|---|
| core | 0.45.1 |
| bin/oay | 0.41.1 |
| bin/oli | 0.41.1 |
| bin/ofs | 0.0.2 |
| bindings/python | 0.45.1 |
| bindings/nodejs | 0.45.1 |
| bindings/c | 0.44.3 |
| bindings/zig | 0.0.0 |
| bindings/dotnet | 0.1.1 |
| bindings/haskell | 0.44.3 |
| bindings/java | 0.45.1 |
| bindings/lua | 0.1.1 |
| bindings/ruby | 0.1.1 |
| bindings/swift | 0.0.0 |
| bindings/ocaml | 0.0.0 |
| bindings/php | 0.1.1 |
| bindings/cpp | 0.44.3 |
| bindings/go | 0.0.0 |
| integrations/object_store | 0.43.0 |
| integrations/dav-server | 0.0.1 |
= by @Xuanwo in https://github.com/apache/opendal/pull/4105core/src/services/http/docs.md by @jbampton in https://github.com/apache/opendal/pull/4226website/README.md by @jbampton in https://github.com/apache/opendal/pull/4228github to GitHub by @jbampton in https://github.com/apache/opendal/pull/4232bindings/c/CONTRIBUTING.md by @jbampton in https://github.com/apache/opendal/pull/4235CONTRIBUTING by @jbampton in https://github.com/apache/opendal/pull/4237bindings/README.md by @jbampton in https://github.com/apache/opendal/pull/4238examples/rust by @jbampton in https://github.com/apache/opendal/pull/4241Who is using OpenDAL list by @jbampton in https://github.com/apache/opendal/pull/4263Full Changelog: https://github.com/apache/opendal/compare/v0.45.0...v0.45.1
The with_speed API has been deprecated. Please use with_io_timeout instead.
To further enhance the optionality of tokio, we have introduced a new feature called layers-blocking. The default usage of the blocking layer has been disabled. To utilize the BlockingLayer, please enable the layers-blocking feature.
with_speedThe with_speed API has been deprecated. Please use with_io_timeout instead.
No raw API changes.
<!-- Release notes generated using configuration in .github/release.yml at v0.45.0 -->
IntoFuture for operator futures to remove an alloc by @Xuanwo in https://github.com/apache/opendal/pull/4098Full Changelog: https://github.com/apache/opendal/compare/v0.44.2...v0.45.0
<!-- Release notes generated using configuration in .github/release.yml at v0.44.2 -->
<!-- Release notes generated using configuration in .github/release.yml at v0.44.2 -->
MultipartUploadWriter by @WenyXu in https://github.com/apache/opendal/pull/3915RangeWriter by @WenyXu in https://github.com/apache/opendal/pull/3923concurrent and buffer parameters into FuzzInput by @WenyXu in https://github.com/apache/opendal/pull/3921next_offset by @WenyXu in https://github.com/apache/opendal/pull/3927Full Changelog: https://github.com/apache/opendal/compare/v0.44.1...v0.44.2
feat(service/memcached): Add MemCachedConfig by @ankit-pn in https://github.com/apache/incubator-opendal/pull/3827
CompleteReader by @WenyXu in https://github.com/apache/incubator-opendal/pull/3861bindings_python.yml by @messense in https://github.com/apache/incubator-opendal/pull/3837BufferReader by @WenyXu in https://github.com/apache/incubator-opendal/pull/3864cur of BufferReader by @WenyXu in https://github.com/apache/incubator-opendal/pull/3865Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.44.0...v0.44.1
<!-- Release notes generated using configuration in .github/release.yml at v0.44.0 -->
<!-- Release notes generated using configuration in .github/release.yml at v0.44.0 -->
thread_pool_enabled option has been removed.After RFC: List Prefix landed, we have changed the behavior of list a path without /. OpenDAL used to return NotADirectory error, but now we will return the list of entries that start with given prefix instead.
Now, the list operation returns Array<Entry> instead of a lister.
Also, we removed scan, you can use list('some/path', {recursive: true})/listSync('some/path', {recursive: true}) instead of scan('some/path')/scanSync('some/path').
actual & expected to pass ci by @Young-Flash in https://github.com/apache/incubator-opendal/pull/3733get_file_id_by_path by @G-XD in https://github.com/apache/incubator-opendal/pull/3801Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.43.0...v0.44.0
After RFC-3526: List Recursive landed, we have changed the list API to accept recursive instead of delimiter:
After RFC-3526: List Recursive landed, we have changed the list API to accept recursive instead of delimiter:
Users will need to change the following usage:
op.list_with(path).delimiter("") -> op.list_with(path).recursive(true)op.list_with(path).delimiter("/") -> op.list_with(path).recursive(false)delimiter other than "" and "/" is not supported anymore.
After RFC: List Prefix landed, we have changed the behavior of stat a dir path:
Here are the behavior list:
| Case | Path | Result |
|---|---|---|
| stat existing dir | abc/ |
Metadata with dir mode |
| stat existing file | abc/def_file |
Metadata with file mode |
stat dir without / |
abc/def_dir |
Error NotFound or metadata with dir mode |
stat file with / |
abc/def_file/ |
Error NotFound |
| stat not existing path | xyz |
Error NotFound |
Services like s3, azblob can handle stat("abc/") correctly by check if there are objects with prefix abc/.
We changed our internal lister implementation to align with the list public API for better performance and readability.
Page => ListPager => ListerBlockingPage => BlockingListBlockingPager => BlockingListerEvery call to next will return an entry instead a page of entries. Also, we changed our async list api into poll based instead of async_trait.
Because of a TLS lib issue, we temporarily disable the services-ftp feature.
Because of a TLS lib issue, we temporarily disable the services-ftp feature.
Because of a TLS lib issue, we temporarily disable the services-ftp feature.
There are no API changes.
<!-- Release notes generated using configuration in .github/release.yml at v0.43.0 -->
ftp for python and java binding by @ZutJoe in https://github.com/apache/incubator-opendal/pull/3659Page to List by @Xuanwo in https://github.com/apache/incubator-opendal/pull/3589raw::tests been excluded unexpectedly by @Xuanwo in https://github.com/apache/incubator-opendal/pull/3623stat and stat_with by @Xuanwo in https://github.com/apache/incubator-opendal/pull/3657Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.42.0...v0.43.0
The enable_exact_buf_write option has been deprecated and is superseded by BufferedWriter, introduced in version 0.40.
OpenDAL bumps it's MSRV to 1.67.0.
enable_exact_buf_write option has been deprecated and is superseded by BufferedWriter, introduced in version 0.40.write_min_size option has been deprecated and replaced by BufferedWriter, also introduced in version 0.40.allow_anonymous, has been added. Since v0.41, OSS will now return an error if credential loading fails. Enabling allow_anonymous to fallback to request without credentials.enable_create_simulation option has been removed. We add this option to allow ghac simulate create empty file, but it's could result in unexpected behavior when users create a file with content length 1. So we remove it.wasabi service native support has been removed. Users who want to access wasabi can use our s3 service instead.
There are no API changes.
There are no API changes.
Operator and BlockingOperator won't accept layers anymore. Instead, we provide a layer API:
op = opendal.Operator("memory").layer(opendal.layers.RetryLayer())
We removed not used layers ConcurrentLimitLayer and ImmutableIndexLayer along with this change.
OpenDAL removes Reader and AsyncReader classes, instead, we provide file-like object File and AsyncFile.
Open a file for reading in blocking way:
with op.open(filename, "rb") as r:
content = r.read()
Open a file for reading in async way:
async with await op.open(filename, "rb") as r:
content = await r.read()
We remove the old error classes and provide a couple of Exception based class for the error handling.
opendal.Error is based class for all the exceptions now.opendal.exceptions.Unexpected is added.opendal.exceptions.Unsupported is added.opendal.exceptions.ConfigInvalid is added.opendal.exceptions.NotFound is added.opendal.exceptions.PermissionDenied is added.opendal.exceptions.IsADirectory is added.opendal.exceptions.NotADirectory is added.opendal.exceptions.AlreadyExists is added.opendal.exceptions.IsSameFile is added.opendal.exceptions.ConditionNotMatch is added.opendal.exceptions.ContentTruncated is added.opendal.exceptions.ContentIncomplete is added.opendal.exceptions.InvalidInput is added.The naming convention for C binding has been altered.
Renaming certain struct names for consistency.
opendal_operator_ptr => opendal_operatoropendal_blocking_lister => opendal_listeropendal_list_entry => opendal_entryWe've eliminated the blocking_ prefix from our API because C binding doesn't currently support async. In the future, we plan to introduce APIs such as opendal_operator_async_write.
opendal_operator_blocking_write => opendal_operator_writeopendal_operator_blocking_read => opendal_operator_read<!-- Release notes generated using configuration in .github/release.yml at v0.42.0 -->
rename support by @G-XD in https://github.com/apache/incubator-opendal/pull/3238Metadata a POJO by @G-XD in https://github.com/apache/incubator-opendal/pull/3277memoryview instead of bytes to avoid copy by @messense in https://github.com/apache/incubator-opendal/pull/3310Metadata::metakey() api by @G-XD in https://github.com/apache/incubator-opendal/pull/3373list_with in Operator::list by @G-XD in https://github.com/apache/incubator-opendal/pull/3305OPENDAL_DISABLE_RANDOM_ROOT by @suyanhanx in https://github.com/apache/incubator-opendal/pull/3456connection_string config description in doc by @xring in https://github.com/apache/incubator-opendal/pull/3388range_reader change in upgrade doc by @wcy-fdu in https://github.com/apache/incubator-opendal/pull/3401testWriteFileWithNonAsciiName behavior test by @G-XD in https://github.com/apache/incubator-opendal/pull/3424linux-aarch_64 by @amunra in https://github.com/apache/incubator-opendal/pull/3527Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.41.0...v0.42.0
There are no public API and raw API changes.
There are no public API and raw API changes.
PR-3166 changes the API for constructing operators:
Previous:
new BlockingOperator(scheme, config);
new Operator(scheme, config);
Current:
BlockingOperator.of(scheme, config);
Operator.of(scheme, config);
Now, there is no public constructor for operators, but only factory methods. In this way, the APIs are free to do arbitrary verifications and preparations before constructing operators.
There are no API changes.
There are no API changes.
prometheus-client crate with PrometheusClientLayer by @flaneur2020 in https://github.com/apache/incubator-opendal/pull/3134BlockingLayer for non-blocking services that don't support blocking by @messense in https://github.com/apache/incubator-opendal/pull/3198copy test by @G-XD in https://github.com/apache/incubator-opendal/pull/3207WebdavFs to dav-server-fs-opendal by @Young-Flash in https://github.com/apache/incubator-opendal/pull/3119Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.40.0...v0.41.0
> Checkout our OwO #1 to know more about this release!
Checkout our OwO #1 to know more about this release!
RFC-2578 merges append into write and removes append API.
op.write() for convenience.op.write_with().append(true) instead of op.append().The same rule applies to writer() and writer_with().
RFC-2774 proposes a new lister API to replace current list and scan. And we add a new API list to return entries directly.
list() for convenience.list_with().delimiter("") or lister_with().delimiter("") instead of scan().lister() or lister_with() instead.RFC-2779 proposes a new op.list_with().metakey() API to allow list with metakey and removes op.metadata(&entry) API.
Please use op.list_with().metakey() instead of op.metadata(&entry), for example:
// Before
let entries: Vec<Entry> = op.list("dir/").await?;
for entry in entris {
let meta = op.metadata(&entry, Metakey::ContentLength | Metakey::ContentType).await?;
println!("{} {}", entry.name(), entry.metadata().content_length());
}
// After
let entries: Vec<Entry> = op
.list_with("dir/")
.metakey(Metakey::ContentLength | Metakey::ContentType).await?;
for entry in entris {
println!("{} {}", entry.name(), entry.metadata().content_length());
}
RFC-2852 proposes new native_capability and full_capability API to allow users to check if the underlying service supports a capability natively.
native_capability returns true if the capability is supported natively.full_capability returns true if the capability is supported, maybe via a layer.Most of time, you can use full_capability to replace capability call. But if to check if the capability is supported natively for better performance design, please use native_capability instead.
OpenDAL v0.40 added buffered writer support!
Users don't need to specify the content_length() for writer anymore!
- let mut w = op.writer_with("path/to/file").content_length(1024).await?;
+ let mut w = op.writer_with("path/to/file").await?;
Users can specify the buffer() to control the size we call underlying storage:
let mut w = op.writer_with("path/to/file").buffer(8 * 1024 * 1024).await?;
If buffer is not specified, we will call underlying storage everytime we call write. Otherwise, we will make sure to call underlying storage when buffer is full or close is called.
RFC-3017 removes copy_from API from the oio::Write trait. Users who implements services and layers by hand should remove this API.
read_dir by @Young-Flash in https://github.com/apache/incubator-opendal/pull/2736OpMonad to support custom IO monad by @silver-ymz in https://github.com/apache/incubator-opendal/pull/2789WebdavMetaData by @Young-Flash in https://github.com/apache/incubator-opendal/pull/2857_pin with !Unpin as argument by @morristai in https://github.com/apache/incubator-opendal/pull/2886content_length to writer by @Xuanwo in https://github.com/apache/incubator-opendal/pull/3044parse_batch_delete_response with Multipart::parse by @G-XD in https://github.com/apache/incubator-opendal/pull/3071webdav_put signatures by using OpWrite. by @laipz8200 in https://github.com/apache/incubator-opendal/pull/3076webdav_get. by @laipz8200 in https://github.com/apache/incubator-opendal/pull/3081oss_put_object signatures by using OpWrite by @sysu-yunz in https://github.com/apache/incubator-opendal/pull/3080http methods signature by using OpRead/OpStat by @miroim in https://github.com/apache/incubator-opendal/pull/3083gcs methods signature by using OpXxxx by @wavty in https://github.com/apache/incubator-opendal/pull/3087fixtures from core/src/services/{service} to top-level fixtures/{service} by @G-XD in https://github.com/apache/incubator-opendal/pull/3088webhdfs methods signature by using OpXxxx by @cxorm in https://github.com/apache/incubator-opendal/pull/3109us-east-1 for non-aws s3 by @G-XD in https://github.com/apache/incubator-opendal/pull/2812flush before sync_all by @WenyXu in https://github.com/apache/incubator-opendal/pull/3053Github to GitHub by @jbampton in https://github.com/apache/incubator-opendal/pull/3012MacOS to macOS by @jbampton in https://github.com/apache/incubator-opendal/pull/3013Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.39.0...v0.40.0
In PR #2687, OpenDAL changed the behavior when role_arn has been specified.
In PR #2687, OpenDAL changed the behavior when role_arn has been specified.
OpenDAL used to override role_arn simply. But since this version, OpenDAL will make sure to use assume_role with specified role_arn and external_id (if supplied).
In PR #2666, RetryLayer supports RetryInterceptor. To implement this change, RetryLayer changed it's in-memory layout by adding a new generic parameter I to RetryLayer<I>.
Users who stores RetryLayer in struct or enum will need to change the type if they don't want to use default behavior.
In PR #2698, OpenDAL re-org the internal structure of opendal::raw::oio and changed some APIs name.
<!-- Release notes generated using configuration in .github/release.yml at v0.39.0 -->
stat by @suyanhanx in https://github.com/apache/incubator-opendal/pull/2649HADOOP_CONF_DIR setting in guidance document by @wcy-fdu in https://github.com/apache/incubator-opendal/pull/2713Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.38.1...v0.39.0
feat(binding/lua): add rename and create_dir operator function by @oowl in https://github.com/apache/incubator-opendal/pull/2564
oay.toml by @messense in https://github.com/apache/incubator-opendal/pull/2615delete_with on blocking operator by @suyanhanx in https://github.com/apache/incubator-opendal/pull/2633Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.38.0...v0.38.1
One column per quarter.