NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI · #382 most downloaded on PyPI
Utils for streaming large files (S3, HDFS, GCS, SFTP, Azure Blob Storage, gzip, bz2, zst...)
Last release 5 days ago
29 Sep 2026
Release timing varies
gaps range from 9 days to 7 months
Nearly every release is documented
notes for 60 of the last 60 stable releases
4 versions withdrawn
withdrawn after publishing
12 years old
80 releases · first in 2015
Fix update_changelog.py triggering lint by @ddelange in #953
Full Changelog: v8.0.1...v8.0.2
One column per quarter.
Document missing major versions (v2, v4, v7) in MIGRATING by @ddelange in #949
Full Changelog: v8.0.0...v8.0.1
Note See MIGRATING_FROM_OLDER_VERSIONS.md for details on breaking changes.
Note
See MIGRATING_FROM_OLDER_VERSIONS.md for details on breaking changes.
Full Changelog: v7.7.1...v8.0.0
See the migration docs for details on breaking changes.
[http,azure] Optimize forward seeks within buffered data to avoid redundant requests by @ddelange in #923
Full Changelog: v7.7.0...v7.7.1
Add support for lz4 compression by @muxiaofan in #921
Full Changelog: v7.6.1...v7.7.0
Bump the github-actions group with 2 updates by @dependabot [bot] in #919
Full Changelog: v7.6.0...v7.6.1
Bump softprops/action-gh-release from 2.5.0 to 2.6.1 in the github-actions group by @dependabot [bot] in #918
append mode for Azure Blob Storage by @geovalexis in #865Full Changelog: v7.5.1...v7.6.0
append mode for Azure Blob Storage (PR #865, @geovalexis)Bump the github-actions group with 2 updates by @dependabot [bot] in #910
Full Changelog: v7.5.0...v7.5.1
[s3] Support very large buckets in iter_bucket, add client_kwargs by @ddelange in #908
[http] Eliminate _read_iter and simplify read by @ddelange in #906
Full Changelog: v7.4.3...v7.4.4
[s3] Simplify WHENCE_END logic by @ddelange in #902
Full Changelog: v7.4.2...v7.4.3
Add test for seek to end + defer_seek by @ddelange in https://github.com/piskvorky/smart_open/pull/901
Full Changelog: https://github.com/piskvorky/smart_open/compare/v7.4.1...v7.4.2
Prefill target branch and new tag on the new GitHub release page by @ddelange in https://github.com/piskvorky/smart_open/pull/897
Full Changelog: https://github.com/piskvorky/smart_open/compare/v7.4.0...v7.4.1
> [s3] As of this release, `smart_open.s3.iter_bucket` will use a single shared `concurrent.futures.ThreadPoolExecutor` with a single shared thread-sa
[!NOTE]
[s3] As of this release,smart_open.s3.iter_bucketwill use a single sharedconcurrent.futures.ThreadPoolExecutorwith a single shared thread-safeS3.Client. For more details, see https://github.com/piskvorky/smart_open/pull/896#discussion_r2444780459.
[!WARNING]
This version was yanked from PyPI because 7.4.0 introduced S3tcp_keepalive, which is available as of boto3 1.9.17. A lower pin was added in 7.4.1.
fallback_version when builing with uv by @DeflateAwning in https://github.com/piskvorky/smart_open/pull/884Full Changelog: https://github.com/piskvorky/smart_open/compare/v7.3.1...v7.4.0
fallback_version when builing with uv (PR #884, @DeflateAwning)Fix release.sh for the final merge back into develop by @ddelange in https://github.com/piskvorky/smart_open/pull/872
Full Changelog: https://github.com/piskvorky/smart_open/compare/v7.3.0.post1...v7.3.1
Fix release.sh merge message and final merge by @ddelange in https://github.com/piskvorky/smart_open/pull/868
Full Changelog: https://github.com/piskvorky/smart_open/compare/v7.3.0...v7.3.0.post1
> This version was yanked from PyPI because pyproject.toml falsely claimed that python 3.7 was still supported.
[!WARNING]
This version was yanked from PyPI becausepyproject.tomlfalsely claimed that python 3.7 was still supported.
Full Changelog: https://github.com/piskvorky/smart_open/compare/v7.2.0...v7.3.0
Add support for python 3.13 (PR #847, @ddelange)
Fix zstd compression in ab mode (PR #833, @ddelange)
get_blob_kwargs to GCS blob read operations (PR #817, @thejcannon)This release deprecates the old ignore_ext parameter. Use the compression parameter instead.
__next__ method to FileLikeProxy (PR #811, @ddelange)~/.ssh/config (PR #790, @wbeardall)KeyError: 'ContentRange' when received full content from S3 (PR #789, @messense)This release deprecates the old ignore_ext parameter.
Use the compression parameter instead.
fin = smart_open.open("/path/file.gz", ignore_ext=True) # No
fin = smart_open.open("/path/file.gz", compression="disable") # Yes
fin = smart_open.open("/path/file.gz", ignore_ext=False) # No
fin = smart_open.open("/path/file.gz") # Yes
fin = smart_open.open("/path/file.gz", compression="infer_from_extension") # Yes, if you want to be explicit
fin = smart_open.open("/path/file", compression=".gz") # Yes
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)This release deprecates the old ignore_ext parameter. Use the compression parameter instead.
__next__ method to FileLikeProxy (PR #811, @ddelange)~/.ssh/config (PR #790, @wbeardall)KeyError: 'ContentRange' when received full content from S3 (PR #789, @messense)This release deprecates the old ignore_ext parameter.
Use the compression parameter instead.
fin = smart_open.open("/path/file.gz", ignore_ext=True) # No
fin = smart_open.open("/path/file.gz", compression="disable") # Yes
fin = smart_open.open("/path/file.gz", ignore_ext=False) # No
fin = smart_open.open("/path/file.gz") # Yes
fin = smart_open.open("/path/file.gz", compression="infer_from_extension") # Yes, if you want to be explicit
fin = smart_open.open("/path/file", compression=".gz") # Yes
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)This release deprecates the old ignore_ext parameter. Use the compression parameter instead.
__next__ method to FileLikeProxy (PR #811, @ddelange)~/.ssh/config (PR #790, @wbeardall)KeyError: 'ContentRange' when received full content from S3 (PR #789, @messense)This release deprecates the old ignore_ext parameter.
Use the compression parameter instead.
fin = smart_open.open("/path/file.gz", ignore_ext=True) # No
fin = smart_open.open("/path/file.gz", compression="disable") # Yes
fin = smart_open.open("/path/file.gz", ignore_ext=False) # No
fin = smart_open.open("/path/file.gz") # Yes
fin = smart_open.open("/path/file.gz", compression="infer_from_extension") # Yes, if you want to be explicit
fin = smart_open.open("/path/file", compression=".gz") # Yes
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)Do not touch botocore unless it is installed (PR #803, @ddelange)
~/.ssh/config (PR #790, @wbeardall)KeyError: 'ContentRange' when received full content from S3 (PR #789, @messense)Upgrade dev status classifier to stable (PR #798, @seebi)
~/.ssh/config (PR #790, @wbeardall)KeyError: 'ContentRange' when received full content from S3 (PR #789, @messense)~/.ssh/config (PR #790, @wbeardall)KeyError: 'ContentRange' when received full content from S3 (PR #789, @messense)Ignore S3 seeks to the current position (PR #782, @beck3905)
upgrade pyopenssl versions as part of github actions workflows by @mpenkov in https://github.com/RaRe-Technologies/smart_open/pull/722
Full Changelog: https://github.com/RaRe-Technologies/smart_open/compare/v6.2.0...v6.3.0
This release deprecates the old ignore_ext parameter. Use the compression parameter instead.
This release deprecates the old ignore_ext parameter.
Use the compression parameter instead.
fin = smart_open.open("/path/file.gz", ignore_ext=True) # No
fin = smart_open.open("/path/file.gz", compression="disable") # Yes
fin = smart_open.open("/path/file.gz", ignore_ext=False) # No
fin = smart_open.open("/path/file.gz") # Yes
fin = smart_open.open("/path/file.gz", compression="infer_from_extension") # Yes, if you want to be explicit
fin = smart_open.open("/path/file", compression=".gz") # Yes
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)Add cert parameter to http transport params (PR #703, @stev-0)
This release deprecates the old ignore_ext parameter. Use the compression parameter instead.
This release deprecates the old ignore_ext parameter.
Use the compression parameter instead.
fin = smart_open.open("/path/file.gz", ignore_ext=True) # 🚫 No
fin = smart_open.open("/path/file.gz", compression="disable") # Yes
fin = smart_open.open("/path/file.gz", ignore_ext=False) # 🚫 No
fin = smart_open.open("/path/file.gz") # Yes
fin = smart_open.open("/path/file.gz", compression="infer_from_extension") # Yes, if you want to be explicit
fin = smart_open.open("/path/file", compression=".gz") # Yes
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)For details, see the README.rst file. You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)For details, see the README.rst file. You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)For details, see the README.rst file. You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)This new function replaces smart_open.smart_open, which is now deprecated. Main differences:
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)This new function replaces smart_open.smart_open, which is now deprecated. Main differences:
s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)This new function replaces smart_open.smart_open, which is now deprecated. Main differences:
s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)This new function replaces smart_open.smart_open, which is now deprecated. Main differences:
s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)Exclude requests from install_requires dependency list. If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].
requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].Fix reading empty file or seeking past end of file for s3 backend (PR #549, @jcushman)
This release modifies the behavior of setup.py with respect to dependencies. Previously, boto3 and other AWS-related packages were installed by defaul
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only.
Or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Include S3 dependencies by default, because removing them in the 2.2.0 minor release was a mistake. It broke existing code in a minor release.
This release modifies the behavior of setup.py with respect to dependencies. Previously, boto3 and other AWS-related packages were installed by defaul
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[aws]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketBypass unnecessary GCS storage.buckets.get permission (PR #516, @gelioz)
Azure storage blob support (@nclsmitchell and @petedannemann)
newline parameter to built-in open function (PR #478, @burkovae)This version supports Python 3 only (3.5+).
Add missing boto dependency (Issue #468)
pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
Fix GCS multiple writes (PR #421, @petedannemann)
pathlib.Path.open (PR #436, @menshikh-iv)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
This is the last version to support Python 2.7. Versions 1.11 and above will support Python 3 only.
This is the last version to support Python 2.7. Versions 1.11 and above will support Python 3 only.
Various webhdfs improvements (PR #383, @mrk-its)
Add version_id transport parameter for fetching a specific S3 object version (PR #325, @interpolatio)
Don't use s3 bucket_head to check for bucket existence (PR #315, @caboteria)
bucket_head to check for bucket existence (PR #315, @caboteria)
Improve S3 read performance by not copying buffer (PR #284, @aperiodic)
Removed dependency on lzma (PR #262, @tdhopper)
__version__ attribute?)This release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see README.rst for details.
This release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
Added support for .xz / lzma (PR #262, @vmarkovtsev)
1.8.1, 6 April 2019
smart_open.open function (PR #268, @mpenkov)smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
Drop deprecated sudo from travis config (PR #256, @cclauss)
1.8.0, 17th January 2019
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)Unpin boto/botocore for regular installation. Fix #227 (PR #232, @menshikh-iv)
1.7.1, 18th September 2018
Drop support for Python 3.3 and 3.4: smart_open now requires either 2.7 or 3.5+.
1.7.0, 18th September 2018
smart_open now requires either 2.7 or 3.5+.moto testing library (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)
s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)Migrate to boto3. Fix #43 (PR #164, @mpenkov)
1.6.0, 29th June 2018
boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)Fix author/maintainer fields in setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177,
1.5.7, 18th March 2018
Fix author/maintainer fields in setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)
Create separate compression-specific layer & add seek support for S3. Fix #91, #42 (PR #131, @mpenkov)
1.5.6, 28th December 2017
seek support for S3. Fix #91, #42 (PR #131, @mpenkov)Fix problems from 1.5.4 release. Fix #153, #154 , partial fix #152 (PR #155, @mpenkov)
Your coding agent can read these notes before it upgrades. Set up the MCP server →