NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI, #399 most downloaded on PyPI
Utils for streaming large files (S3, HDFS, GCS, SFTP, Azure Blob Storage, gzip, bz2, zst...)
Latest stable
10.0.0
Last release 2 days ago
20 Nov 2026
Release timing varies
gaps range from 8 days to 4 months
Nearly every release is documented
notes for 60 of the last 60 stable releases
4 versions withdrawn
withdrawn after publishing
12 years old
90 releases · first in 2015
Drag across the ruler to compare, or tap a release to open it.
Note SSH now refuses servers that are not in ~/.ssh/known_hosts , and raises paramiko.SSHException where it used to connect and send your password. Se
Note
SSH now refuses servers that are not in ~/.ssh/known_hosts, and raises paramiko.SSHException where it used to connect and send your password.
See MIGRATING_FROM_OLDER_VERSIONS.md for the new known_hosts transport param, the opt out, and the other SSH config changes.
Full Changelog: v9.0.6...v10.0.0
Your coding agent can read these notes before it upgrades. Set up the MCP server →
[bytebuffer] Speed up readline when the line is already in the buffer by @g4lb in #1010
Full Changelog: v9.0.5...v9.0.6
Fail CI for pull requests that target master by @ddelange in #1006
Full Changelog: v9.0.4...v9.0.5
Remove MIT License classifier from pyproject.toml by @APN-Pucky in #1003
Full Changelog: v9.0.3...v9.0.4
Remove the locals() debug dumps that log credentials by @kakkar87d in #997
Full Changelog: v9.0.2...v9.0.3
Bump pypa/gh-action-pypi-publish from 1.14.2 to 1.14.3 in the github-actions group by @dependabot [bot] in #994
Full Changelog: v9.0.1...v9.0.2
[http] Release responses when numeric header parsing fails by @edsonsims in #992
Full Changelog: v9.0.0...v9.0.1
Note HTTP range requests are now pinned to the version first read, so seeks fail with 412 against servers whose validators change between requests. Se
Note
HTTP range requests are now pinned to the version first read, so seeks fail with 412 against servers whose validators change between requests.
See MIGRATING_FROM_OLDER_VERSIONS.md for the opt out and the other HTTP behaviour changes.
Full Changelog: v8.0.4...v9.0.0
Use binary unit labels (GiB, MiB) in upload size logs by @ddelange in #978
Full Changelog: v8.0.3...v8.0.4
Avoid consuming source data during zero-size buffer fills by @xiehuanyi in #958
Full Changelog: v8.0.2...v8.0.3
Fix update_changelog.py triggering lint by @ddelange in #953
Full Changelog: v8.0.1...v8.0.2
Document missing major versions (v2, v4, v7) in MIGRATING by @ddelange in #949
Full Changelog: v8.0.0...v8.0.1
Note See MIGRATING_FROM_OLDER_VERSIONS.md for details on breaking changes.
Note
See MIGRATING_FROM_OLDER_VERSIONS.md for details on breaking changes.
Full Changelog: v7.7.1...v8.0.0
See the migration docs for details on breaking changes.
[http,azure] Optimize forward seeks within buffered data to avoid redundant requests by @ddelange in #923
Full Changelog: v7.7.0...v7.7.1
Add support for lz4 compression by @muxiaofan in #921
Full Changelog: v7.6.1...v7.7.0
Bump the github-actions group with 2 updates by @dependabot [bot] in #919
Full Changelog: v7.6.0...v7.6.1
Bump softprops/action-gh-release from 2.5.0 to 2.6.1 in the github-actions group by @dependabot [bot] in #918
append mode for Azure Blob Storage by @geovalexis in #865Full Changelog: v7.5.1...v7.6.0
append mode for Azure Blob Storage (PR #865, @geovalexis)Bump the github-actions group with 2 updates by @dependabot [bot] in #910
Full Changelog: v7.5.0...v7.5.1
[s3] Support very large buckets in iter_bucket, add client_kwargs by @ddelange in #908
[http] Eliminate _read_iter and simplify read by @ddelange in #906
Full Changelog: v7.4.3...v7.4.4
[s3] Simplify WHENCE_END logic by @ddelange in #902
Full Changelog: v7.4.2...v7.4.3
Add test for seek to end + defer_seek by @ddelange in https://github.com/piskvorky/smart_open/pull/901
Full Changelog: https://github.com/piskvorky/smart_open/compare/v7.4.1...v7.4.2
Prefill target branch and new tag on the new GitHub release page by @ddelange in https://github.com/piskvorky/smart_open/pull/897
Full Changelog: https://github.com/piskvorky/smart_open/compare/v7.4.0...v7.4.1
> [s3] As of this release, `smart_open.s3.iter_bucket` will use a single shared `concurrent.futures.ThreadPoolExecutor` with a single shared thread-sa
[!NOTE]
[s3] As of this release,smart_open.s3.iter_bucketwill use a single sharedconcurrent.futures.ThreadPoolExecutorwith a single shared thread-safeS3.Client. For more details, see https://github.com/piskvorky/smart_open/pull/896#discussion_r2444780459.
[!WARNING]
This version was yanked from PyPI because 7.4.0 introduced S3tcp_keepalive, which is available as of boto3 1.9.17. A lower pin was added in 7.4.1.
fallback_version when builing with uv by @DeflateAwning in https://github.com/piskvorky/smart_open/pull/884Full Changelog: https://github.com/piskvorky/smart_open/compare/v7.3.1...v7.4.0
fallback_version when builing with uv (PR #884, @DeflateAwning)Fix release.sh for the final merge back into develop by @ddelange in https://github.com/piskvorky/smart_open/pull/872
Full Changelog: https://github.com/piskvorky/smart_open/compare/v7.3.0.post1...v7.3.1
Fix release.sh merge message and final merge by @ddelange in https://github.com/piskvorky/smart_open/pull/868
Full Changelog: https://github.com/piskvorky/smart_open/compare/v7.3.0...v7.3.0.post1
> This version was yanked from PyPI because pyproject.toml falsely claimed that python 3.7 was still supported.
[!WARNING]
This version was yanked from PyPI becausepyproject.tomlfalsely claimed that python 3.7 was still supported.
Full Changelog: https://github.com/piskvorky/smart_open/compare/v7.2.0...v7.3.0
Add support for python 3.13 (PR #847, @ddelange)
Fix zstd compression in ab mode (PR #833, @ddelange)
get_blob_kwargs to GCS blob read operations (PR #817, @thejcannon)This release deprecates the old ignore_ext parameter. Use the compression parameter instead.
__next__ method to FileLikeProxy (PR #811, @ddelange)~/.ssh/config (PR #790, @wbeardall)KeyError: 'ContentRange' when received full content from S3 (PR #789, @messense)This release deprecates the old ignore_ext parameter.
Use the compression parameter instead.
fin = smart_open.open("/path/file.gz", ignore_ext=True) # No
fin = smart_open.open("/path/file.gz", compression="disable") # Yes
fin = smart_open.open("/path/file.gz", ignore_ext=False) # No
fin = smart_open.open("/path/file.gz") # Yes
fin = smart_open.open("/path/file.gz", compression="infer_from_extension") # Yes, if you want to be explicit
fin = smart_open.open("/path/file", compression=".gz") # Yes
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)This release deprecates the old ignore_ext parameter. Use the compression parameter instead.
__next__ method to FileLikeProxy (PR #811, @ddelange)~/.ssh/config (PR #790, @wbeardall)KeyError: 'ContentRange' when received full content from S3 (PR #789, @messense)This release deprecates the old ignore_ext parameter.
Use the compression parameter instead.
fin = smart_open.open("/path/file.gz", ignore_ext=True) # No
fin = smart_open.open("/path/file.gz", compression="disable") # Yes
fin = smart_open.open("/path/file.gz", ignore_ext=False) # No
fin = smart_open.open("/path/file.gz") # Yes
fin = smart_open.open("/path/file.gz", compression="infer_from_extension") # Yes, if you want to be explicit
fin = smart_open.open("/path/file", compression=".gz") # Yes
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)This release deprecates the old ignore_ext parameter. Use the compression parameter instead.
__next__ method to FileLikeProxy (PR #811, @ddelange)~/.ssh/config (PR #790, @wbeardall)KeyError: 'ContentRange' when received full content from S3 (PR #789, @messense)This release deprecates the old ignore_ext parameter.
Use the compression parameter instead.
fin = smart_open.open("/path/file.gz", ignore_ext=True) # No
fin = smart_open.open("/path/file.gz", compression="disable") # Yes
fin = smart_open.open("/path/file.gz", ignore_ext=False) # No
fin = smart_open.open("/path/file.gz") # Yes
fin = smart_open.open("/path/file.gz", compression="infer_from_extension") # Yes, if you want to be explicit
fin = smart_open.open("/path/file", compression=".gz") # Yes
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)Do not touch botocore unless it is installed (PR #803, @ddelange)
~/.ssh/config (PR #790, @wbeardall)KeyError: 'ContentRange' when received full content from S3 (PR #789, @messense)Upgrade dev status classifier to stable (PR #798, @seebi)
~/.ssh/config (PR #790, @wbeardall)KeyError: 'ContentRange' when received full content from S3 (PR #789, @messense)~/.ssh/config (PR #790, @wbeardall)KeyError: 'ContentRange' when received full content from S3 (PR #789, @messense)Ignore S3 seeks to the current position (PR #782, @beck3905)
upgrade pyopenssl versions as part of github actions workflows by @mpenkov in https://github.com/RaRe-Technologies/smart_open/pull/722
Full Changelog: https://github.com/RaRe-Technologies/smart_open/compare/v6.2.0...v6.3.0
This release deprecates the old ignore_ext parameter. Use the compression parameter instead.
This release deprecates the old ignore_ext parameter.
Use the compression parameter instead.
fin = smart_open.open("/path/file.gz", ignore_ext=True) # No
fin = smart_open.open("/path/file.gz", compression="disable") # Yes
fin = smart_open.open("/path/file.gz", ignore_ext=False) # No
fin = smart_open.open("/path/file.gz") # Yes
fin = smart_open.open("/path/file.gz", compression="infer_from_extension") # Yes, if you want to be explicit
fin = smart_open.open("/path/file", compression=".gz") # Yes
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)Add cert parameter to http transport params (PR #703, @stev-0)
This release deprecates the old ignore_ext parameter. Use the compression parameter instead.
This release deprecates the old ignore_ext parameter.
Use the compression parameter instead.
fin = smart_open.open("/path/file.gz", ignore_ext=True) # 🚫 No
fin = smart_open.open("/path/file.gz", compression="disable") # Yes
fin = smart_open.open("/path/file.gz", ignore_ext=False) # 🚫 No
fin = smart_open.open("/path/file.gz") # Yes
fin = smart_open.open("/path/file.gz", compression="infer_from_extension") # Yes, if you want to be explicit
fin = smart_open.open("/path/file", compression=".gz") # Yes
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)For details, see the README.rst file. You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)For details, see the README.rst file. You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
urllib.parse.urlsplit (PR #633, @judahrand)This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)For details, see the README.rst file. You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release introduces a new top-level parameter: compression.
It controls compression behavior and partially overlaps with the old ignore_ext parameter.
For details, see the README.rst file.
You may continue to use ignore_ext parameter for now, but it will be deprecated in the next major release.
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)This new function replaces smart_open.smart_open, which is now deprecated. Main differences:
This release modifies the handling of transport parameters for the S3 back-end in a backwards-incompatible way. See the migration docs for details.
tests directory from package (PR #589, @e-nalepa)s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)This new function replaces smart_open.smart_open, which is now deprecated. Main differences:
s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)This new function replaces smart_open.smart_open, which is now deprecated. Main differences:
s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)This new function replaces smart_open.smart_open, which is now deprecated. Main differences:
s3 submodule to minimize resource usage (PR #569, @mpenkov)download_as_string to download_as_bytes in gcs submodule (PR #571, @alexandreyc)requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketnewline parameter to built-in open function (PR #478, @burkovae)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
__version__ attributeThis release rolls back support for transparently decompressing .xz files, previously introduced in 1.8.1. This is a useful feature, but it requires a tricky dependency. It's still possible to handle .xz files with relatively little effort. Please see the README.rst file for details.
smart_open.open function (PR #268, @mpenkov)This new function replaces smart_open.smart_open, which is now deprecated.
Main differences:
transport_params dict parameter to contain keyword parameters for the transport layer (S3, HTTPS, HDFS, etc).Main advantages of the new function:
The old smart_open.smart_open function is deprecated, but continues to work as previously.
python3.7 support (PR #240, @menshikh-iv)http/https schema correctly (PR #242, @gliv)S3 (PR #235, @rileypeterson)_parse_uri_s3x, resolve edge cases (PR #237, @mpenkov)sudo from travis config (PR #256, @cclauss)ValueError if s3 key does not exist (PR #245, @adrpar)_list_bucket uses continuation token for subsequent pages (PR #246, @tcsavage)python3.3 and python3.4 & workaround for broken moto (PR #225, @menshikh-iv)s3a:// support for S3. Fix #210 (PR #229, @mpenkov)@ in object (key) names for S3. Fix #94 (PRs #204 & #224, @dkasyanov & @mpenkov)close idempotent & add dummy flush for S3 (PR #212, @mpenkov)open whenever possible. Fix #207 (PR #208, @mpenkov)uri in smart_open_lib.py. Fix #213 (PR #214, @cclauss)boto3. Fix #43 (PR #164, @mpenkov)python2.6 compatibility. Fix #156 (PR #192, @mpenkov)boto3.Session instance (support STS AssumeRole). Fix #130, #149, #199 (PR #201, @eschwartz)multipart_upload parameters (supports ServerSideEncryption) for S3. Fix (PR #202, @eschwartz)pathlib.Path. Fix #170 (PR #175, @clintval)ParsedUri class with functions, cleanup internal argument parsing (PR #191, @mpenkov)f._current_pos when call f.readline() (PR #182, @inksink)seek for S3. Fix #187 (PR #188, @inksink)setup.py, avoid bug from setuptools==39.0.0 and add workaround for botocore and python==3.3. Fix #176 (PR #178 & #177, @menshikh-iv & @baldwindc)s3_iter_bucket_process_key to address S3 Read Timeout errors. (PR #96, @bbbco)encrypt_key and other parameters to initiate_multipart_upload (PR #63, @asieira)host and profile_name to smart_open (PR #71 #68, @robcowie)LC_ALL=C environment variable setup (PR #40, @nikicc)Exclude requests from install_requires dependency list. If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].
requests from install_requires dependency list.
If you need it, use pip install smart_open[http] or pip install smart_open[webhdfs].Fix reading empty file or seeking past end of file for s3 backend (PR #549, @jcushman)
This release modifies the behavior of setup.py with respect to dependencies. Previously, boto3 and other AWS-related packages were installed by defaul
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[s3]
to install the AWS dependencies only.
Or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Include S3 dependencies by default, because removing them in the 2.2.0 minor release was a mistake. It broke existing code in a minor release.
This release modifies the behavior of setup.py with respect to dependencies. Previously, boto3 and other AWS-related packages were installed by defaul
This release modifies the behavior of setup.py with respect to dependencies.
Previously, boto3 and other AWS-related packages were installed by default.
Now, in order to install them, you need to run either:
pip install smart_open[aws]
to install the AWS dependencies only, or
pip install smart_open[all]
to install all dependencies, including AWS, GCS, etc.
Summary of changes:
newline parameter to built-in open function (PR #478, @burkovae)to_boto3 method (PR #539, @interpolatio)Functionality on the left hand side will be removed in future releases. Use the functions on the right hand side instead.
smart_open.s3_iter_bucket → smart_open.s3.iter_bucketBypass unnecessary GCS storage.buckets.get permission (PR #516, @gelioz)
Azure storage blob support (@nclsmitchell and @petedannemann)
newline parameter to built-in open function (PR #478, @burkovae)This version supports Python 3 only (3.5+).
Add missing boto dependency (Issue #468)
pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
Fix GCS multiple writes (PR #421, @petedannemann)
pathlib.Path.open (PR #436, @menshikh-iv)pathlib.Path.open (PR #436, @menshikh-iv)Starting with this release, you will have to run:
pip install smart_open[gcs] to use the GCS transport.
In the future, all extra dependencies will be optional. If you want to continue installing all of them, use:
pip install smart_open[all]
See the README.rst for details.
This is the last version to support Python 2.7. Versions 1.11 and above will support Python 3 only.
This is the last version to support Python 2.7. Versions 1.11 and above will support Python 3 only.
Various webhdfs improvements (PR #383, @mrk-its)
Add version_id transport parameter for fetching a specific S3 object version (PR #325, @interpolatio)
Don't use s3 bucket_head to check for bucket existence (PR #315, @caboteria)
bucket_head to check for bucket existence (PR #315, @caboteria)
One column per quarter.