py7zr

Added

• SevenZipFile.needs_password() method.(#208, #235)
• CLI: Support append mode command line.(#228)
• Support "APPEND" mode. User can open SevenZipFile() class with mode='a' (#227)

Changed

• read(), extract(): improve performance when specifying parts of archived file, by skipping rest of arcvhive when target file has extracted.(#239,#242)
• read(), extract(): improve performance when specifying parts of archived file, by not running threads for unused compression blocks(folders).(#239,#242)
• docs: improve API documentation.(#244)
• setup: set minimum required python version as >=3.5
• Compression will be happened when call write() not close() (#222, #226)
• Handle file read/write in SevenZipCompressor/Decompressor class (#213)

Fixed

• Raise PasswordRequired when encrypted header without passing password (#234, #237)
• CLI: don't raise exception when password is wrong or not given.(#229)
• Fix specification typo.
• Catch exception in threading extraction(#218,#219)

Fixed

• Catch exception in threading extraction(#218,#219)

• avoid reading header data to calculate crc32 after writing header data, now calculation happened when writing header data.

Fixed

• Raise PasswordRequired when encrypted header and no password given(#238)

Fixed

• Catch exception in threading extraction(#218, #220)

Changed

• Raise 'PasswordRequired' exception when no password given for encrypted archive (#215, #216)

Fixed

• Support COPY compression method.

Changed

• ZStandard compression/decompression handle property.

Changed

• Utilize max_length argument for each decompressor.(#210, #211)
• Change READ_BUFFER_SIZE 32768 for python 3.7.5 and before.
• Extend Buffer size when necessary.(#209)

Changed

• Improve DecompressionChain.decompress() logics.(#207)

Fixed

• Fix BCJ filter for decompression that can cause infinite loop or wrong output.(#204,#205,#206)

CAUTION: v0.9.0 has a BCJ bug that produce broken archive., or fail to extract. Please use v0.9.1 bug fix version for LZMA1+BCJ.

Added

• BCJ Decoder/Encoder written by python.(#198, #199)
• Support Bzip2, Defalte + BCJ(X86, PPC, ARM, ARMT, SPARC) (#199)
• Add Copy method as an extraction only support.(#184)

Changed

• Use large(1MB) read blocksize for Python 3.7.5 and later and PyPy 7.2.0 and later.
• Set ZStandard compression as unsupported because of a bug with unknown reason.(#198)
• Manage compression methods to handle whether decompressor requires coder['property'] or not.

Fixed

• Significantly improve decompress performance which is as same speed as v0.7.*. by updating buffer handling.
• Fix decompression max_size to pass lzma module. Now it is as same as out_remaining.
• Support LZMA+BCJ(X86, PPC, ARM, ARMT, SPARC) with alternative BCJ filter.(#198, #199)
• Fix packinfo crc read and write (#187, #189)
• Accept archive which Method ID is NULL(size=0)(#181, #182)
• CLI: Does not crash when trying extract archive which use unsupported method(#183)

• Always use max_length as out_remianing to avoid enlarge queue inside lzma module.
• Use READ_BLOCKSIZE 1MB for recent python versions which has fixed the lzma bug.

This improve performance for large size of archived files.

Chnaged

• Internal helper class Buffer now grow when addition or set request a larger data.(#202)

Fixed

• Significantly improve decompress performance which is as same speed as v0.7.*. by updating buffer handling.(#203)

Changed

• Use large(1MB) read blocksize for Python 3.7.5 and later and PyPy 7.2.0 and later.

Fixed

• Fix decompression max_size to pass to lzma module as same as out_remaining.

Fixed

• Fix packinfo crc read and write (#187, #189)

Added

• Add Copy method as an extraction only support.(#184)

Fixed

• Accept archive which Method ID is NULL(size=0)(#181, #182)

• CLI: Does not crash when trying extract archive which use unsupported method(#183)

• Update documentation, artwork and specification document.

Added

• File format specification: add ISO/IEC standard style specification document.

Changed

• Update password handling and drop get_password() helper (#162)

Fixed

• Now return correct header size by archiveinfo() method.(#169)

Removed

• Drop ArchiveProperty class: A field has already deprecated or not used.(#170)
• Drop AntiFile property: a property has already deprecated or not used.

Changed

• Enable encoded header and add more test with 7zip compatibility.(#164)

Fixed

• Disable adding CRC for encoded header packinfo.(#164)

Fixed

• Fix password leak/overwrite among SevenZipFile objects in a process.(#159) This can cause decryption error or encryption with unintended password.
• Release password on close()

Changed

• Refactoring SevenZipFile class internals. (#160)
• Refactoring classes in compressor module. (#161)

Fixed

• SevenZipFile.test() method now working properly. (#155)

Changed

• Add 'packinfo.crcs' field digests data when creating archive.(#157) It help checking archive integrity without extraction.
• CLI: help option to show py7zr version and python version.
• Use importlib for performance improvement instead of pkg_resources module.

Added

• Support extra methods for archiveinfo() method.(#150)

Added

• test: unit tests for Sparc, ARMT and IA64 filters.

Changed

• Documents: additional methods, filter examples.
• CI configurations: Manage coverage with Coveralls.

Changed

• Refactoring decompression classes to handle data precisely with folder.unpacksizes(#146)

Added

Support for PPC and ARM filters.

Added

• Support encryption(#145)
• Export supported filter constants, such as FILTER_ZSTD(#145)

Changed

• Default compression mode is LZMA2+BCJ which is as same as 7zip and p7zip(#145)
• Enhance encryption strength, IV is now 16 bytes, and generated with cryptodom.random module.(#145)

Fixed

• Support combination of filters(#145)
• Compression of Delta, BZip2, ZStandard, and Deflate(#145)

Fixed

• Extraction fails on python 3.5.(#151)

Added

• Support for encrypted header (#139, #140)

Changed

• Fix CRC32 check and introduce test and testzip methods (#138)

Fixed

• Allow decryption of data which is encrypted without any compression.(#140)

Changed

• Decryption: performance improvement. Introduce helpers.calculate_key3(), which utilize list comprehension expression, bytes generation with join(). It reduces a number of calls of hash library and improve decryption performance.

Changed

• Extraction: Unlink output file if exist when it become a symbolic link. When overwrite extracted files and there are symlinks, it may cause an unexpected result. Unlinking it may help it.

Added

• Support dereference option of SevenZipFile class. (#131) If dereference is False, add symbolic and hard links to the archive. If it is True, add the content of the target files to the archive. This has no effect on systems that do not support symbolic links.
• Introduce progress callback mechanism (#130)

Changed

• CLI: add --verbose option for extraction
• win32: update win32compat

Fixed

• Fix archiveinfo() for 7zAES archives
• Release variables when close() (#129)

Fixed

• Support extraction of file onto a place where path length is > 260 bytes on Windows 10, Windows Server 2016R2 and later. (Windows Vista, 7 and Windows Server 2012 still have a limitation of path length as a OS spec)(#116, #126)

Added

• Support memory API.(#111, #119) Introduce read(filter) and readall() method for SevenZipFile class.
• Support ZStandard codec compression algorithm for extraction.(#124, #125)

Changed

• Drop pywin32 dependency
• Introduce internal win32compat.py
• Archive: Looking for symbolic link object in the archived list, and if found, record as relative link.(#112, #113, #122)

Removed

• Revmoed requirements.txt. When you want to install dependencies for development you can do it with 'pip install -e path/to/py7zr_project'

Added

• Test: SevenZipFile.archiveinfo() for various archives.
• Test: extraction of LZMA+BCJ archive become fails as marked known issue.
• Support deflate decompression method.

Changed

• Update documents and README about supported algorithms.
• Re-enable coverage report.
• Refactoring SevenZipFile._write_archive() method to move core chunk into compression module Worker.archive() method.
• Update calculate_key helper to improve performance.
• Introduce zero-copy buffer helper.
• Change decompressor class interface
  • change max_length type to int and defualt to -1.
• Update decryption function to improve performance.

Fixed

• Fix SevenZipFIle.archiveinfo() crash for LZMA+BCJ archive.(#100)
• Fix SevenZipFile.test() method defeated from v0.6b2 (#103)
• Fix SevenZipFile.solid() method to return proper value. (#72,#97)

Added

• Introduce context manager for SevenZipFile (#95)

Changed

• SevenZipFile(file-object, 'r') now can run extract() well even unlink before extract().

Fixed

• Fix README example for extraction option.
• Some of decryption of encrypted archive fails.(#75)

After run several benchmark test, py7zr takes more benefit with threading than multiprocessing. Decompression/compression is purely CPU bind but the process produces I/O read-write, then threading is optimal.

Added

• Test: add benchmarking test.

Changed

• Concurrency strategy: change to threading instead of multiprocessing. (#92)

Fixed

• Make pywin32 a regular runtime dependency

• Fix a bug that cause file-no-found error on multi-thread application because py7zr do a file operation with temporal context of os.chdir().

Added

• Add concurrent extraction test.
• Add remote data test for general application test.
• Add class for multi volume header.
• Add readlink helper function for windows.

Changed

• Release process is done by Github Actions
• Temporary disable to measure coverage, which is not working with threading.
• Tox: now pass PYTEST_ADDOPTS environment variable.

Fixed

• Build with pep517 utility.
• Fix race condition for changing current working directory of caller, which cause failures in multithrea ding.(#80,#82)

Security

• CLI: Use 'getpass' standard library to input password.(#59)

Changed

• extract: decompression is done as another process in default.
• extract: default multiprocessing mode is spawn
• extract: single process mode for password protected archive.

Fixed

• extract: catch UnsupportedMethod exception properly when multiprocessing.

• A versions v0.3 - v0.6b1 has a problem on concurrency method. It uses 'concurrency.futures' . If application uses it and call py7zr, it become freeze or dead lock.

This release replace it with 'multiprocessing.Process()'.

Changed

• Concurrency implementation changes to use multiprocessing.Process() instead of concurrency.futures to avoid freeze or deadlock with application usage of it.(#70)
• Stop checking coverage because coverage.py > 5.0.0 produce error when multiprocessing.Process() usage.
• Drop handlers, NullHandler, BufferHnalder, and FileHander.

Known Issues

• Extraction of encrypted archive which has multiple compression folders fails when multiprocessing mode is not 'fork', that is python3.8 and later on MacOS, and on Windows. see. test_extract_encrypted_2()