rarfile

Fixes

• Disallow extraction outside extraction path, in case of existing symlink. [#114]
• Disallow creating symlinks to outside of extraction path. [#118]
• Apply length limit to passwords, so too long password give same result as for unrar.

Features:

• Support unrar-free >= 0.2.0. [#103]
• Add RarFile.is_solid to check if archive uses solid compression. [#101]

Fixes:

• Support old multi-volume archives better where ENDARC does not contain NEXTVOL. [#97]

Cleanups:

• ci: Drop Python 3.7, add 3.12
• ci: upgrade actions

Features:

• Support 7zip/p7zip as decompression backend. [#71]
• RAR5: check password before trying to read file (chigusa) [#79]

New APIs:

• Make get_rar_version a public function (Safihre) [#63]
• New option: part_only for RarFile, to read only single file and allow it to be middle-part of multi-volume archive.
• Add RarFile.printdir, use it in dumprar. Needed to examine FILE_COPY or HARD_LINK entries that do not contain data.

Fixes:

• Use OS separator to access filename. Should fix subdirectory entry access on Windows. [#96]
• DirectReader: check seek position on each read. Fixes read reading from multiple entries in parallel on RarFile backed by file object. [#81]
• DirectReader: properly disable CRC check when seeking. [#73]
• Reset _hdrenc_main before processing a new volume. Fixes CRC checks on multi-volume reads. [#80]
• Adapt to Python 3.10 argparse (MeggyCal) [#85]
• SFX: Handle volume numbering special cases better.
• nsdatetime: support pypy internal use
• Throw error if backend does not support passwords.

Cleanups:

• ci: Use proper unrar on Windows. MingW one tolaretes paths with / better than upstream build.
• ci: Add Python 3.10 to the testing (Christian Clauss) [#76]
• Avoid isascii, not in 3.6

Main goals are:

• Increased zipfile-compatibility, thus also achieving smaller difference between RAR3 and RAR5 archives.
• Implement RarFile.extract on top of RarFile.open instead using unrar x directly, thus making maintenance of alternative backends more manageable. Negative aspect of that is that there are features that internal extract code does not support - hard links, NTFS streams and junctions.

Breaking changes:

• Directory names will have "/" appended. [#31]
• RarFile.extract operates only on single entry, so when used on directory it will create directory but not extract files under it.
• RarFile.extract/RarFile.extractall/RarFile.testrar will not launch special unrar command line, instead they are implemented on top of RarFile.open.
• Keyword args in top-level APIs were renamed to match zipfile:
  • RarFile(rarfile) -> RarFile(file)
  • RarFile.setpassword(password) -> .setpassword(pwd)
  • RarFile.getinfo(fname) -> .getinfo(name)
  • RarFile.open(fname, mode, psw) -> .open(name, mode, pwd)
  • RarFile.read(fname, psw) -> .read(name, pwd)
• PATH_SEP cannot be changed from "/".

New features:

• RarFile.extract will return final sanitized filename for target file. [#42, #52]
• RarInfo.is_dir is now preferred spelling of isdir(). Old method kept as alias. [#44]
• New RarInfo.is_file and RarInfo.is_symlink methods. Only one of ~RarInfo.is_file, ~RarInfo.is_dir or ~RarInfo.is_symlink can be True.
• RarFile.printdir has file argument for output.
• RarFile.__iter__ loops over RarInfo entries.
• RAR3: throw NeedFirstVolume exception with current volume number, like RAR5 does. [#58]
• Nanosecond timestamp support. Visible as nsdatetime instance.
• Minimal CLI when run as script: python3 -m rarfile
• Skip old file versions in versioned archive.

Cleanups:

• Use PBKDF2 implementation from hashlib.
• Improve test coverage.

Fixes:

• Add the .sfx test files to MANIFEST.in for inclusion in pypi tarball. [#60]
• Add all files in git to tarball.

New features:

• Support unar as decompression backend. It has much better support for RAR features than bsdtar. [#36]
• Support SFX archives - archive header is searched in first 2MB of the file. [#48]
• Add HACK_TMP_DIR option, to force temp files into specific directory. [#43]

Fixes:

• Always use "/" for path separator in command-line, gives better results on Windows.

Cleanups:

• Drop module-level options from docs, they create confusion. [#47]
• Drop support for Python 2 and 3.5 and earlier. Python 2 is dead and requiring Python 3.6 gives blake2s, stdlib that supports pathlib, and ordered dict without compat hacks.
• Replace PyCrypto with PyCryptodome in tests.
• Use Github Actions for CI.

This will be last version with support for Python 2.x

New feature:

• Accept pathlib objects as filenames. (Aleksey Popov)

• Accept bytes filenames in Python 3 (Nate Bogdanowicz)

Fixes:

• Use bug-compatible SHA1 for longer passwords (> 28 chars) in RAR3 encrypted headers. (Marko Kreen)

• Return true/false from _check_unrar_tool (miigotu)

• Include all test files in archive (Benedikt Morbach)

• Include volume number in NeedFirstVolume exception if available (rar5).

Cleanups:

• Convert tests to pytest.

New feature:

• Support RAR5 archive format. It is actually completely different archive format from RAR3 one, only is uses same file extension and tools are old one.

  Except incompatibilies noted below, most of code should notice no change, existing RarInfo fields will continue using RAR3-compatible values (eg. RarInfo.host_os). RAR5-specific values will use new fields.

Incompatibilities between rarfile v2.x and 3.x:

• Default PATH_SEP is now '/' instead '\'.

• Removed NEED_COMMENTS option, comments are always extracted.

• Removed UNICODE_COMMENTS option, they are always decoded.

• Removed USE_DATETIME option, RarInfo.date_time is always tuple, RarInfo.mtime, RarInfo.atime, RarInfo.ctime and RarInfo.arctime are always datetime.datetime objects.

Fixes:

• Fixed bug when calling rarfp.open() on a RarInfo structure.

Cleanups:

• Code refactor to allow 2 different file format parsers.

• Code cleanups to pass modern linters.

• New testing and linting setup based on Tox.

• Use setuptools instead distutils for install.

• Fix: support solid archives from in-memory file object. Full archive will be written out to temp file. [#21]

• Fix: ask unrar stop switches scanning, to handle archive names starting with "-". (Alexander Shadchin) [#12]

• Fix: add missing _parse_error variable to RarFile object. (Gregory Mazzola) [#20]

• Fix: return proper boolean from RarInfo.needs_password. [#22]

• Fix: do not insert non-string rarfile into exception string. (Tim Muller) [#23]

• Fix: make RarFile.extract and RarFile.testrar support in-memory archives.

• Use cryptography module as preferred crypto backend. PyCrypto will be used as fallback.

• Cleanup: remove compat code for Python 2.4/2.5/2.6.

• Allow use of bsdtar as decompression backend. It sits on top of libarchive, which has support for reading RAR archives.

  Limitations of libarchive RAR backend:

  • Does not support solid archives.

  • Does not support password-protected archives.

  • Does not support "parsing filters" used for audio/image/executable data, so few non-solid, non-encrypted archives also fail.

  Now rarfile checks if unrar and if not then tries bsdtar. If that works, then keeps using it. If not then configuration stays with unrar which will then appear in error messages.

• Both RarFile and is_rarfile now accept file-like object. Eg. io.BytesIO. Only requirement is that the object must be seekable. This mirrors similar funtionality in zipfile.

  Based on patch by Chase Zhang.

• Uniform error handling. RarFile accepts errors="strict" argument.

  Allow user to tune whether parsing and missing file errors will raise exception. If error is not raised, the error string can be queried with RarFile.strerror method.

• Add context manager support for RarFile class. Both RarFile and RarExtFile support with statement now. (Wentao Han)

• RarFile.volumelist method, returns filenames of archive volumes.

• Re-throw clearer error in case unrar is not found in PATH.

• Sync new unrar4.x error code from rar.txt.

• Use Sphinx for documentation, push docs to rtfd.org

Fixes:

• RarExtFile.read and RarExtFile.readinto now do looping read to work properly on short reads. Important for Python 3.2+ where read from pipe can return short result even on blocking file descriptor.

• Proper error reporting in RarFile.extract, RarFile.extractall and RarFile.testrar.

• RarExtFile.read from unrar pipe: prefer to return unrar error code, if thats not available, do own error checks.

• Avoid string addition in RarExtFile.read, instead use always list+join to merge multi-part reads.

• dumprar: dont re-encode byte strings (Python 2.x). This avoids unneccessary failure when printing invalid unicode.

Fixes:

• USE_DATETIME: survive bad values from RAR

• Fix bug in corrupt unicode filename handling

• dumprar: make unicode chars work with both pipe and console

Features:

• Support .seek() method on file streams. (Kristian Larsson)

• Support .readinto() method on file streams. Optimized implementation is available on Python 2.6+ where memoryview is available.

• Support file comments - RarInfo.comment contains decompressed data if available.

• File objects returned by RarFile.open() are io.RawIOBase-compatible. They can further wrapped with io.BufferedReader and io.TextIOWrapper.

• Now .getinfo() uses dict lookup instead of sequential scan when searching archive entry. This speeds up prococessing for archives that have many entries.

• Option UNICODE_COMMENTS to decode both archive and file comments to unicode. It uses TRY_ENCODINGS for list of encodings to try. If off, comments are left as byte strings. Default: 0

• Option PATH_SEP to change path separator. Default: r'\', set rarfile.PATH_SEP='/' to be compatibe with zipfile.

• Option USE_DATETIME to convert timestamps to datetime objects. Default: 0, timestamps are tuples.

• Option TRY_ENCODINGS to allow tuning attempted encoding list.

• Reorder RarInfo fiels to better show zipfile-compatible fields.

• Standard regtests to make sure various features work

Compatibility:

• Drop RarInfo.unicode_filename, plain RarInfo.filename is already unicode since 2.0.

• .read(-1) reads now until EOF. Previously it returned empty buffer.

Fixes:

• Make encrypted headers work with Python 3.x bytes() and with old 2.x 'sha' module.

• Simplify subprocess.Popen usage when launching unrar. Previously it tried to optimize and work around OS/Python bugs, but this is not maintainable.

• Use temp rar file hack on multi-volume archives too.

• Always .wait() on unrar, to avoid zombies

• Convert struct.error to BadRarFile

• Plug some fd leaks. Affected: Jython, PyPy.

• Broken archives are handled more robustly.

Fixes:

• Relaxed volume naming. Now it just calculates new volume name by finding number in old one and increasing it, without any expectations what that number should be.

• Files with 4G of compressed data in one colume were handled wrong. Fix.

• DOS timestamp seconds need to be multiplied with 2.

• Correct EXTTIME parsing.

Cleanups:

• Compressed size is per-volume, sum them together, so that user sees complete compressed size for files split over several volumes.

• dumprar: Show unknown bits.

• Use struct.Struct to cache unpack formats.

• Support missing os.devnull. (Python 2.3)

Features:

• Minimal implmentation for RarFile.extract, RarFile.extractall, RarFile.testrar. They are simple shortcuts to unrar invocation.

• Accept RarInfo object where filename is expected.

• Include dumprar.py in .tgz. It can be used to visualize RAR structure and test module.

• Support for encrypted file headers.

Fixes:

• Don't read past ENDARC, there could be non-RAR data there.

• RAR 2.x: It does not write ENDARC, but our volume code expected it. Fix that.

• RAR 2.x: Support more than 200 old-style volumes.

Cleanups:

• Load comment only when requested.

• Cleanup of internal config variables. They should have now final names.

• RarFile.open: Add mode=r argument to match zipfile.

• Doc and comments cleanup, minimize duplication.

• Common wrappers for both compressed and uncompressed files, now RarFile.open also does CRC-checking.

Features:

• Python 3 support. Still works with 2.x.

• Parses extended time fields. (.mtime, .ctime, .atime)

• RarFile.open method. This makes possible to process large entries that do not fit into memory.

• Supports password-protected archives.

• Supports archive comments.

Cleanups:

• Uses subprocess module to launch unrar.

• .filename is always Unicode string, .unicode_filename is now deprecated.

• .CRC is unsigned again, as python3 crc32() is unsigned.

• First release.