pex

This release brings support for creating PEXes that target Android. The Pip 25.1 upgrade in Pex 2.34.0 brought support for resolving Android platform-specific wheels and this releases upgrade of Pex's vendored packaging to 25.0 brings support for Pex properly dealing with those Android platform-specific wheels when packaging PEXes and when booting up from a PEX.

• Upgrade to latest packaging for Python 3.8+. (#2748)

This release adds support for the --resume-retries option available in Pip 25.1. If you configure Pex to use Pip 25.1 or newer, it will now try to resume incomplete downloads 3 times by default.

• Add support for --resume-retries in Pip 25.1. (#2746)

This release adds support for --pip-version 25.1 as well as --pip-version latest-compatible. The latest-compatible version will be the latest --pip-version supported by Pex compatible with the current interpreter running Pex.

• Add support for --pip-version 25.1. (#2744)

This release follows up on the PEX scie argv0 fix in #2738 to further ensure the argv0 of a PEX scie is the absolute path of the scie. In addition, a regression for PEX scies with no entry point is fixed, allowing such PEX scies to be used as --python targets in Pex invocations.

• Fix PEX scie argv0 to be the scie absolute path. (#2741)
• Fix entrypoint-less PEX scies used as --python. (#2742)

Fix argv0 in PEX scies to point to the scie itself instead of the unpacked PEX in the nce cache.

• Fix argv0 in PEX scies to point to the scie itself. (#2738)

This release only upgrades the Pex PEX scies from Python 3.13.1 to 3.13.3.

The main thrust of the release is to kick the tires on Pex's new build system which is powered by uv + dev-cmd and make sure all the action machinery is still working properly.

This release fixes PEX_TOOLS=1 ./path/to/pex for PEXes using venv-execution and sh-bootstrapping (that is, built with --sh-boot --venv=... --include-tools ). Previously, the PEX_TOOLS=1 was ignored if the venv already existed in the PEX_ROOT (for instance, if the PEX had already been run).

• Avoid fast-path in --sh-boot script when PEX_TOOLS=1. (#2726)

Fix PEP-723 script metadata parsing to skip metadata blocks found in multiline strings.

• Fix PEP-723 script metadata parsing. (#2722)

This release fixes rate limit issues building CPython Pex scies by bumping to science 0.12.2 which is fixed to properly support bearer authentication via the SCIENCE_AUTH_<normalized_host>_BEARER environment variable.

• Upgrade to science 0.12.2 to fix PBS rate limits. (#2720)

This release fixes PEX scies to exclude a ptex binary for --scie eager scies saving ~5MB on scies targeting 64 bit systems.

• Do not include ptex in --scie eager scies. (#2717)

This release fixes Pex Zip64 support such that PEX zips do not use Zip64 extensions unless needed. Previously, any zip between ~2GB and ~4GB that actually fell under Zip64 limits would still use Zip64 extensions. This prevented the file from being bootable under Python before the 3.13 release since the zipimporter was not fixed to support ZIp64 extensions until then.

The --scie-only option is fixed for the case when the -o / --output-file name does not end in .pex. Previously there would be no scie (or PEX) output at all!

Finally, this release fixes PEX scies such that, when split, the embedded PEX is both executable and retains the expected name as provided by -o / --output-file.

• Enable true Zip64 support. (#2714)
• Fix --scie-only for -o not ending in .pex. (#2715)
• Fix PEX scie contents when split. (#2713)

This release fixes PEXes build with root requirements like foo[bar] foo[baz] (vs. foo[bar,baz], which worked already).

• Fix dup requirement extra merging during PEX boot. (#2707)

This release fixes a bug in both pex3 lock subset and pex3 lock {create,sync,update} --elide-unused-requires-dist for --style universal locks whose locked requirements have dependencies de-selected by the following environment markers:

• os_name
• platform_system
• sys_platform
• python_version
• python_full_version

The first three could lead to errors when the universal lock was generated with --target-systems and the last two could lead to errors when the universal lock was generated with --interpreter-constraint.

• Fix pex3 lock subset. (#2684)

This release adds support for Pip 25.0.1.

• Add support for --pip-version 25.0.1. (#2671)

This release fixes a long-standing bug handling development versions of CPython (any non-tagged release of the interpreter). These interpreters report a full version of X.Y.Z+ and the trailing + leads to a non PEP-440 compliant version number. This, in turn, causes issues with the packaging library leading to failures to evaluate markers for these interpreters which surface as inscrutable Pex errors.

• Fix support for CPython development releases. (#2655)

This release adds support for Pip 25.0.

• Add support for --pip-version 25.0. (#2652)

This release adds pex3 lock subset <reqs...> --lock existing.lock for creating a subset of an existing lock file. This is a fast operation that just trims un-used locked requirements from the lock but otherwise leaves the lock unchanged.

• Add support for pex3 lock subset. (#2647)

This release brings --sh-boot support to PEXes with --layout {loose,packed}. Previously, the --sh-boot option only took effect for traditional PEX zip files. Now all PEX output and runtime schemes, in any combination, can benefit from the reduced boot latency --sh-boot brings on all runs of a PEX after the first.

• Support --sh-boot for --layout {loose,packed}. (#2645)

This release brings 1st class support for newer Pip's --keyring-provider option. Previously you could only use keyring based authentication via --use-pip-config and either the PIP_KEYRING_PROVIDER environment variable or Pip config files. Although using --keyring-provider import is generally unusable in the face of Pex hermeticity strictures, --keyring-provider subprocess is viable; just ensure you have a keyring provider on the PATH. You can read more here.

This release also brings PEP-723 support to Pex locks. You can now pass pex3 lock {create,sync,update} --exe <script> ... to include the PEP-723 declared script requirements in the lock.

• add --keyring-provider flag to configure keyring-based authentication (#2592)
• Support locking PEP-723 requirements. (#2642)

This release upgrades science for use in building PEX scies with --scie {eager,lazy}. The upgraded science fixes issues dealing handling failed Python distribution downloads and should now be more robust and clear when downloads fail.

• Upgrade science minimum requirement to 0.10.1. (#2637)

This release adds Pex --scie {eager,lazy} support for Linux ppc64le and s390x.

• Add --scie support for Linux ppc64le and s390x. (#2635)

This release fixes a bug in PEX_ROOT handling that could manifest with symlinked HOME dirs or more generally symlinked dirs being parents of the PEX_ROOT. Although this was claimed to be fixed in the Pex 2.20.4 release by #2574, there was one missing case not handled.

• Ensure that the PEX_ROOT is always a realpath. (#2626)

This release adds a Pex PEX scie for armv7l.

• Add a Pex PEX scie for armv7l. (#2624)

This release adds Pex --scie {eager,lazy} support for Linux armv7l.

In addition, a spurious warning when using PEX_PYTHON=pythonX.Y against a venv PEX has been fixed.

• Added support for armv7l (#2620)
• Fix incorrect regex for PEX_PYTHON precision warning (#2622)

This release fixes the --elide-unused-requires-dist lock option once again. The fix in 2.25.1 could lead to locked requirements having only a partial graph of extras which would allow a subsequent subset of those partial extras to silently resolve an incomplete set of dependencies.

In addition, the Pex REPL for PEXes without entry points or else when forced with PEX_INTERPRETER=1 is now fixed such that readline support always works. Previously, the yellow foreground color applied to the PS1 and PS2 prompts would interfere with the tracked cursor position in some Pythons; so the yellow foreground color for those prompts is now dropped.

• Fix --elide-unused-requires-dist: don't expose partial extras. (#2618)
• Fix Pex REPL prompt. (#2617)

This is a hotfix release that fixes a bug in the implementation of the --elide-unused-requires-dist lock option introduced in Pex 2.25.0.

• Fix --elide-unused-requires-dist for unactivated deps. (#2615)

This release adds support for pex3 lock {create,sync} --elide-unused-requires-dist. This new lock option causes any dependencies of a locked requirement that can never be activated to be elided from the lock file. This leads to no material difference in lock file use, but it does cut down on the lock file size.

• Add --elide-unused-requires-dist lock option. (#2613)

This release fixes a long-standing bug in resolve checking. Previously, only resolve dependency chains where checked, but not the resolved distributions that satisfied the input root requirements.

In addition, the 2.24.2 release included a wheel with no compression (~11MB instead of ~3.5MB). The Pex wheel is now fixed to be compressed.

• Fix resolve check to cover dists satisfying root reqs. (#2610)
• Fix build process to produce a compressed .whl. (#2609)

This release fixes a long-standing bug in "YOLO-mode" foreign platform speculative wheel builds. Previously if the speculatively built wheel had tags that did not match the foreign platform, the process errored pre-emptively. This was correct for complete foreign platforms, where all tag information is known, but not for all cases of abbreviated platforms, where the failure was overly aggressive in some cases. Now foreign abbreviated platform speculative builds are only rejected when there is enough information to be sure the speculatively built wheel definitely cannot work on the foreign abbreviated platform.

• Accept more foreign --platform "YOLO-mode" wheels. (#2607)

This release fixes pex3 cache prune handling of cached Pips. Previously, performing a pex3 cache prune would bump the last access time of all un-pruned cached Pips artificially. If you ran pex3 cache prune in a daily or weekly cron job, this would mean Pips would never be pruned.

• Fix pex3 cache prune handling of cached Pips. (#2589)

This release adds pex3 cache prune as a likely more useful Pex cache management command than the existing pex3 cache purge. By default pex3 cache prune prunes any cached items not used for the last 2 weeks and is likely suitable for use as a daily cron job to keep Pex cache sizes down. The default age of 2 weeks can be overridden by specifying --older-than "1 week" or --last-access-before 14/3/2024, etc. See pex3 cache prune --help for more details.

• Support pex3 cache prune --older-than .... (#2586)

This release adds support for drawing requirements from PEP-735 dependency groups when creating PEXes or lock files. Groups are requested via --group <name>@<project dir> or just --group <name> if the project directory is the current working directory.

• Add support for PEP-735 dependency groups. (#2584)

This release adds support for --pip-version 24.3.1.

• Add support for --pip-version 24.3.1. (#2582)

This release adds support for --pip-version 24.3.

• Add support for --pip-version 24.3. (#2580)

This release carries several bug fixes and a performance improvement for lock deletes.

Although there were no direct reports in the wild, @iritkatriel noticed by inspection the Pex safe_mkdir utility function would mask any OSError besides EEXIST. This is now fixed.

It was observed by @b-x that when PEX_ROOT was contained in a symlinked path, PEXes would fail to execute. The most likely case leading to this would be a symlinked HOME dir. This is now fixed.

This release also fixes a bug where --pip-log <path>, used multiple times in a row against the same file could lead to pex3 lock errors. Now the specified path is always truncated before use and a note has been added to the option --help that using the same --pip-log path in concurrent Pex runs is not supported.

In addition, pex3 lock {update,sync} is now optimized for the cases where all the required updates are deletes. In this case neither Pip nor the network are consulted leading to speed improvements proportional to the size of the resolve.

• Fix safe_mkdir swallowing non-EEXIST errors. (#2575)
• Fix PEX_ROOT handling for symlinked paths. (#2574)
• Fix --pip-log re-use. (#2570)
• Optimize pure delete lock updates. (#2568)

This release fixes both PEX building and lock creation via pex3 lock {create,sync} to be reproducible in more cases. Previously, if a requirement only available in source form (an sdist, a local project or a VCS requirement) had a build that was not reproducible due to either file timestamps (where the SOURCE_DATE_EPOCH standard was respected) or random iteration order (e.g.: the setup.py used sets in certain in-opportune ways), Pex's outputs would mirror the problematic requirement's non-reproducibility. Now Pex plumbs a fixed SOURCE_DATE_EPOCH and PYTHONHASHSEED to all places sources are built.

• Plumb reproducible build env vars more thoroughly. (#2554)

This release fixes an old bug handling certain sdist zips under Python 2.7 as well missing support for Python 3.13's PYTHON_COLORS env var.

• Fix Zip extraction UTF-8 handling for Python 2.7. (#2546)
• Add repl support for PYTHON_COLORS. (#2545)

This release fixes Pex --interpreter-constraint handling such that any supplied interpreter constraints which are in principle unsatisfiable either raise an error or else cause a warning to be issued when other viable interpreter constraints have also been specified. For example, --interpreter-constraint ==3.11.*,==3.12.* now errors and --interpreter-constraint '>=3.8,<3.8' --interpreter-constraint ==3.9.* now warns, culling >3.8,<3.8 and continuing using only ==3.9.*.

• Pre-emptively cull unsatisfiable interpreter constraints. (#2542)

This release adds the --pip-log alias for the existing --preserve-pip-download-log option as well as the ability to specify the log file path. So, to debug a resolve, you can now specify --pip-log log.txt and Pex will deposit the Pip resolve log to log.txt in the current directory for easy tailing or post-resolve inspection. In addition, the log file itself is more useful in some cases. When you specify any abbreviated --platform targets, those targets calculated wheel compatibility tags are included in the Pip log. Also, when multiple targets are specified, their log outputs are now merged at the end of the resolve in a serialized fashion with prefixes on each log line indicating which target the log line corresponds to.

In addition, a race in Pex's PEP-517 implementation that could (rarely) lead to spurious metadata generation errors or sdist creation errors is fixed.

• Fix intermittent PEP-517 failures. (#2540)
• Plumb --pip-version to Platform tag calculation. (#2538)
• Add the ability to specify the --pip-log path. (#2536)

This release fixes a regression introduced by #2512 in the 2.19.0 release when building PEXes using abbreviated --platform targets. Instead of failing certain builds that used to succeed, Pex now warns that the resulting PEX may fail at runtime and that --complete-platform should be used instead.

• Only warn when --platform resolves fail tag checks. (#2533)

This release adds support for a new --pre-resolved-dists resolver as an alternative to the existing Pip resolver, --lock resolver and --pex-repository resolvers. Using --pre-resolved-dists dists/dir/ behaves much like --no-pypi --find-links dists/dir/ except that it is roughly 3x faster.

• Support --pre-resolved-dists resolver. (#2512)

This release fixes --scie-name-style platform-parent-dir introduced in #2523. Previously the target platform name also leaked into scies targeting foreign platforms despite using this option.

• Fix --scie-name-style platform-parent-dir. (#2526)

This release adds support for pex3 cache {dir,info,purge} for inspecting and managing the Pex cache. Notably, the pex3 cache purge command is safe in the face of concurrent PEX runs, waiting for in flight PEX runs to complete and blocking new runs from starting once the purge is in progress. N.B.: when using pex3 cache purge it is best to install Pex with the 'management' extra; e.g.: pip install pex[management]. Alternatively, one of the new Pex scie binary releases can be used.

In order to release a Pex binary that can support the new pex3 cache management commands first class, a set of enhancements to project locking and scie generation were added. When using --project you can now specify extras; e.g.: --project ./the/project-dir[extra1,extra2]. When creating a Pex scie, you can now better control the output files using --scie-only to ensure no PEX file is emitted and --scie-name-style to control how the scie target platform name is mixed into the scie output file name. Additionally, you can request one or more shasum-compatible checksum files be emitted for each scie with --scie-hash-alg.

On the locking front, an obscure bug locking project releases that contain artifacts that mis-report their version number via their file name has been fixed.

Finally, the vendored Pip has had its own vendored CA cert bundle upgraded from that in certifi 2024.7.4 to that in certifi 2024.8.30.

• Fix locking of sdists rejected by Pip. (#2524)
• Add --scie-only & --scie-name-style. (#2523)
• Support --project extras. (#2522)
• Support shasum file gen via --scie-hash-alg. (#2520)
• Update vendored Pip's CA cert bundle. (#2517)
• Introduce pex3 cache {dir,info,purge}. (#2513)

This release brings support for overriding the versions of setuptools and wheel Pex bootstraps for non-vendored Pip versions (the modern ones you select with --pip-version) using the existing --extra-pip-requirement option introduced in the 2.10.0 release.

• Support custom setuptools & wheel versions. (#2514)

This release brings a slew of small fixes across the code base.

When creating locks for foreign platforms, pex3 lock {create,update,sync} now allows locking sdists that use PEP-517 build backends that do not support the prepare_metadata_for_build_wheel hook and whose product is a wheel not compatible with the foreign platform. This is decidedly a corner case, but one encountered with the mesonpy build backend which seems to have traction in the scientific computing world in particular.

The recent re-vamp of the PEX REPL is now fixed to respect common conventions for controlling terminal output via the NO_COLOR, FORCE_COLOR and TERM environment variables.

The examples in the buildingpex docs had bit-rotted. They have been refreshed and now all work.

Finally, both the Pex CLI and PEX files support the ambient OS standards for user cache directories. Instead of using ~/.pex as the default PEX_ROOT cache location, the default is now ~/.cache/pex on Linux ( but respecting XDG_CACHE_HOME when set) and ~/Library/Caches/pex on Mac.

• Lock sdists in more cases for foreign platforms. (#2508)
• Respect NO_COLOR, FORCE_COLOR & TERM=dumb. (#2507)
• Fix buildingpex.rst examples. (#2506)
• Respect OS user cache location conventions. (#2505)

This release fixes the PEX repl for [Python Standalone Builds][PBS] Linux CPython PEX scies. These PEXes ship using a version of libedit for readline support that does not support naive use of ansi terminal escape sequences for prompt colorization.

• Fix PEX repl prompt for Linux PBS libedit. (#2503)

This release adds support for --venv-system-site-packages when creating a --venv PEX and --system-site-packages when creating a venv using the pex-tools / PEX_TOOLS=1 venv command or when using the pex3 venv create command. Although this breaks PEX hermeticity, it can be the most efficient way to ship partial PEX venvs created with --excludes to machines that have the excluded dependencies already installed in the site packages of a compatible system interpreter.

• Support --system-site-packages when creating venvs. (#2500)

This release enhances the REPL your PEX drops into when it either doesn't have an entry point or you force interpreter mode with the PEX_INTERPRETER environment variable. There is now clear indication you are running in a PEX hermetic environment and a pex command added to the REPL that you can use to find out more details about the current PEX environment.

• Add PEX info to the PEX repl. (#2496)

This release fixes --inject-env when used in combination with a --scie-busybox so that the injected environment variable can be overridden at runtime like it can form a traditional PEX.

In addition, running a PEX with the Python interpreter -i flag or PYTHONINSPECT=x in the environment causes the PEX to enter the Python REPL after evaluating the entry point, if any.

• Allow --inject-env overrides for --scie-busybox. (#2490)
• Fix PEXes for -i / PYTHONINSPECT=x. (#2491)

This release brings support for creating PEX scies for PEXes targeting PyPy. In addition, for PEX scies targeting CPython, you can now specify --scie-pbs-stripped to select a stripped version of the [Python Standalone Builds][PBS] CPython distribution embedded in your scie to save transfer bandwidth and disk space at the cost of losing Python debug symbols.

Finally, support is added for --scie-busybox to turn your PEX into a multi-entrypoint BusyBox-like scie. This support is documented in depth at https://docs.pex-tool.org/scie.html

• Support --scie for PyPy & support stripped CPython. (#2488)
• Add support for --scie-busybox. (#2468)

This release fixes the --scie option to support building a Pex PEX scie with something like pex pex -c pex --venv --scie eager -o pex. Previously, due to the output filename of pex colliding with fixed internal scie lift manifest file names, this would fail.

• Handle all output file names when building scies. (#2484)

This release improves error message detail when there are failures in Pex sub-processes. In particular, errors that occur in pip download when building a PEX or creating a lock file now give more clear indication of what went wrong.

Additionally, this release adds support for --pip-version 24.2.

• Add more context for Job errors. (#2479)
• Add support for --pip-version 24.2. (#2481)

This release refreshes the root CA cert bundle used by --pip-version vendored (which is the default Pip Pex uses for Python <3.12) from certifi 2019.9.11's cacert.pem to certifi 2024.7.4's cacert.pem. This refresh addresses at least CVE-2023-37920 and was spearheaded by a contribution from Nash Kaminski in https://github.com/pex-tool/pip/pull/12. Thank you, Nash!

• Update vendored Pip's CA cert bundle. (#2476)

This release adds support for passing --site-packages-copies to both pex3 venv create ... and PEX_TOOLS=1 ./my.pex venv .... This is similar to pex --venv --venv-site-packages-copies ... except that instead of preferring hard links, a copy is always performed. This is useful to disassociate venvs you create using Pex from Pex's underlying PEX_ROOT cache.

This release also adds partial support for statically linked CPython. If the statically linked CPython is <3.12, the default Pip ( --pip-version vendored) used by Pex will work. All newer Pips will not though, until Pip 24.2 is released with the fix in https://github.com/pypa/pip/pull/12716 and Pex releases with support for --pip-version 24.2.

• Add --site-packages-copies for external venvs. (#2470)
• Support statically linked CPython. (#2472)

This release adds support for creating native PEX executables that contain their own hermetic CPython interpreter courtesy of Python Standalone Builds and the Science project.

You can now specify --scie {eager,lazy} when building a PEX file and one or more native executable PEX scies will be produced (one for each platform the PEX supports). These PEX scies are single file executables that look and behave like traditional PEXes, but unlike PEXes they can run on a machine with no Python interpreter available.

• Add --scie option to produce native PEX exes. (#2466)

This release fixes a long-standing bug in Pex parsing of editable requirements. This bug caused PEXes containing local editable project requirements to fail to import those local editable projects despite the fact the PEX itself contained them.

• Fix editable requirement parsing. (#2464)

This release adds support for injecting requirements into the isolated Pip PEXes Pex uses to resolve distributions. The motivating use case for this is to use the feature Pip 23.1 introduced for forcing --keyring-provider import.

Pex already supported using a combination of the following to force non-interactive use of the keyring:

1. A keyring script installation that was on the PATH
2. A --pip-version 23.1 or newer.
3. Specifying --use-pip-config to pass --keyring-provider subprocess to Pip.

You could not force --keyring-provider import though, since the Pips Pex uses are themselves hermetic PEXes without access to extra installed keyring requirements elsewhere on the system. With --extra-pip-requirement you can now do this with the primary benefit over --keyring-provider subprocess being that you do not need to add the username to index URLs. This is ultimately because the keyring CLI requires username whereas the API does not; but see https://pip.pypa.io/en/stable/topics/authentication/#keyring-support for more information.

• Add support for --extra-pip-requirement. (#2461)

This release adds support for Pip 24.1.2.

• Add support for --pip-version 24.1.2. (#2459)