Repository navigation
Release 44.1.1.post3: fix CVE-2026-59890 (Python 2.7 line) - #6
Conversation
…patterns GHSA-h35f-9h28-mq5c. FileList matched MANIFEST.in patterns against on-disk names byte-for-byte with no Unicode normalization. On macOS APFS/HFS+, a file stored NFD and a pattern authored NFC denote the same file but differ byte-for-byte, so an exclude/global-exclude/recursive-exclude/prune rule could silently fail to drop a non-ASCII-named file, publishing it in the sdist despite the rule. Normalize both the pattern and the candidate path to NFC before matching, via a new unicode_utils.normalize() helper and a _NormalizedMatcher wrapper around the compiled pattern returned by translate_pattern(). All five FileList callers (exclude, recursive_exclude, prune, global_include, global_exclude) only call .match() on the result, which the wrapper keeps. Upstream: pypa/setuptools dd9f436 (v83.0.0); already shipped on our 82.0.1 line as v82.0.1.post1. Py2.7 adaptation: on Python 2, MANIFEST.in lines and walked file names are native byte strings, so normalize() follows the existing decompose() helper: text is normalized directly; bytes are decoded as UTF-8, normalized and re-encoded, and left unchanged if they are not UTF-8. Upstream normalized str only, which on Python 2 would have made the fix a no-op. No f-strings or type hints; _NormalizedMatcher is a new-style class; test literals use \u escapes. Tests: translate_pattern NFC<->NFD for text and (Py2) bytes, plus an end-to-end egg_info global-exclude of an NFD-named file. All three fail on the pristine base and pass with the fix under Python 2.7.18; test_egg_info/test_sdist/test_manifest: 145 passed (pristine 142 + 3 new). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bump version 44.1.1+security.2 -> 44.1.1.post3 and add a changelog entry for the backported MANIFEST.in Unicode-normalization fix. New security releases use the PEP 440 .postN scheme in place of the +security.N local label; N continues the line's count (44.1.1.1, 44.1.1+security.2, now 44.1.1.post3). 44.1.1.post3 sorts after 44.1.1+security.2 and before 44.1.2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ezequielp-activestate
left a comment
There was a problem hiding this comment.
0 blockers. Approving, but please fix the tag_build problem below before tagging v44.1.1.post3.
The fix is a faithful port of upstream dd9f436a (shipped in v83.0.0). Upstream history from v83.0.0 to v84.2.1 has no follow-up that fixes or hardens it; the later commits touching egg_info.py, unicode_utils.py and test_manifest.py only change lint and typing. The Python 2 change, which decodes byte strings as UTF-8 before normalizing, is needed: upstream's isinstance(text, str) check would make the fix a no-op on Python 2.
🔴 Blockers: None
🟡 Should fix
- Building from the source tree produces an invalid version.
setup.cfg:1-3still has[egg_info] tag_build = .post/tag_date = 1. Withversion = 44.1.1.post3, apip wheel .orpip install .from the checkout or the tag givesVersion: 44.1.1.post3.post-20261022, whichpackaging.Versionrejects because it has two post segments. On the base branch the equivalent,44.1.1+security.2.post20261022, is valid.setuptools/tests/test_integration.py::test_stevedorepasses on base and fails on head: stevedore raisesInvalidVersion: '44.1.1.post3.post-2026…'.- Recent pip refuses wheels with invalid versions, so
pip install git+…@v44.1.1.post3would break the same way. Artifacts built through thereleasealias (egg_info -Db '') are fine. - Fix: clear
tag_buildandtag_dateon this branch.
- The version-ordering note leaves out
v44.1.1.1. The 2023 GitHub release is namedv44.1.1.1, and under PEP 44044.1.1.1sorts above44.1.1.post3. That tag's ownsetup.cfgsaysversion = 44.1.1, so this only matters if some index or catalog lists that build as44.1.1.1. If one does,setuptools<45would resolve to a build without CVE-2024-6345, CVE-2025-47273 or this fix. Worth confirming, or use a number above it such as44.1.1.3. - On Python 2, an exclusion still leaks when the pattern and the path are different string types. This happens on base and head alike, for NFC against NFC too, so it isn't caused by this PR. With a byte-string
MANIFEST.inpattern and aunicodeentry inFileList.files(or the reverse),global_excludeleaves the file in. That can happen when a file-finder plugin returns text paths for non-ASCII names, and it causes the same harm the CVE describes. Cheap fix: in_NormalizedMatcher.match/.search(setuptools/command/egg_info.py:50-54), convert the path to the pattern's type (UTF-8 encode or decode) before matching.
⚪ Nits
setuptools/command/egg_info.py:35: only one blank line beforeclass _NormalizedMatcher(flake8 E302).- The
f5acb4f1commit message says the test literals use\uescapes, buttest_manifest.py:163-164uses a literalé. It works because of the coding header; only the message is wrong. include,recursive-include,graftandbuild_py.exclude_data_filesstill match without normalization, the same as upstream. The include cases fail safe;exclude_package_datais the same kind of bug but outside this advisory.
📋 Checked
- Line by line against upstream:
_NormalizedMatcher, the normalization intranslate_pattern, and both new tests. The only differences are Python 2 adaptations. - Every
translate_patterncaller:exclude,recursive_exclude,prune,global_includeandglobal_excludeonly test the match, which runs on a normalized copy, soFileList.filesis never modified. 44.1.1 has no vendored copy of this code. - Reproduction: an NFC pattern against an NFD path. On base the file stays in the list for
exclude,global_exclude,recursive_excludeandprune; on head all four remove it. A path that isn't valid UTF-8 doesn't crash and behaves as before. - Tests on Python 2.7.18: all 3 new tests fail on base and pass on head.
test_manifest,test_egg_infoandtest_sdistgive 145 passed on head. In the fullsetuptools+pkg_resourcessuites, base and head share the same 5 failures, all from the container; the only failure new on head istest_stevedore, from the first should-fix. setup.cfgand.bumpversion.cfgagree, theCHANGES.rstentry cites the CVE and GHSA, and the changed code has no Python 3-only syntax.
Review follow-up. On Python 2 a MANIFEST.in pattern and a FileList entry can be of different string types, for example when a file-finder plugin returns text paths for non-ASCII names. A byte pattern never matches a non-ASCII text path or vice versa, so an exclusion leaked the file: the same harm the CVE describes. This happened on the base too, NFC against NFC included. _NormalizedMatcher now brings the normalized path to the pattern's string type (UTF-8 encode or decode; left unchanged if that fails) before matching. New tests cover byte pattern against text path and the reverse, NFC and NFD, through translate_pattern() and FileList.global_exclude(); both fail without this change. Also: add the second blank line before class _NormalizedMatcher (E302), and spell the test literals as é escapes, as the f5acb4f message already claimed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With version = 44.1.1.post3, the inherited [egg_info] tag_build = .post / tag_date = 1 made a build from the source tree or tag (pip install ., pip install git+...@v44.1.1.post3) report 44.1.1.post3.post-YYYYMMDD, which PEP 440 rejects (two post segments). That broke test_integration::test_stevedore and would make recent pip refuse the wheel. A source build now reports 44.1.1.post3. The release alias (egg_info -Db '') already cleared the tag, so it is unaffected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the review. Pushed 96796ea and d522234. 🟡 🟡 Version ordering vs 🟡 Mixed str/bytes on Python 2: fixed. ⚪ E302: fixed. |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ezequielp-activestate
left a comment
There was a problem hiding this comment.
Re-review of 8cfa412: should-fixes addressed. Approving.
| Earlier finding | Status | Evidence (Python 2.7) |
|---|---|---|
🟡 tag_build/tag_date produce an invalid version |
Fixed (d5222346) |
pip wheel . from the checkout now builds setuptools-44.1.1.post3-py2.py3-none-any.whl, a valid PEP 440 version. test_stevedore passes. |
| 🟡 str/bytes pattern–path mismatch on Python 2 | Fixed (96796eac) |
_NormalizedMatcher._prepare converts the path to the pattern's type, UTF-8 with a fallback on undecodable input. Both test_translate_pattern_mixed_string_types cases fail against the previous egg_info.py and pass here. |
🟡 44.1.1.1 ordering |
Documented (8cfa4127) |
See below. |
⚪ E302 before _NormalizedMatcher |
Fixed | |
⚪ literal é in the tests |
Fixed | Now é. |
Version ordering, to act on before release. The new CHANGES note says a build labelled 44.1.1.1 is listed in the ActiveState Platform inventory, and 44.1.1.post3 sorts below it under PEP 440. Keeping the team's .postN scheme is a reasonable call, but the note alone won't stop a resolver: setuptools<45 would still pick 44.1.1.1 if it's served next to this release. Someone needs to retire or relabel that inventory entry, or confirm the Platform's version ordering puts it below .post3, before this ships. I couldn't check the inventory myself.
📋 Checked: test_manifest.py and test_integration.py (manifest, normalization, mixed-type and stevedore tests): 73 passed.
⚪ Nit: three blank lines before test_translate_pattern_mixed_string_types in test_manifest.py (E303).
ActiveState security release setuptools 44.1.1.post3 (Python 2.7 line).
The vulnerability
FileListmatched MANIFEST.in patterns against on-disk names byte-for-byte. On macOS APFS/HFS+ a file stored NFD and a pattern authored NFC are the same file but different bytes. So anexclude/global-exclude/recursive-exclude/prunerule could silently miss a non-ASCII-named file, and the file ended up in the sdist.The fix
Port of upstream dd9f436, which we already shipped as 82.0.1.post1 (PR #5). Both the pattern and the candidate path are normalized to NFC through a new
unicode_utils.normalize()and a_NormalizedMatcherwrapper returned bytranslate_pattern().Py2.7 adaptation: on Python 2, MANIFEST.in lines and walked file names are byte strings. Upstream's helper normalizes
stronly, so on Py2 it would have done nothing.normalize()therefore follows the existingdecompose()helper: bytes are decoded as UTF-8, normalized and re-encoded, and left unchanged if they aren't UTF-8.Validation (Python 2.7.18)
translate_patternNFC↔NFD for text and bytes, plus an end-to-end egg_infoglobal-excludeof an NFD-named file. All three fail on the pristine base and pass with the fix.test_egg_info+test_sdist+test_manifest: 145 passed (pristine: 142, plus the 3 new tests).cannot import name 'sic'on the pristine base too (44.x predates 3.10's distutils shim). Thetranslate_patterntest goes from fail to pass there as well.Versioning moves to the
.postNscheme: 44.1.1.post3 sorts after 44.1.1+security.2 and before 44.1.2. Tagv44.1.1.post3will be created after merge.