Skip to content

fix(epub): report an unreadable spine item instead of losing the book - #2509

Open
Lukas (L4XB) wants to merge 2 commits into
microsoft:mainfrom
L4XB:fix/epub-unreadable-spine-item
Open

Lukas (L4XB) wants to merge 2 commits into
microsoft:mainfrom
L4XB:fix/epub-unreadable-spine-item

Conversation

@L4XB

@L4XB Lukas (L4XB) commented Sep 16, 2026 •

Copy link
Copy Markdown

The bug

EpubConverter.convert walks the spine with no error handling at all, so one chapter
zipfile cannot decompress takes the whole book with it.

Measured on a three-chapter EPUB whose second chapter has one flipped byte, so its CRC
fails:

result
before FileConversionException: EpubConverter threw BadZipFile with message: Bad CRC-32 for file 'OEBPS/c2.xhtml'
after **Title:** Book / CHAPTER ONE / This part could not be read: Bad CRC-32 for file 'OEBPS/c2.xhtml' / CHAPTER THREE

Chapters one and three were intact both times. Before, the reader got neither.

Same principle as #2508 for the ZIP converter, one converter over: a part that cannot be
read should cost that part, not the document.

The change

Read each spine item, then convert it:

try:
    payload = z.read(file)
except (RuntimeError, zipfile.BadZipFile, NotImplementedError, EOFError) as exc:
    markdown_content.append(f"This part could not be read: {exc}")
    continue

Reading first is what makes the failure attributable. z.open() raises immediately for an
encrypted entry, but a bad CRC only surfaces mid-read — which, with the old streaming call,
happened inside HtmlConverter.convert, where the EPUB converter has no idea which member
it came from. The four exception types cover the ways zipfile refuses a member: encrypted
without a password, bad CRC, a compression method this build does not carry, and a truncated
entry.

The loop also now tests membership against the zip_names set the method already built for
href resolution, instead of rebuilding namelist() once per spine item.

A readable book converts byte for byte as before — that is what the second test pins.

Tests

tests/test_epub.py (renamed from test_epub_converter.py in #2579), using the _build_epub helper already there:

  • test_unreadable_spine_item_does_not_cost_the_rest_of_the_book — three chapters, the
    middle one damaged. It first asserts plain zipfile really does raise BadZipFile on that
    member, so the fixture cannot quietly stop reproducing the bug, then asserts the surviving
    chapters are present and in spine order, the damaged body is absent, and the failure is
    named.
  • test_a_readable_book_carries_no_failure_note — the control, so the guard cannot start
    annotating healthy books.

The first fails on main.

packages/markitdown      1097 passed, 12 skipped   (after merging main)
black 23.7.0 --check     test_epub.py unchanged

The spine loop had no error handling, so one chapter zipfile cannot decompress
took the whole book with it. A bad CRC on chapter two returned a
FileConversionException and chapters one and three with it, even though both
were intact.

Read each item before converting it and report the ones that fail. Reading
first is what makes the failure attributable: z.open raises immediately for an
encrypted entry, but a bad CRC only surfaces mid-read, inside the HTML
converter's own call. The loop also now uses the zip_names set it already
built rather than rebuilding namelist() for every item.

A readable book converts byte for byte as before.
…pine-item

main renamed test_epub_converter.py to test_epub.py (microsoft#2579). The two
spine-item tests and their helper stay at the end of the archive href
section, and main's __main__ block replaces the old one.

@kokokoXUY XU (kokokoXUY) left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Measured locally on fb2264d against main 4cc9fa1, with a
three-chapter book whose middle chapter is damaged:

  • bad CRC on a stored chapter: main raises "BadZipFile: Bad CRC-32 for file 'OEBPS/c2.xhtml'"
    and the reader gets nothing at all; here the result is FIRST_BODY /
    "This part could not be read: Bad CRC-32 for file 'OEBPS/c2.xhtml'" / THIRD_BODY, in spine
    order, matching what the PR body claims.
  • encrypted chapter (flag bit 0): main raises "RuntimeError: File 'OEBPS/c2.xhtml' is
    encrypted, password required for extraction"; here the chapter is reported in place.
  • compression method 99, which is the shape an AES entry has: main raises
    "NotImplementedError: That compression method is not supported"; here it is reported.
  • no regressions found: a clean book, a spine item absent from the archive, a chapter whose
    bytes are random noise, and a zero-length chapter all produce identical output before and
    after.

Tests: packages/markitdown/tests/test_epub.py is 10 passed here and 8 passed on main; putting
the new test file on main leaves
test_unreadable_spine_item_does_not_cost_the_rest_of_the_book failing with
zipfile.BadZipFile: Bad CRC-32 (1 failed, 1 passed), so the new test is a real red test and its
reader-side precondition check does its job. The patch applies cleanly to current main and both
files are LF-only.

One gap, the same one that affects the ZIP converter in #2508: a corrupt DEFLATE chapter makes
zipfile raise zlib.error, which is not in the caught tuple (zlib.error inherits straight from
Exception, so zipfile.BadZipFile does not cover it). A chapter with one flipped byte inside its
deflated stream still costs the whole book here, exactly as on main: "Error -3 while
decompressing data: invalid code lengths set". Adding "import zlib" and zlib.error to the tuple
closes it - the same archive then converts with that chapter reported as "This part could not be
read: Error -3 while decompressing data: invalid code lengths set" and the other chapters intact
(measured by applying the change locally, then reverting it). Since #2508 carries the identical
tuple, it may be worth handling both in one pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants