Skip to content

Fix offset rewrite collision in raw VTU parsing - #1574

Open
barlowa124 wants to merge 1 commit into
nschloe:mainfrom
barlowa124:vtu-raw-offset-collision-1571
Open

barlowa124 wants to merge 1 commit into
nschloe:mainfrom
barlowa124:vtu-raw-offset-collision-1571

Conversation

@barlowa124

Copy link
Copy Markdown

Summary

Fixes #1571.

The file linked in the issue reads correctly on 5.3.5, so the reporter's
binascii.Error: Incorrect padding comes from a raw VTU with a different
block layout. I found the mechanism that produces exactly this failure
class.

_parse_raw_binary rewrites each format="appended" DataArray's offset
to its position in the re-encoded base64 stream, then looks the next
block's tag up with root.find(".//DataArray[@offset='{i}']"). When a
just-rewritten offset happens to equal a later array's original offset,
the find matches the wrong element again, bumping that array's offset a
second time while the later array is never rewritten at all. Its offset
then points at an arbitrary position in the base64 stream, so reads either
return the wrong array's data or hit Incorrect padding.

This snapshots the appended offsets into a map before rewriting anything,
so every array is located by its original offset. A small constructed file
where the connectivity block's new offset collides with the offsets
block's original offset reads back with connectivity and offsets swapped
before this change, correctly after.

Hardening in the same function, all matching its existing error contract.
Non-raw encodings now raise ReadError instead of being parsed as raw
binary, matching the reporter's expectation for every encoding other than
raw. The payload no longer loses its final byte when the binary happens
to end in 0x0a, because the block loop consumes data by declared size
and skips only trailing whitespace. rfind replaces find for the
closing tag so a payload containing that byte sequence can't truncate the
segment. Missing delimiters and a missing AppendedData element now raise
ReadError instead of IndexError or AssertionError, and the remaining
asserts became real checks so python -O can't strip them.

New tests cover the offset collision and the encoding check.
pytest tests/test_vtu.py gives 114 passed, including the three existing
raw-binary fixture reads.

AI-assisted (Devin).

_parse_raw_binary rewrote DataArray offsets while looking each block's
tag up by that offset. A rewritten offset could equal a later array's
original offset, leaving that array un-rewritten and pointing into the
middle of the base64 stream; reads then silently returned wrong data or
failed with binascii errors. Snapshot the appended offsets first.

Also: enforce encoding="raw", stop truncating the payload on a trailing
newline byte, find the closing tag from the end, and turn bare asserts
into ReadError.

Fixes nschloe#1571

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Meshio doesn't raise ReadError when attempting to read raw VTU format

1 participant