Skip to content

fix: Preserve data when splitting Upstash chunks - #6916

Open
Arthur031221 wants to merge 1 commit into
ChatGPTNextWeb:mainfrom
Arthur031221:fix/upstash-lossless-chunks-s11
Open

Arthur031221 wants to merge 1 commit into
ChatGPTNextWeb:mainfrom
Arthur031221:fix/upstash-lossless-chunks-s11

Conversation

@Arthur031221

Copy link
Copy Markdown

Change Type

  • feat
  • fix
  • refactor
  • perf
  • style
  • test
  • docs
  • ci
  • chore
  • build

Description of Change

Upstash sync can lose spaces from backups because the splitter drops the space at each boundary and the reader joins chunks without a separator. Long text without spaces can also exceed the chunk limit. This change splits by byte length, keeping UTF-8 characters and byte-order-mark characters intact so uploads and downloads preserve the serialized backup.

Additional Information

Added regression coverage for exact Upstash round trips and UTF-8 chunk boundaries. Tested with yarn test:ci --runInBand test/format-chunks.test.ts test/upstash.test.ts.

Assisted by Claude/Codex.

@Arthur031221

Copy link
Copy Markdown
Author

Anything else you want changed? I can rebase if needed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant