Skip to content

feat(input): preserve accepted connections across reloads - #475

Merged
timtofan merged 7 commits into
masterfrom
atimofieiev/up-input-handoff
Nov 4, 2026
Merged

timtofan merged 7 commits into
masterfrom
atimofieiev/up-input-handoff

Conversation

@timtofan

@timtofan timtofan commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

Depends on #474

Why

A graceful reload inherits listeners but closes accepted sockets. Unread successful writes, partial lines, and incomplete frames can disappear even while parsed output drains. Persistent clients need their accepted input preserved through a compatible replacement.

Architecture

The ready-successor exchange negotiates input handoff version 1. After readiness and any service-manager ownership barrier, the predecessor transfers each accepted socket and a raw state pipe over a private Unix socket using SCM_RIGHTS. Parsed records stay with the old pipeline; the successor routes the unparsed tail under its new configuration.

Plaintext preserves partial lines. Stream input replays RESET/STATE names and predictors before buffered wire bytes. Streaming zstd journals the current frame in an unlinked file and skips previously consumed plaintext on replay. Bounded read-ahead starts before state generation and spills excess bytes to temporary storage.

Older binaries cannot restore accepted state, so both upgrade and rollback retain the original PID-only readiness and connection-close fallback until both processes support the same version. Transfer errors keep the live successor running and expose unparsed input at risk through counters and unsampled error logs.

Before / after

Before After
Reload transfers listening sockets Compatible binaries also transfer accepted sockets and unparsed state
Partial lines and frames can be discarded The successor consumes them before new socket bytes
A large snapshot can block socket consumption Successor read-ahead begins before serialization
Old parsed records drain in the old process The same drain ownership remains
Older binaries close accepted sockets The same compatibility fallback is explicit and observable

Usage

The existing SIGUSR2 path uses handoff automatically between compatible binaries. No producer switch is required. Reserve temporary-storage headroom for compressed-frame journals and read-ahead spill; monitor handoff success and failure counters. docs/input-reload.md documents the exchange and the local persistent-client harness. Crashes and ambiguous output writes retain TCP's existing uncertainty.

Validation

Go 1.27.0, vendored dependencies: build, vet, serialized full tests, and upstream-configured golangci-lint v2. Tests cover byte-cut boundaries, continuous zstd flushes, inherited state, read-ahead spill, and admission rejection. The external receiver harness and larger dictionary stress test remain separately invoked checks.

@timtofan
timtofan merged commit b99f909 into master Nov 4, 2026
4 checks passed
@timtofan
timtofan deleted the atimofieiev/up-input-handoff branch November 4, 2026 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants