Skip to content

Avoid per-packet epoll registration when packets are already queued - #232

Open
daberkow wants to merge 1 commit into
mainfrom
rx-drain
Open

daberkow wants to merge 1 commit into
mainfrom
rx-drain

Conversation

@daberkow

@daberkow daberkow commented Nov 2, 2026 •

Copy link
Copy Markdown
  • A new recv() helper first tries a non-blocking dlpi::recv. When packets are already queued, which is the usual case under load, it returns one with no epoll registration and no tokio wakeup.
  • When the queue is empty, it waits with the existing dlpi::recv_async, exactly as SoftNPU does today. If that wakes up and finds nothing queued, it waits again.
  • tokio::task::consume_budget() lets other tasks run, since the fast path otherwise never yields to tokio.
  • The two receive loops call recv() instead of recv_async().

This PR is AI assisted, please check it :)

This change along with #231 and oxidecomputer/p4#281 helped simulated omicron perf:

Before After
512 MiB oxide disk import 19.2 s (26.7 MiB/s) 14.0–14.5 s (35–36.5 MiB/s)
Stream-head drops during the import 2,622 1,089–1,399
rx error log lines 0 0

All in reference of #230.

@matoushybl

Copy link
Copy Markdown

matoushybl#1 I made additional changes to improve perf. Came across the same softnpu performance problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants