Skip to content

linux: add experimental io_uring restrictions - #2386

Open
gvozdetsky wants to merge 2 commits into
containers:mainfrom
gvozdetsky:io-uring-restrictions
Open

gvozdetsky wants to merge 2 commits into
containers:mainfrom
gvozdetsky:io-uring-restrictions

Conversation

@gvozdetsky

@gvozdetsky gvozdetsky commented Oct 19, 2026 •

Copy link
Copy Markdown

seccomp can't filter io_uring operations: they are submitted through the ring, not as syscalls, so runtimes currently block io_uring entirely. Linux 7.0 added task-level io_uring BPF filters, which are inherited across fork and exec and can only be tightened, like seccomp.

This adds experimental support behind the run.oci.io_uring annotation, in the shape proposed for linux.ioUring in opencontainers/runtime-spec#1328 (proposal: opencontainers/runtime-spec#1323):

{
  "ops": ["IORING_OP_READ", "IORING_OP_WRITE", "IORING_OP_SOCKET"],
  "socketFamilies": ["AF_UNIX", "AF_INET", "AF_INET6"]
}
  • ops is an allowlist, every other operation is denied. It doesn't depend on the SQE flags.
  • socketFamilies, when present, limits IORING_OP_SOCKET to these families. It requires IORING_OP_SOCKET in ops, and an empty list is refused; to deny all sockets, leave IORING_OP_SOCKET out of ops.
  • The policy is validated, and kernel support checked, before the container is created or restored, and before crun exec.
  • It is applied to the container init and to every crun exec process (they don't inherit it from init). Like seccomp, registering a filter needs no_new_privs or CAP_SYS_ADMIN, so it is applied before the capabilities are dropped, or, with noNewPrivileges, once no_new_privs is set; in both cases before seccomp.
  • CRIU doesn't restore the restrictions, so on crun restore crun applies them to itself before running CRIU, and the restored processes inherit them.
  • Only BPF filters are used, never IORING_REGISTER_RESTRICTIONS, so the workload can still add its own filters.

youki has the same semantics in youki-dev/youki#3805, so the spec shape is now implemented by two runtimes. Once the spec has the field, the annotation will be replaced by linux.ioUring.

Tests: tests/test_io_uring.py checks the policy in init and in exec (with and without noNewPrivileges), SQE-flag independence, deny-all, and invalid policies; checkpoint-restore-io-uring checks the init of a restored container. They are skipped on kernels without io_uring BPF filters. On Linux 7.0 they pass, and each of them fails with the corresponding apply call removed. The exec, seccomp, capabilities and checkpoint/restore tests pass too.

@giuseppe giuseppe left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left some comments, we also need the check and libcrun_apply_io_uring_restrictions for the libcrun_container_restore case

Comment thread src/libcrun/io_uring.c
};

/* The io_uring opcodes, in `enum io_uring_op` order. */
static const char *io_uring_ops[] = {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks painful to maintain, but I can't see any alternative given what the uapi offers

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. enum io_uring_op has no macros, so there is no way to get the names, or even to check the table against the headers. New opcodes are appended, so it is one line per release, and an opcode the running kernel doesn't know is skipped (EINVAL), so the table may be ahead of the kernel.

Comment thread src/libcrun/io_uring.c
IO_URING_ANNOTATION);
goto exit;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opus 5.5 suggests to not accept socketFamilies without IORING_OP_SOCKET_VALUE:

  +      if (UNLIKELY (! policy->ops[IORING_OP_SOCKET_VALUE]))
  +        {
  +          ret = crun_make_error (err, 0, "%s: `socketFamilies` requires IORING_OP_SOCKET in `ops`", IO_URING_ANNOTATION);
  +          goto exit;
  +        }
  +

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, a policy with socketFamilies but without IORING_OP_SOCKET in ops is now refused. I changed the spec proposal (opencontainers/runtime-spec#1328) and youki (youki-dev/youki#3805) the same way, so they stay in sync.

Comment thread src/libcrun/container.c Outdated
Comment on lines +1280 to +1282
/* Before seccomp, which may deny io_uring_register, and before the
capabilities are dropped: like seccomp, the restrictions need
CAP_SYS_ADMIN or no_new_privs. */

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/* Before seccomp, which may deny io_uring_register, and before the
capabilities are dropped: like seccomp, the restrictions need
CAP_SYS_ADMIN or no_new_privs. */
/* Before seccomp, which may deny io_uring_register, and before the
capabilities are dropped: registering a filter needs CAP_SYS_ADMIN
in the user namespace. */

no_new_privs is set later as part of libcrun_set_caps.

Should we do like seccomp that we try to apply it in two different places?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, no_new_privs isn't set at that point, so it only worked thanks to CAP_SYS_ADMIN. It is now done like seccomp: without noNewPrivileges before the capabilities are dropped, otherwise once libcrun_set_caps has set no_new_privs, both for the init and for exec, and in both cases before seccomp. The tests now run with both values of noNewPrivileges.

Comment thread src/libcrun/container.c

if (UNLIKELY (container_paused))
return crun_make_error (err, 0, "cannot exec in a paused container `%s`", id);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to add here a check for exec'ed processes?

    ret = libcrun_check_io_uring_restrictions (container, err);
    if (UNLIKELY (ret < 0))
      return ret;

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@gvozdetsky

Copy link
Copy Markdown
Author

@giuseppe thanks for the review. About restore: added both. The check is done before anything is created, and crun applies the restrictions to itself before running CRIU, like the keyring: CRIU doesn't restore them, but the restored processes inherit them from CRIU. crun itself stays restricted for the rest of the restore, which is fine as it doesn't use io_uring. The new checkpoint-restore-io-uring test checks the init of the restored container, not an exec.

The fixes are fixup commits for now, I'll squash them once you are happy with them.

@giuseppe giuseppe left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, LGTM

@kolyshkin PTAL

seccomp cannot filter io_uring operations, as they are submitted
through the ring and not as syscalls, so today the only option is to
block io_uring entirely.  Linux 7.0 added task-level io_uring BPF
filters, which are inherited across fork and exec and can only be
tightened, like seccomp.

Add the run.oci.io_uring annotation, in the shape proposed for
linux.ioUring in opencontainers/runtime-spec#1328:

  {"ops": [...], "socketFamilies": [...]}

ops is an allowlist of IORING_OP_* names, every other operation is
denied.  socketFamilies, when present, limits IORING_OP_SOCKET to the
listed address families.  It must not be empty, and ops must contain
IORING_OP_SOCKET.

The policy is validated, and kernel support checked, before the
container is created or restored, and before a process is exec'ed in
it.  Registering a filter needs either no_new_privs or CAP_SYS_ADMIN,
so, like seccomp, the restrictions are applied to the container init
and to every exec'ed process before the capabilities are dropped, or
with noNewPrivileges once no_new_privs is set.  In both cases they are
applied before seccomp, which may deny io_uring_register.  CRIU does
not restore them, so on restore crun applies them to itself before
running CRIU, and the restored processes inherit them.

Only BPF filters are used, never IORING_REGISTER_RESTRICTIONS, so the
SQE flags are not restricted and the workload can still add its own
filters.

The annotation will be replaced by linux.ioUring once it is in the
runtime spec.

Assisted-by: Claude Code (claude-opus-5-5)
Signed-off-by: Eugene Gvozdetsky <gvozdet@gmail.com>
Add an "io_uring" command to tests/init that submits one SQE per
argument on a new ring and reports whether it was denied, an
"io_uring-once-restored" command that does the same once the
container is restored, and a "check-feature io_uring-restrictions"
probe.

The tests check the policy in the container init and in an exec'ed
process, with and without noNewPrivileges, and in a restored
container, that whether an operation is allowed does not depend on its
SQE flags, that an empty ops denies everything, and that invalid
policies are refused.  They are skipped on kernels without io_uring
BPF filters.

Assisted-by: Claude Code (claude-opus-5-5)
Signed-off-by: Eugene Gvozdetsky <gvozdet@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants