Repository navigation
linux: add experimental io_uring restrictions - #2386
gvozdetsky wants to merge 2 commits into
Conversation
giuseppe
left a comment
There was a problem hiding this comment.
left some comments, we also need the check and libcrun_apply_io_uring_restrictions for the libcrun_container_restore case
| }; | ||
|
|
||
| /* The io_uring opcodes, in `enum io_uring_op` order. */ | ||
| static const char *io_uring_ops[] = { |
There was a problem hiding this comment.
this looks painful to maintain, but I can't see any alternative given what the uapi offers
There was a problem hiding this comment.
Agreed. enum io_uring_op has no macros, so there is no way to get the names, or even to check the table against the headers. New opcodes are appended, so it is one line per release, and an opcode the running kernel doesn't know is skipped (EINVAL), so the table may be ahead of the kernel.
| IO_URING_ANNOTATION); | ||
| goto exit; | ||
| } | ||
|
|
There was a problem hiding this comment.
Opus 5.5 suggests to not accept socketFamilies without IORING_OP_SOCKET_VALUE:
+ if (UNLIKELY (! policy->ops[IORING_OP_SOCKET_VALUE]))
+ {
+ ret = crun_make_error (err, 0, "%s: `socketFamilies` requires IORING_OP_SOCKET in `ops`", IO_URING_ANNOTATION);
+ goto exit;
+ }
+There was a problem hiding this comment.
Done, a policy with socketFamilies but without IORING_OP_SOCKET in ops is now refused. I changed the spec proposal (opencontainers/runtime-spec#1328) and youki (youki-dev/youki#3805) the same way, so they stay in sync.
| /* Before seccomp, which may deny io_uring_register, and before the | ||
| capabilities are dropped: like seccomp, the restrictions need | ||
| CAP_SYS_ADMIN or no_new_privs. */ |
There was a problem hiding this comment.
| /* Before seccomp, which may deny io_uring_register, and before the | |
| capabilities are dropped: like seccomp, the restrictions need | |
| CAP_SYS_ADMIN or no_new_privs. */ | |
| /* Before seccomp, which may deny io_uring_register, and before the | |
| capabilities are dropped: registering a filter needs CAP_SYS_ADMIN | |
| in the user namespace. */ |
no_new_privs is set later as part of libcrun_set_caps.
Should we do like seccomp that we try to apply it in two different places?
There was a problem hiding this comment.
Good catch, no_new_privs isn't set at that point, so it only worked thanks to CAP_SYS_ADMIN. It is now done like seccomp: without noNewPrivileges before the capabilities are dropped, otherwise once libcrun_set_caps has set no_new_privs, both for the init and for exec, and in both cases before seccomp. The tests now run with both values of noNewPrivileges.
|
|
||
| if (UNLIKELY (container_paused)) | ||
| return crun_make_error (err, 0, "cannot exec in a paused container `%s`", id); | ||
|
|
There was a problem hiding this comment.
do we need to add here a check for exec'ed processes?
ret = libcrun_check_io_uring_restrictions (container, err);
if (UNLIKELY (ret < 0))
return ret;
|
@giuseppe thanks for the review. About restore: added both. The check is done before anything is created, and crun applies the restrictions to itself before running CRIU, like the keyring: CRIU doesn't restore them, but the restored processes inherit them from CRIU. crun itself stays restricted for the rest of the restore, which is fine as it doesn't use io_uring. The new The fixes are fixup commits for now, I'll squash them once you are happy with them. |
giuseppe
left a comment
There was a problem hiding this comment.
thanks, LGTM
@kolyshkin PTAL
seccomp cannot filter io_uring operations, as they are submitted through the ring and not as syscalls, so today the only option is to block io_uring entirely. Linux 7.0 added task-level io_uring BPF filters, which are inherited across fork and exec and can only be tightened, like seccomp. Add the run.oci.io_uring annotation, in the shape proposed for linux.ioUring in opencontainers/runtime-spec#1328: {"ops": [...], "socketFamilies": [...]} ops is an allowlist of IORING_OP_* names, every other operation is denied. socketFamilies, when present, limits IORING_OP_SOCKET to the listed address families. It must not be empty, and ops must contain IORING_OP_SOCKET. The policy is validated, and kernel support checked, before the container is created or restored, and before a process is exec'ed in it. Registering a filter needs either no_new_privs or CAP_SYS_ADMIN, so, like seccomp, the restrictions are applied to the container init and to every exec'ed process before the capabilities are dropped, or with noNewPrivileges once no_new_privs is set. In both cases they are applied before seccomp, which may deny io_uring_register. CRIU does not restore them, so on restore crun applies them to itself before running CRIU, and the restored processes inherit them. Only BPF filters are used, never IORING_REGISTER_RESTRICTIONS, so the SQE flags are not restricted and the workload can still add its own filters. The annotation will be replaced by linux.ioUring once it is in the runtime spec. Assisted-by: Claude Code (claude-opus-5-5) Signed-off-by: Eugene Gvozdetsky <gvozdet@gmail.com>
Add an "io_uring" command to tests/init that submits one SQE per argument on a new ring and reports whether it was denied, an "io_uring-once-restored" command that does the same once the container is restored, and a "check-feature io_uring-restrictions" probe. The tests check the policy in the container init and in an exec'ed process, with and without noNewPrivileges, and in a restored container, that whether an operation is allowed does not depend on its SQE flags, that an empty ops denies everything, and that invalid policies are refused. They are skipped on kernels without io_uring BPF filters. Assisted-by: Claude Code (claude-opus-5-5) Signed-off-by: Eugene Gvozdetsky <gvozdet@gmail.com>
45fde5f to
216ab3b
Compare
seccomp can't filter io_uring operations: they are submitted through the ring, not as syscalls, so runtimes currently block io_uring entirely. Linux 7.0 added task-level io_uring BPF filters, which are inherited across fork and exec and can only be tightened, like seccomp.
This adds experimental support behind the
run.oci.io_uringannotation, in the shape proposed forlinux.ioUringin opencontainers/runtime-spec#1328 (proposal: opencontainers/runtime-spec#1323):{ "ops": ["IORING_OP_READ", "IORING_OP_WRITE", "IORING_OP_SOCKET"], "socketFamilies": ["AF_UNIX", "AF_INET", "AF_INET6"] }opsis an allowlist, every other operation is denied. It doesn't depend on the SQE flags.socketFamilies, when present, limitsIORING_OP_SOCKETto these families. It requiresIORING_OP_SOCKETinops, and an empty list is refused; to deny all sockets, leaveIORING_OP_SOCKETout ofops.crun exec.crun execprocess (they don't inherit it from init). Like seccomp, registering a filter needsno_new_privsorCAP_SYS_ADMIN, so it is applied before the capabilities are dropped, or, withnoNewPrivileges, onceno_new_privsis set; in both cases before seccomp.crun restorecrun applies them to itself before running CRIU, and the restored processes inherit them.IORING_REGISTER_RESTRICTIONS, so the workload can still add its own filters.youki has the same semantics in youki-dev/youki#3805, so the spec shape is now implemented by two runtimes. Once the spec has the field, the annotation will be replaced by
linux.ioUring.Tests:
tests/test_io_uring.pychecks the policy in init and in exec (with and withoutnoNewPrivileges), SQE-flag independence, deny-all, and invalid policies;checkpoint-restore-io-uringchecks the init of a restored container. They are skipped on kernels without io_uring BPF filters. On Linux 7.0 they pass, and each of them fails with the corresponding apply call removed. The exec, seccomp, capabilities and checkpoint/restore tests pass too.