Skip to content

feat: support kubernetes init containers and least privilege - #34

Merged
amannocci merged 1 commit into
mainfrom
feat/kubernetes-init-containers
Oct 20, 2026
Merged

amannocci merged 1 commit into
mainfrom
feat/kubernetes-init-containers

Conversation

@amannocci

Copy link
Copy Markdown
Member

What is the change being made?

  • IGNITY_MODE (full default, init, services) splits the init phase and the services phase across two containers (a Kubernetes init container and the main container). They share files through a user-mounted volume, and no env is handed over.
  • /etc/ignity/envs/* become non-secret defaults applied before s6-svscan. The container env wins and is never written to disk. with-env is a passthrough by default: -k narrows the inherited env and -d loads an envdir for one process.
  • The CMD drops to USERMAP_UID/USERMAP_GID when PID 1 is root.
  • Least privilege is documented and tested: six capabilities (CHOWN, FOWNER, DAC_OVERRIDE, SETUID, SETGID, KILL) and a read-only root filesystem. IGNITY_ACCOUNTS_DIR (gen-accounts) lets an init container generate passwd/group for read-only roots.
  • fix-perms never follows symlinks and exits non-zero when chown/chmod fail.
  • stage1 no longer passes -- to s6-rmrf, which takes no option.
  • stage3 waits for services (eltest -d instead of -f), and a grace time of 0 skips the wait.
  • Tests: shared image fixture and image_fixture() factory, default Containerfile template, env_args/READ_ONLY helpers, tuple constants.

Behaviour changes:

  • A failing chown/chmod in fix-perms now aborts the boot.
  • with-env no longer restores env from a dump. It is POSIX sh and needs awk for -k.
  • /run/ignity/envs no longer exists.
  • An invalid IGNITY_MODE fails the container.
  • Without CAP_KILL, shutdown never ends (stage3 waits for all children).

Why is the change being made?

ignity was designed for Docker Swarm, with everything in one container. On Kubernetes the init phase can run in an init container and the services in the main container, so that secrets only given to the init container never reach the main one. The same work lets ignity run with minimal capabilities and a read-only root filesystem, which Pod Security policies ask for.

How has this been tested?

  • uv run poe test: 67 passed.
  • New tests/test_modes.py (two-container flow with a shared volume, failing init, invalid mode) and tests/test_least_privilege.py (minimal set, plus one test per capability showing what breaks without it).
  • New cases in the envs, boot, preboot and perms tests (env inheritance, with-env -k/-d, CMD privilege drop, account generation, symlink handling).
  • Not tested: the README pod example on a real cluster, and a non-root PID 1.

How has this been documented?

  • README: sections on Kubernetes init containers, scoped secrets, least privilege (capability table), read-only mode, account generation and non-root PID 1 (documented as an untested exception). Also fixes the IGNITY_SERVICES_GRACETIME default and a /run/ingity typo.
  • .claude/CLAUDE.md: architecture, environment variables and test helpers.

Related Issues

None.

Add IGNITY_MODE (full | init | services) to split the init phase and the
services phase across two containers sharing files through a volume. No
env is handed over, so secrets of the init container stay there.

- env: /etc/ignity/envs become non-secret defaults applied before
  s6-svscan, container env wins and is never written to disk
- with-env: passthrough, -k narrows the env, -d loads an envdir
- stage2-cmd: the CMD drops to USERMAP_UID/USERMAP_GID
- gen-accounts: IGNITY_ACCOUNTS_DIR for read-only root filesystems
- fix-perms: never follow symlinks and fail when chown/chmod fail
- stage1: s6-rmrf takes no option, fails on a read-only root
- stage3: wait for services (-d, not -f), skip the wait at grace time 0
- tests: shared image fixture, default Containerfile, tuple constants,
  mode and per-capability tests
- docs: Kubernetes, least privilege and read-only sections
@amannocci amannocci self-assigned this Oct 20, 2026
@amannocci
amannocci merged commit 6e9782c into main Oct 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant