You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(metalman): support rack manager virtual media installs - #910
Metalman can now manage rack managers that expose one Redfish service per slot under a URL path prefix, using a bootloader served as virtual media - no DHCP needed.
Also adds an Azure linux host image in order to support machines with MANA NICs.
Metalman can now drive rack managers that expose one Redfish service per
slot under a URL path prefix, such as https://manager/7. The client finds
these endpoints on its own, so the Machine API stays vendor neutral.
A new VirtualMedia boot protocol installs a machine without DHCP boot
options. Metalman builds a small per-operation UEFI ISO with a static
network GRUB config, stages it on the rack manager over TFTP, mounts it,
and boots it. The installer then fetches the kernel, initrd, and disk image
over HTTP. A durable state machine with a finalizer handles boot retries,
cleanup, and a reboot barrier so the host never boots the media twice.
Also adds Azure Linux 3.0 machine images (with an NVIDIA variant), runs the
installer on the Azure Linux kernel so MANA NICs work, and fixes issues
found while installing a real arm64 GPU host.
Recheck site ownership and outstanding media on each finalizer update. Keep incomplete acquisition protected and verify the persisted finalizer before media effects.
Cover cross-site cleanup, restart recovery, stale status, and conflicting updates.
Reject live-IP and forwarded-IP fallback for retained media leases and VirtualMedia Machines. Keep media reports on the authorized owner path and preserve PXE and HTTP forwarding behavior.
End callback authorization when media retention is abandoned. Keep callbacks available for failed targets before abandonment, and test rejected requests leave status unchanged.
Pass the serve-pxe site to the media provider and filter frozen media claims and live successors by site. Treat an empty site as an exact match, not a wildcard.
Cover cross-site lease collisions, callback rejection, and same-site access with regression tests.
Return 503 with Retry-After when saving a media cloud-init condition fails, while preserving the error log and legacy report behavior. Cover failed patches, successful persistence, and events without condition updates.
Preparation succeeds without cached frozen disk image
internal/metalman/netboot/media.go:319
Preparation validates the netboot image but never checks that the frozen disk image is cached. If its pull is failing, preparation still succeeds and the controller can power-cycle the host into an installer that cannot download its image. Once download retries expire, the installer drops into a shell (images/netboot/assets/init:333-335); LoaderStartedAt prevents a boot retry (machineops/virtualmedia.go:401). Resolve the frozen disk.img.gz before preparation succeeds, returning ErrNotYetDownloaded while its image is pending. Update the preparation fixtures to cache the disk image too.
ISO release prematurely removes frozen image authorization
internal/metalman/netboot/media_http.go:206
ISO release does not end first-boot authorization: firstBootOwner still accepts terminal targets with ImageWritten and RebootRequestedAt until the final cloud-init report (media_http.go:47-53). This predicate nevertheless stops the OCI reconciler from restoring their frozen image mappings (reconciler.go:117). After a restart, if the Machine's netboot reference has changed, authorized cloud-init template requests cannot resolve the frozen image (oci_cache.go:193-195). Retain those image references until first-boot authorization ends, and cover this restart case.
Remove trailing slashes and normalize decoded paths before routing so authorization and cache lookup use the same path. Keep boot media capability aliases rejected. Add alias tests for artifacts, first-boot data, reports, and kernel range requests.
Ignore foreign-site media before matching a Machine name or lease. Share exact site matching across media lookups and image reconciliation.
Cover direct and forwarded PXE/HTTP datasource and report requests, including name and lease collisions, empty sites, and same-site rejection.
Released ISO artifacts lose frozen image mappings after restart
internal/metalman/netboot/reconciler.go:119
Releasing ISO artifacts does not end first-boot authorization. A failed target with ImageWritten and RebootRequestedAt remains authorized until its final cloud-init report or abandonment (media_http.go:47-53), but this filter stops restoring its frozen image mappings. After a Metalman restart, if the Machine's netboot image has changed, neither watch restores the old mapping. Authorized cloud-init requests then fail even when the frozen files remain on disk, since NewOCICache starts with empty mappings (oci_cache.go:49-54). Keep the frozen netboot image reconciled for the full first-boot authorization lifetime and add a released/failed-target restart test.
Blade PowerOn does not complete pending ResetOn recovery
internal/metalman/redfish/rackmanager.go:229
A successful blade PowerOn read leaves a pending ResetOn recovery incomplete. Slot power-on itself boots the host, and the virtual-media controller advances to Installing on this read without retrying ResetOn (machineops/virtualmedia.go:525-531). Consequently, pendingCompleted stays false and later outages on the same client cannot trigger recovery because rackMutateLocked requires pending == recoveryNone. Mark the pending ResetOn recovery complete when the blade confirms PowerOn, while preserving the grace window.
…ease path
The noble release alias no longer serves 24.04.4 files. Use the path for that point release so the download keeps working after newer point releases.
Read the namespace ID from sysfs, falling back to the device name, and create namespace-qualified model/serial aliases. Reserve the legacy alias for namespace 1 so later namespaces cannot redirect installer writes; retain WWID aliases.
Compare retained media ownership in both directions before creation time and name to prevent cyclic conflict waits. Use the same ordering to select blockers, and cover media priority, tie-breaking, and install timeout progress after a Redfish URL change.
Share the first-boot lifetime predicate with OCI reconciliation so released targets retain frozen netboot templates until a final report or abandonment. Restore only the netboot image after artifact release and cover restart recovery, ended claims, and site isolation.
The Ubuntu shim and GRUB reject the Azure Linux kernel with Secure Boot
enabled. The amd64 Ubuntu generic kernel already has the MANA driver, so
only arm64 needs the Azure Linux kernel.
…image
The default netboot image keeps the Ubuntu kernel on every architecture so the Ubuntu shim and GRUB can verify it with Secure Boot on. netboot-azurelinux runs the Azure Linux kernel for arm64 hosts with NICs the Ubuntu kernel does not support.
The virtual-media controller looks solid. My concern is that the image and installer work feels too tied to Azure Linux, MANA, and NVIDIA.
Right now, supporting another OS, NIC, or GPU setup may require another image variant and more special cases in the shared installer scripts. Can we keep these pieces separate?
the generic virtual-media install flow
installer requirements such as architecture, drivers, and Secure Boot support
OS-specific disk and network setup
optional hardware support such as MANA, NVIDIA, Fabric Manager, or IMEX
The above was generated by the agent but in some sense I agree with the concern. What do we think about AZL vs Ubuntu and how we are going to set up the lab vs other setups that might use metalman that dont use azure nics.
The virtual-media controller looks solid. My concern is that the image and installer work feels too tied to Azure Linux, MANA, and NVIDIA.
Right now, supporting another OS, NIC, or GPU setup may require another image variant and more special cases in the shared installer scripts. Can we keep these pieces separate?
the generic virtual-media install flow
installer requirements such as architecture, drivers, and Secure Boot support
OS-specific disk and network setup
optional hardware support such as MANA, NVIDIA, Fabric Manager, or IMEX
The above was generated by the agent but in some sense I agree with the concern. What do we think about AZL vs Ubuntu and how we are going to set up the lab vs other setups that might use metalman that dont use azure nics.
The netboot changes will still work on vanilla hardware - they just add support for MANA, which is easier to do by pivoting to Azure linux host images. We can build a custom Ubuntu image that includes the MANA drivers but I wanted to keep that out of scope here
Copilot resolve the merge conflicts in this pull request
Merged current main and resolved the netboot Containerfile conflict in 017431e, preserving the separate installer kernel variants and main's stable Noble download URLs. Targeted release, filesystem, and Metalman tests passed, along with both container-definition checks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Metalman can now manage rack managers that expose one Redfish service per slot under a URL path prefix, using a bootloader served as virtual media - no DHCP needed.
Also adds an Azure linux host image in order to support machines with MANA NICs.