Repository navigation
Conversation
…the leg luet pulls quay.io/kairos/packages:repository.yaml on first use. When that pull returns an empty blob it resolves against an empty tree and reports the packages as not found, so the step dies with a message that reads like a repository content problem. Run 36871582383 lost encryption-remote-https-pinned this way while the other 13 encryption legs of the same run passed. The repository is cached, so retrying on its own reads the same empty index back. Drop /var/cache/luet/repos/kairos between attempts, three attempts with a growing delay, and keep the non-zero exit when all three fail. Give both curl calls -f and --retry as well. Without -f curl writes the body of an HTTP error to the output file and still exits 0, so a bad gateway from the release host installs an error page as /usr/bin/luet, or pipes one into sudo bash for cinc-auditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Ettore Di Giacinto <mudler@kairos.io>
mudler/luet#404 makes luet report a pull that unpacked nothing as the download failure it is, and pulls the image again before giving up, so the workflow has no reason to loop over `luet install` and clear the repository cache itself. What stays is the `curl` hardening, which is a separate fault: without `-f` curl writes the body of an HTTP error to the output file and still exits 0, so a bad gateway from the release host installs an error page as /usr/bin/luet, or pipes one into `sudo bash` for cinc-auditor. Signed-off-by: Ettore Di Giacinto <mudler@kairos.io> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Taking a look at this PR now and will post a review shortly. |
itxaka-agent
left a comment
There was a problem hiding this comment.
Automated triage agent (
kairos-triage-agent) running as@itxaka-agent.
This comment was generated by software. A human maintainer can override
any action taken here.
Looks good. Adding -f stops an HTTP error page from being saved as /usr/bin/luet, and the retries cover short outages on the release host. The flags match what release-scan.yml already uses.
Audit summary: one review round on head c220fab. Checked the workflow change to the two |
… pull c220fab dropped the retry on the premise that mudler/luet#404 had made it unnecessary. That pull request is still open, and this step pins the luet binary released in 2022, so merging it upstream would not change what runs here either. luet 0.33.0 leaves the artifact nil when a pull fails and then dereferences it, so a truncated blob from quay is a SIGSEGV rather than an error: WARNING failed downloading image quay.io/kairos/packages:kubectl-container-1.31.0-1: unexpected EOF panic: runtime error: invalid memory address or nil pointer dereference artifact.(*PackageArtifact).Verify(0x0, ...) That is the only failing job in run 37599588356 on master b3cd998. Three attempts with a growing delay, dropping /var/cache/luet in between: both the repository index and the partial artifact are cached, so a retry that keeps the cache reads the same bytes back. The non-zero exit after the third attempt stays, so a real outage still reddens the leg. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Ettore Di Giacinto <mudler@kairos.io>
|
Restored the retry around Evidence it is still needed: run 37599588356 on master |
kairos-io#5277 renamed the cinc-auditor step to "for the InSpec profile tests" and gated it on the new `cis` test as well, in the same region where this branch replaces the single `luet install` line with the retry loop. Both sides kept: master's step name and condition, this branch's loop. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Ettore Di Giacinto <mudler@kairos.io>
The encryption deps step downloaded luet 0.33.0, released in October 2022. That build does not survive a failed pull: when a blob from quay arrives truncated it logs the download as a warning, leaves the artifact nil, and then dereferences it. WARNING failed downloading image quay.io/kairos/packages:kubectl-container-1.31.0-1: unexpected EOF panic: runtime error: invalid memory address or nil pointer dereference artifact.(*PackageArtifact).Verify(0x0, ...) installer.(*LuetInstaller).getPackage(...) That is what took master red on b3cd998 (run 37599588356) and again on 3dd0f1f (run 37740630820), on a different encryption leg each time. luet 0.37.0 returns the download error instead of panicking, and quay.io/kairos/packages is published with luet 0.36.0, so the consumer was the one version nothing else in the project used. No built-in renovate manager scans a release binary fetched by URL, which is how the pin stayed four years behind. The version now lives in a LUET_VERSION variable behind a renovate comment, and a custom manager reads that shape in any workflow file. The retry loop stays. A clean error is still a red leg. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Ettore Di Giacinto <mudler@kairos.io>
itxaka-agent
left a comment
There was a problem hiding this comment.
Automated triage agent (
kairos-triage-agent) running as@itxaka-agent.
This comment was generated by software. A human maintainer can override any
action taken here.
Round 1 at 236736c: approve. The exit 0 inside the retry loop ends the step cleanly, three failed attempts end in exit 1, and the new Renovate matcher captures 0.37.0 from the LUET_VERSION line without overlapping the existing manager.
|
Picking this up for QA. |
QA Result: PASSTested head The PR's own CI does not run this step: the encryption legs are not in the PR matrix, and
Summary: Under the same truncated quay download, master's step panics exactly as in #5123 and the PR's step either recovers inside luet 0.37.0 or through the new 3-attempt loop. When the registry stays broken it fails with a clear message. The Not covered: a real GitHub runner run of an |
encryption-*legs die in theEncryption depsstep, in three different waysthat all come from the same six lines.
1. The step pinned luet 0.33.0, released in October 2022
That build does not survive a failed pull. When a blob from quay arrives
truncated it logs the download as a warning, leaves the artifact nil, and then
dereferences it:
That took
masterred on b3cd998 (run 37599588356) and again on 3dd0f1f(run 37740630820), on a different encryption leg each time.
luet 0.37.0 returns the download error instead (mudler/luet#393), and
quay.io/kairos/packagesis itself published with luet 0.36.0, so 0.33.0 wasthe one version nothing else in the project used.
No built-in renovate manager scans a release binary fetched by URL, which is
how the pin stayed four years behind. The version moves into a
LUET_VERSIONvariable behind a
# renovate:comment, andrenovate.jsongets a custommanager that reads that shape in any workflow file. It is the same matcher the
repository already uses for
.shfiles, with the quotedname:tagformreplaced by a bare version.
2. A transient failure still has no retry
A clean error is still a red leg. Run 36871582383 on #5111 lost
encryption-remote-https-pinnedwhile the other 13 encryption legs of the samerun, against the identical repository, all passed. The step now tries three
times. The repository index and the partial artifact are both cached, so the
cache is dropped between attempts or the next one reads the same bytes back.
The other half of that belongs in luet, not here: mudler/luet#404. luet was
reporting a layer it could not fetch as a successful unpack of an empty image,
because go-containerregistry closes the tar writer before closing the
extraction stream with the error, so the unpack stops at the end-of-archive
marker and never reads it.
3. Neither
curlchecked its statusBoth calls get
-fand--retry. Without-f, curl writes the body of anHTTP error to the output file and still exits 0, so a bad gateway from the
release host installs an error page as
/usr/bin/luet, or pipes one intosudo bashfor cinc-auditor.How it was tested
There is no runner for a workflow change, so each piece was checked on its own:
luet-0.37.0-linux-amd64from the release URL this builds, ranluet --version, and confirmedrepo addwrites the same repository file as0.33.0 apart from an added empty
auth:block, and thatinstalltakes thesame flags.
quay.io/kairos/packageswith 0.37.0. It reaches the registry andresolves the repository. The install itself cannot finish off a runner,
because unpacking needs to
lchownas root.0.37.0tag, not only onluet master.
matchStringsentry inrenovate.jsonwith Go'sregexp,which is the same RE2 engine renovate uses, and ran them over every workflow
file. The new manager matches once, in this file, and reports
datasource=github-releases depName=mudler/luet currentValue=0.37.0.yaml.v3and ranbash -nover all 11 of itsrunblocks.
Fixes #5123
Written with AI assistance. No human has read it yet.
🤖 Generated with Claude Code