Repository navigation
ci: run only the lanes a PR affects + build Docker on main only - #49272
Conversation
🔎 Lint report:
|
| Rule | Count |
|---|---|
unresolved-import |
1 |
First entries
tests/ci/test_classify_changes.py:13: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
✅ Fixed issues: none
Unchanged: 5858 pre-existing issues carried over.
Diagnostics are surfaced as warnings — this check never fails the build.
2626c9b to
e6814bd
Compare
e6814bd to
9353b43
Compare
Heavy PR checks run on every PR because the workflows deliberately avoid `on.paths` filters — a path-gated workflow leaves its required check pending forever when no matching file changes, blocking merge. So a docs-only PR still spins up the TypeScript matrix, the full Python suite, and ruff/ty. Keep every workflow triggering on every PR (checks always report) but gate the expensive *steps* on what the PR touches. Skipping a step (not the job) leaves the job green, so required checks never hang — the same idiom already proven in contributor-check.yml. A classifier (scripts/ci/classify_changes.py) maps the PR diff to three lanes — python, frontend, site — surfaced as step outputs by a composite action (.github/actions/detect-changes). Fail-open: an empty diff or any .github/ change runs everything; python is a denylist (skipped only when every file is provably prose or a frontend-only package); skills/**/SKILL.md counts as python-relevant since the skill-doc tests read that tree. Non-PR events always run the full pipeline.
The image build + smoke test + integration suite are the heaviest jobs in CI (~9-11 min) and ran on every PR. Gate them to push-to-main and release: a broken build surfaces on the main push, while the cheap pre-merge guards (docker-lint hadolint/shellcheck, uv-lockfile-check) still run on PRs to catch the common Dockerfile/lockfile breakage. Steps skip on PRs so the job stays green; the dead PR-only arm64 cache-warm build is removed.
`npm ci` / `uv sync` / toolchain header fetches occasionally die on transient network blips — e.g. node-pty's node-gyp fetching Node headers (an undici assert) during the typecheck job's `npm ci`, which killed the job before `tsc` ever ran. "Re-run and it goes green" is exactly what CI should do itself. - New reusable `.github/actions/retry` composite action wraps a command and retries on failure (3x / 10s, command passed via env so it can't inject). Applied to every PR-path network install: npm ci (typecheck, desktop build, docs site), uv sync (tests, e2e), uv tool install (lint), pip install (docs site). - typecheck now runs `npm ci --ignore-scripts`: `tsc` needs only sources + type defs, so skipping install scripts drops node-pty's native rebuild (whose header fetch was the flake) and is faster. Validated locally — tsc passes for ui-tui, apps/shared, and apps/desktop with scripts skipped. - ripgrep download uses `curl --retry`. Docker (main-only) and the release/windows workflows are intentionally left for a follow-up.
51b7477 to
3c73a5c
Compare
|
1 similar comment
|
3c73a5c to
5479ed9
Compare
ci: centralize path-gating behind single orchestrator + all-checks-pass gate Replace the scattered per-workflow detect-changes pattern with a single ci.yml orchestrator that runs the classifier once, then conditionally calls sub-workflows via workflow_call based on lane outputs. A final all-checks-pass job (if: always()) aggregates all results so branch protection only needs to require one check. Changes: - New .github/workflows/ci.yml orchestrator (detect + conditional calls + all-checks-pass gate) - Extend classify_changes.py with scan/deps/mcp_catalog lanes, absorbing supply-chain-audit's internal changes job - Update detect-changes/action.yml to expose the new lane outputs - Convert all 10 PR-gated sub-workflows to workflow_call-only triggers, removing their push/pull_request triggers and per-step detect-changes guards (gating now happens at the orchestrator level) - lint.yml + supply-chain-audit.yml receive event_name as a workflow_call input to replace github.event_name (which is "workflow_call" inside called workflows) - supply-chain-audit.yml: remove internal changes job + *-gate jobs (orchestrator handles gating, booleans arrive as inputs) - contributor-check.yml: remove internal filter step - Update test_classify_changes.py for 6-lane output + new supply-chain test cases
5479ed9 to
912e8a7
Compare
What this does
Replaces the scattered per-workflow path-gating pattern with a single orchestrator (
ci.yml) that runs the classifier once, then conditionally calls sub-workflows viaworkflow_call. A finalall-checks-passgate job aggregates all results so branch protection only needs to require one check.Also moves the heaviest job (multi-arch Docker) off PRs entirely — it builds on push-to-main and release only.
Why
Every heavy workflow previously had no
on.pathsfilter and ran on every PR: each carried a comment that a path-gated workflow leaves its required check pending forever when no matching file changes, which blocks merge. The price was that a docs-only PR still ran everything — the TypeScript matrix, the full Python suite, ruff/ty, the Docusaurus build, and the ~9-11 min multi-arch Docker build — none of which the change could touch.The old approach ran
detect-changesinside every workflow and gated individual steps withif:guards. That worked but wasted runners (every workflow spins up just to no-op) and the required-checks list in branch protection had to name every individual job.The new approach:
detect-changesonce in the orchestrator.workflow_call— skipped workflows don't spin up runners at all.all-checks-passjob (withif: always()) runs after everything, treatingskippedas success and only failing on actualfailure. Branch protection requires only this one check.How it works
Orchestrator (
ci.yml)All sub-workflows run in parallel after
detectfinishes. Each sub-workflow keeps its own job definitions, matrices, and concurrency settings — they just no longer have their ownpush:/pull_request:triggers (everything flows throughci.yml).Classifier (
scripts/ci/classify_changes.py)Maps the PR diff to 7 lanes, surfaced as outputs by the
.github/actions/detect-changescomposite action:pythonfrontendui-tui/,web/,apps/, rootpackage*.jsonsitewebsite/,skills/,optional-skills/docker_metaDockerfile,docker/,.hadolint.ymlscan.py,.pth,setup.py,pyproject.tomldepspyproject.tomlmcp_catalogoptional-mcps/,hermes_cli/mcp_catalog.pyRace-condition fix
The detect-changes action uses the GitHub compare endpoint with pinned base/head SHAs from the event payload (
repos/{repo}/compare/{base}...{head}) instead of the "current PR files" endpoint (pulls/N/files). The SHAs are frozen at trigger time, so the file list is deterministic even if the PR receives a new push between trigger and detect.Docker
docker-publish.ymlgates every build/smoke/integration step ongithub.event_name != pull_request. Pre-merge Dockerfile/lock sanity is still covered on PRs bydocker-lint(hadolint/shellcheck, gated on thedocker_metalane) anduv-lockfile-check(always runs).workflow_callevent name passthroughWhen a workflow is called via
workflow_call,github.event_nameis"workflow_call", not the event that triggered the caller. The orchestrator passes the real event name as aworkflow_callinput (event_name), andlint.yml+supply-chain-audit.ymluseinputs.event_nameinstead ofgithub.event_namein their conditions.Other changes
build-windows-installer.ymlremoved — it wasworkflow_dispatchonly, not a PR-gated check, and is built elsewhere.retry/action.yml— new composite action for retrying flaky dependency installs (npm ci, uv sync) whose only failures are transient network flakes.changesjob and all*-gatejobs removed; path-gating now handled centrally by the orchestrator. Thescan/deps/mcp_catalogbooleans arrive asworkflow_callinputs.Safety contract — fail open, never closed
.github/change → run everything.pythonis a denylist: skipped only when every changed file is provably prose, a frontend-only package, or docker metadata; an unrecognized path keeps it on.skills/**/SKILL.mdis python-relevant (the skill-doc tests read that tree), not pure docs.main, release) always run the full pipeline.What stays separate
docker-publish.yml,deploy-site.yml,skills-index.yml,skills-index-freshness.yml,upload_to_pypi.yml— not PR-gated required checks, trigger on release/schedule/dispatch.Branch protection change
Replace the long list of individual required checks with just
all-checks-pass(from theCIworkflow).Validation
tests/ci/test_classify_changes.py.docker-lintskipped when no Dockerfile changes,lint-diffskipped on PRs).