Skip to content

README.md

Latest commit

 

History

History
88 lines (69 loc) · 3.81 KB

README.md

File metadata and controls

88 lines (69 loc) · 3.81 KB
Raw

build/ — Pipeline definitions

This directory contains Azure DevOps pipeline YAML files and the pod-scheduler configs that generate them.

Generated files (do not edit by hand)

The CI pipelines that run the daily benchmark sweep are generated by the pod-scheduler. Each pipeline maps to a JSON config that lists the available pods (machine groups) and scenarios.

Generated YAML Source config Notes
benchmarks-ci-01.yml, benchmarks-ci-02.yml benchmarks_ci_pods.json On-prem fleet (citrine + mono); split across two YAMLs to spread load
benchmarks-ci-azure.yml benchmarks_ci_azure_pods.json Azure cloud pods (includes the merged eastus2 cobalt cloud pods)
benchmarks-ci-cobalt.yml benchmarks_ci_cobalt_pods.json Cobalt-hosted pods

To regenerate, from the repo root:

python ./scripts/pod-scheduler/main.py \
    --config ./build/benchmarks_ci_pods.json \
    --yaml-output ./build

python ./scripts/pod-scheduler/main.py \
    --config ./build/benchmarks_ci_azure_pods.json \
    --base-name benchmarks-ci-azure \
    --yaml-output ./build

python ./scripts/pod-scheduler/main.py \
    --config ./build/benchmarks_ci_cobalt_pods.json \
    --base-name benchmarks-ci-cobalt \
    --yaml-output ./build

The header of each generated YAML embeds the exact regen command for that file.

Hand-maintained scenario templates

Everything else in this directory is hand-maintained. The most important groups:

  • *-scenarios.yml — AzDO job templates that the generated pipelines invoke (e.g. proxies-scenarios.yml, grpc-scenarios.yml). One per benchmark family. The pod-scheduler picks which template to run via the template field on each scenario in the JSON config.
  • prbenchmarks.*.config.yml — PR-trigger pipelines for individual PR runs.
  • job-variables.yml — shared variable definitions consumed by the scenario templates.

When adding a new benchmark family, drop a *-scenarios.yml template in this directory and reference it from the relevant benchmarks_ci_*_pods.json.

Editing the schedule

Common edits, all done in JSON:

  • Add a scenario: append to scenarios[] in the appropriate benchmarks_ci_*_pods.json.
  • Add a pod / machine class: append to pods[] and reference its name from any scenario you want to target it.
  • Change cadence or splits: edit metadata.schedule (cron), metadata.yaml_generation.target_yaml_count, or metadata.yaml_generation.schedule_offset_hours.
  • Override a job's timeout: add "timeout": <minutes> to the scenario.

After any edit, regenerate the YAML and commit both the JSON and the YAML together. The snapshot tests in scripts/pod-scheduler/tests/test_snapshots.py will fail in CI if the generated YAML drifts from the configs.

Minimal Todo

minimal-todo-scenarios.yml dispatches scenarios/minimal-todo/minimal-todo.benchmarks.yml (1 CPU, 536,870,912 bytes / 512 MiB, 25200 req/s, enabled) for the gold-lin and Azure azure-arm64/cobalt-cloud-lin pods. C# runs on every regular scheduled slot using the direct (non-Docker) csharp-project build; Go and Rust run the Docker go/rust builds but only on a shared weekly slot (condition: Math.round(Date.now() / 43200000) % 14 == 0, combined with the pod's own condition, matching the containers-scenarios.yml idiom). The Docker csharp/go/rust builds (floating, multi-arch base images, no pinned digests, no requiredArchitecture gating) remain the canonical generation/update/admission/pre-promotion comparison path and are unaffected -- see scenarios/minimal-todo/README.md for the full execution-path mapping. Generation, admission, and calibration tooling for this cohort live in the separate dotnet-performance-tools repository; this directory only carries the scheduled dispatch template.