ci: publish OpenVMM source releases by manual dispatch - #4140
Draft
benhillis wants to merge 5 commits into
Draft
Conversation
`openvmm --version` did not exist, and there was nothing to report if it had: the workspace set no `version`, so every crate in it was `0.0.0`. OpenVMM ships as a source archive that packagers build themselves, which rules out the usual approach of having CI inject a version at build time -- the packager builds long after our pipeline has exited, from a tree with no git history to recover a version from. The version therefore has to be committed to the tree. QEMU, systemd, the kernel, cloud-hypervisor and rustc all do this, and each falls back gracefully when `.git` is absent rather than deriving the release version from a tag. So: set `version` in `[workspace.package]`, inherit it in `openvmm` and `openvmm_entry`, and wire up clap's `version`. `OPENVMM_PKGVERSION` lets a packager append their own build identity, as QEMU's `-Dpkgversion` and cloud-hypervisor's `CH_EXTRA_VERSION` do; an empty value is ignored, since build systems routinely pass an undefined variable through as "". Two things fall out of that: - The Windows VERSIONINFO resource is a second version surface, stamped from `OPENVMM_MAJOR`/`MINOR`/`PATCH`/`REVISION` and defaulting to `0.0.0.0`. No in-repo caller sets those, so leaving it alone would have left one binary reporting `0.1.0-dev` from `--version` and `0.0.0.0` from its file properties. It now defaults to the crate version, with the env vars still overriding per-component so a pipeline can stamp a build number in. - `cargo xtask fmt` deliberately strips `version` from `[package]`, partly because doing so also makes a crate unpublishable. The three crates on its exception list now set `publish = false` explicitly, restoring the property the lint was relying on. Verified by building `openvmm` from an extracted archive with no `.git` anywhere above it, which is the case that actually matters.
The workspace version advances only in the reviewed pull request that selects the next release. Development-build identity is added separately from the committed product version. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0ba1c0df-faf0-4a0e-b39d-c3eb0e363fce
OpenVMM is meant to be built and packaged by Linux distributions from a source archive, which makes that configuration a shipping interface. It is also the one configuration nothing in CI covers: every other build provisions native dependencies through `.packages/`, which a packager cannot use. A change that only resolves through `.packages/` therefore breaks downstream packagers silently. Add a job that does what a packager does, in order: 1. assemble a source archive from the commit under test, plus a `SHA256SUMS` covering it; 2. `sha256sum --check --strict`; 3. unpack outside the repository, so nothing can reach back into the checkout; 4. assert there is no `.git` inside; 5. assert the archive's embedded `.openvmm-release.json` matches what it was assembled as; 6. `cargo build --release --locked -p openvmm` with `PROTOC` and `OPENSSL_NO_VENDOR=1`, matching what a spec file does; 7. assert the binary's own `NEEDED` entries name the system OpenSSL. Building the checkout rather than an archive would let this pass on a tree a packager cannot reproduce, since a packager has no `.git` directory and no untracked files. The archive also has to carry its identity in a file, for the same reason. Step 7 is what keeps the build honest: if something acquired a vendored or static OpenSSL the build would still succeed, but the packaged binary would no longer be one the distribution can service. It reads `readelf -d` rather than `ldd` because `ldd` reports the whole transitive closure, and so would be satisfied by an unrelated library pulling OpenSSL in. Assembly is reproducible -- `git archive` is deterministic for a commit, `gzip -n` omits the timestamp, and `tar.umask` is pinned rather than inherited from the machine. Measured at 5m35s on a GitHub-hosted runner, against an existing 11m x64-linux baseline. Dropping debug info from the release profile and disabling incremental compilation is what keeps it there. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0ba1c0df-faf0-4a0e-b39d-c3eb0e363fce
Remove stamped release metadata from the assembled source. The archive already contains the canonical workspace version, so the distribution-build gate now builds the extracted tree and requires the resulting binary to report that exact version. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0ba1c0df-faf0-4a0e-b39d-c3eb0e363fce
Pin the ref selected in GitHub's Run workflow UI, require its commit to be reachable from main, validate a stable version newer than existing OpenVMM tags, build the assembled archive in the distribution configuration, attest it, and create a human-reviewed draft release. Reruns replace drafts but published releases and tags remain immutable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0ba1c0df-faf0-4a0e-b39d-c3eb0e363fce
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a manually dispatched, source-only OpenVMM release workflow implemented via Flowey pipelines/nodes, with deterministic source-archive assembly, a distribution-style build gate, provenance attestation, and draft GitHub release creation tied to the pinned commit and workspace version.
Changes:
- Define and propagate a canonical
[workspace.package]version intoopenvmm/openvmm_entry(and guard it via linting/tests), plus mark versioned crates aspublish = false. - Add Flowey nodes/jobs to assemble a reproducible source archive + checksums, validate a distro-style build from that archive, and publish a draft GitHub release with attestations.
- Extend the shared GitHub release publisher to support generated vs explicit notes and configurable behavior when a release already exists.
Reviewed changes
Copilot reviewed 26 out of 27 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| xtask/src/tasks/fmt/lints/package_info.rs | Document and expand the “version exceptions” lint policy for crates that must carry versions. |
| vm/vmgs/vmgstool/Cargo.toml | Mark vmgstool as publish = false while retaining its explicit version. |
| openvmm/openvmm/Cargo.toml | Inherit workspace version and set publish = false for openvmm. |
| openvmm/openvmm/build.rs | Default Windows VERSIONINFO stamping to the crate version (with env overrides still taking precedence). |
| openvmm/openvmm_entry/src/cli_args.rs | Make --version report either OPENVMM_PKGVERSION or the crate version; add a test preventing silent 0.0.0. |
| openvmm/openvmm_entry/Cargo.toml | Inherit workspace version and set publish = false for openvmm_entry. |
| flowey/flowey_lib_hvlite/src/lib.rs | Export the new source-release assembly node. |
| flowey/flowey_lib_hvlite/src/assemble_openvmm_source_release.rs | New deterministic source-archive assembler + checksum writer, with identity derived from workspace version + git revision. |
| flowey/flowey_lib_hvlite/src/_jobs/publish_vmgstool_gh_release.rs | Update vmgstool release publishing params to use the enhanced release publisher API. |
| flowey/flowey_lib_hvlite/src/_jobs/publish_openvmm_gh_release.rs | New job to enforce release invariants and publish a draft OpenVMM source release. |
| flowey/flowey_lib_hvlite/src/_jobs/mod.rs | Wire in the new distro build check and OpenVMM release publish jobs. |
| flowey/flowey_lib_hvlite/src/_jobs/check_distro_build.rs | New job that builds openvmm from the assembled source archive using distro-style dependencies/env. |
| flowey/flowey_lib_hvlite/Cargo.toml | Add toml_edit dependency for workspace-version parsing. |
| flowey/flowey_lib_common/src/publish_gh_release.rs | Enhance GH release publishing with notes mode, existing-release policy, and prerequisites gating. |
| flowey/flowey_lib_common/src/lib.rs | Export the new provenance attestation node. |
| flowey/flowey_lib_common/src/attest_build_provenance.rs | New node for generating GitHub artifact provenance attestations (or stubbing locally). |
| flowey/flowey_hvlite/src/pipelines/release_openvmm.rs | New workflow_dispatch-driven pipeline for OpenVMM source releases (build gate → attest → draft release). |
| flowey/flowey_hvlite/src/pipelines/mod.rs | Register the new release-openvmm pipeline CLI entrypoint. |
| flowey/flowey_hvlite/src/pipelines/checkin_gates.rs | Add the distro-build gate to the standard check-in gates pipeline. |
| ci-flowey/openvmm-pr.yaml | Add the distro-build job to the ADO PR pipeline. |
| Cargo.toml | Define canonical OpenVMM product version in [workspace.package]. |
| Cargo.lock | Record the new workspace-derived versions and dependency additions. |
| .github/workflows/openvmm-release.yaml | New autogenerated GitHub Actions workflow for manual source release publishing. |
| .github/workflows/openvmm-pr.yaml | Autogenerated updates to include the distro-build gate job in PR CI. |
| .github/workflows/openvmm-pr-release.yaml | Autogenerated updates to include the distro-build gate job in release-required PR CI. |
| .github/workflows/openvmm-ci.yaml | Autogenerated reshuffling to include distro-build and adjust internal artifact IDs. |
| .flowey.toml | Register the new GitHub workflow generation mapping for the OpenVMM release pipeline. |
Comment on lines
+64
to
+80
| pipeline.inject_all_jobs_with(move |job| { | ||
| job.dep_on(&cfg_common_params) | ||
| .dep_on(|_| flowey_lib_hvlite::_jobs::cfg_versions::Request::Init) | ||
| .dep_on( | ||
| |_| flowey_lib_hvlite::_jobs::cfg_hvlite_reposource::Params { | ||
| hvlite_repo_source: RepoSource::GithubSelf, | ||
| }, | ||
| ) | ||
| .gh_grant_permissions::<flowey_lib_common::git_checkout::Node>([( | ||
| GhPermission::Contents, | ||
| GhPermissionValue::Read, | ||
| )]) | ||
| .gh_grant_permissions::<flowey_lib_common::gh_task_azure_login::Node>([( | ||
| GhPermission::IdToken, | ||
| GhPermissionValue::Write, | ||
| )]) | ||
| }); |
Comment on lines
95
to
+99
| title, | ||
| files, | ||
| notes: flowey_lib_common::publish_gh_release::GhReleaseNotes::Text("TODO".into()), | ||
| draft: true, | ||
| // This job runs on every push to main, but the tag only |
Comment on lines
+68
to
+80
| // The ref selected in the workflow UI may be a branch or tag, | ||
| // but releases must come from committed mainline history. | ||
| flowey::shell_cmd!( | ||
| rt, | ||
| "git fetch --no-tags --unshallow origin +refs/heads/main:refs/remotes/origin/main" | ||
| ) | ||
| .run()?; | ||
| let reachable = std::process::Command::new("git") | ||
| .args(["merge-base", "--is-ancestor", "HEAD", "origin/main"]) | ||
| .current_dir(&path) | ||
| .status() | ||
| .context("failed to check whether the release commit is on main")?; | ||
| if !reachable.success() { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #4075, #4132, and #4139. Once those prerequisites merge, this diff is the release publication workflow.
The workflow is deliberately manual and source-only:
main.MAJOR.MINOR.PATCHworkspace version newer than every existingopenvmm-v*release tag.openvmm-v<VERSION>.Rerunning safely replaces an existing draft for the version. A published release or existing tag is immutable and causes the workflow to fail.
There is no push trigger, version-change trigger, release branch,
-devtransition, follow-up commit, or merge freeze.