Skip to content

README.md

Latest commit

 

History

History
658 lines (540 loc) · 25.9 KB

README.md

File metadata and controls

658 lines (540 loc) · 25.9 KB
Raw

Cloud images

This Makefile automates the process of building cloud images for hotstack deployments. The tasks are executed using the make utility.

Images

  • controller: A customized CentOS 9 Stream image with packages needed for the hotstack controller node. Built using diskimage-builder (DIB) with custom elements from the dib/ subdirectory.
  • microshift: A CentOS 9 Stream image with MicroShift packages installed from GitHub releases. Built using diskimage-builder (DIB) with the hotstack-microshift element. The image is intentionally unconfigured and requires cloud-init or manual setup to enable services and configure runtime settings.
  • blank: A minimal blank image used for virtual baremetal node disks with Redfish virtual BMC
  • nat64: A NAT64 appliance image built using ci-framework for IPv6-only environments
  • switch-host (experimental): A CentOS 9 Stream image with libvirt/qemu for running virtual network switches using nested virtualization
  • ceos: A CentOS 9 Stream image that runs Arista cEOS (containerized EOS) as a podman container with macvlan networking. Built using diskimage-builder (DIB) with the hotstack-ceos element
  • nxos: A CentOS 9 Stream image that runs Cisco NXOS (Nexus 9300v) as a nested KVM virtual machine with macvtap passthrough networking. Uses GNS3's custom OVMF firmware for UEFI boot. Built using diskimage-builder (DIB) with the hotstack-nxos element
  • vjunos: A CentOS 9 Stream image that runs Juniper vJunos-switch as a nested KVM virtual machine with macvtap passthrough networking. The unmodified vendor QCOW2 is launched by QEMU with a vmm-data USB config disk for initial Junos configuration. Built using diskimage-builder (DIB) with the hotstack-vjunos element
  • uefi-netboot: A minimal 10MB raw disk image with a GPT partition table and an EFI System Partition containing HotPxeChain.efi as the default bootloader (/EFI/BOOT/BOOTX64.EFI). HotPxeChain is a custom UEFI application (built from hot-pxe-chain/ using the edk2 SDK) that drives the firmware's native PXE boot stack: it connects all controllers, locates PXE Base Code protocol handles, performs DHCP (IPv4 first, IPv6 fallback), then TFTP-downloads and chainloads the network boot program (e.g. snponly.efi). This produces clean UEFI PXE client identifiers (PXEClient:Arch:00007) without iPXE fingerprints, allowing the DHCP server to distinguish the two boot stages. Intended for use as an OpenStack Nova rescue image to test the full PXE chain (HotPxeChain → DHCP → NBP download → iPXE script → kernel/ramdisk)

Download Pre-built Images

Pre-built images are available from GitHub releases. These are automatically built by CI workflows and can be downloaded instead of building locally.

Controller Image

# Download latest controller image
curl -L -O https://github.com/openstack-k8s-operators/hotstack/releases/download/latest-controller/controller-latest.qcow2

Blank Image

# Download latest blank image
curl -L -O https://github.com/openstack-k8s-operators/hotstack/releases/download/latest-blank/blank-image-latest.qcow2

NAT64 Appliance Image

The NAT64 appliance image is built and released in the openstack-k8s-operators-ci repository:

# Download latest NAT64 appliance image
curl -L -O https://github.com/openstack-k8s-operators/openstack-k8s-operators-ci/releases/download/latest/nat64-appliance-latest.qcow2

For specific versions, see the respective releases pages:

Building Images Locally

The following sections describe how to build images locally using the included Makefiles.

Variables

  • CONTROLLER_IMAGE_NAME: The name of the controller image file to be created (default: controller.qcow2).
  • CONTROLLER_IMAGE_FORMAT: The desired format for the controller image (default: raw). Set to qcow2 to skip conversion and keep the original DIB output format.
  • CONTROLLER_DIB_VENV: Path to the Python virtual environment for diskimage-builder (default: ~/controller-dib-venv).
  • CONTROLLER_DIB_WORKDIR: Working directory for DIB build artifacts and cache (default: .controller-build).
  • BLANK_IMAGE_NAME: The name of the blank image file to be created.
  • BLANK_IMAGE_FORMAT: The format for the blank image (default: raw). Can be set to qcow2 or any format supported by qemu-img.
  • BLANK_IMAGE_SIZE: The size of the blank image file in megabytes.
  • UEFI_NETBOOT_IMAGE_NAME: The name of the UEFI netboot image file (default: uefi-netboot.img).
  • UEFI_NETBOOT_IMAGE_SIZE: Total size of the disk image (default: 10M).
  • UEFI_NETBOOT_ESP_SIZE: Size of the EFI System Partition (default: 8M).
  • HOT_PXE_CHAIN_DIR: Path to the hot-pxe-chain/ source directory containing the UEFI application source and Containerfile (default: ./hot-pxe-chain).
  • HOT_PXE_CHAIN_CONTAINER_IMAGE: Container image tag used for the edk2 build environment (default: localhost/hot-pxe-chain-build:latest). Cached after first build.
  • NAT64_IMAGE_NAME: The name of the NAT64 appliance image file.
  • NAT64_IMAGE_FORMAT: The desired format for the NAT64 image (default: raw). Set to qcow2 to keep the original format.
  • NAT64_CIFMW_DIR: Directory where ci-framework will be cloned (default: .ci-framework).
  • NAT64_BASEDIR: Build directory for NAT64 appliance artifacts (default: .nat64-build).
  • SWITCH_HOST_IMAGE_URL: The URL to download the CentOS 9 Stream image for switch host.
  • SWITCH_HOST_IMAGE_NAME: The name of the switch host image file.
  • SWITCH_HOST_IMAGE_FORMAT: The desired format for the switch host image (default: raw). Set to qcow2 to keep the original format.
  • SWITCH_HOST_INSTALL_PACKAGES: A list of packages to install on the switch host image (includes libvirt, qemu-kvm, networking tools).
  • FORCE10_10_IMAGE: Path to Force10 OS10 image file (zip archive like OS10_Virtualization_10.6.0.2.74V.zip). Will be copied to /opt/force10_10/ in the image. (Experimental support only)
  • CEOS_IMAGE_NAME: The name of the cEOS image file to be created (default: ceos-switch-host.qcow2).
  • CEOS_IMAGE_FORMAT: The desired format for the cEOS image (default: qcow2). Set to raw to convert to raw format after building.
  • CEOS_DIB_VENV: Path to the Python virtual environment for diskimage-builder (default: ~/ceos-dib-venv).
  • CEOS_DIB_WORKDIR: Working directory for DIB build artifacts and cache (default: .ceos-build).
  • CEOS_TAR_IMAGE: Path to the Arista cEOS tar.xz image file (e.g., cEOS64-lab-4.35.3F.tar.xz). This is required when building the cEOS image.
  • NXOS_IMAGE_NAME: The name of the NXOS image file to be created (default: nxos-switch-host.qcow2).
  • NXOS_IMAGE_FORMAT: The desired format for the NXOS image (default: qcow2). Set to raw to convert to raw format after building.
  • NXOS_DIB_VENV: Path to the Python virtual environment for diskimage-builder (default: ~/nxos-dib-venv).
  • NXOS_DIB_WORKDIR: Working directory for DIB build artifacts and cache (default: .nxos-build).
  • NXOS_QCOW2_IMAGE: Path to the Cisco NXOS qcow2 image file (e.g., nexus9300v64.10.5.3.F.qcow2). This is required when building the NXOS image.
  • VJUNOS_IMAGE_NAME: The name of the vJunos image file to be created (default: vjunos-switch-host.qcow2).
  • VJUNOS_IMAGE_FORMAT: The desired format for the vJunos image (default: qcow2). Set to raw to convert to raw format after building.
  • VJUNOS_DIB_VENV: Path to the Python virtual environment for diskimage-builder (default: ~/vjunos-dib-venv).
  • VJUNOS_DIB_WORKDIR: Working directory for DIB build artifacts and cache (default: .vjunos-build).
  • VJUNOS_QCOW2_IMAGE: Path to the Juniper vJunos-switch qcow2 image file (e.g., vJunos-switch-26.2R1.7.qcow2). This is required when building the vJunos image.

Note: Raw format is required for cloud backends using Ceph, as Ceph cannot directly use qcow2 images for VM disks.

Targets

  • all: The default target that depends on controller, blank, and nat64.
  • clean: A target that removes the controller, blank, and NAT64 images, as well as build artifacts.
  • controller: Builds the controller image using diskimage-builder (DIB). Depends on controller_dib_setup, controller_dib_build, and controller_convert.
    • controller_dib_setup: Creates a Python virtual environment and installs diskimage-builder.
    • controller_dib_build: Builds the controller image using DIB with the configuration from dib/controller-image.yaml and the custom hotstack-controller element from dib/elements/.
    • controller_convert: Converts the image to the format specified by CONTROLLER_IMAGE_FORMAT (in-place conversion if raw).
    • controller_clean: Removes the controller image, virtual environment, and build artifacts.
  • microshift: Builds the MicroShift image using diskimage-builder (DIB). Depends on microshift_dib_setup, microshift_dib_build, and microshift_convert.
    • microshift_dib_setup: Creates a Python virtual environment and installs diskimage-builder.
    • microshift_dib_build: Builds the MicroShift image using DIB with the configuration from dib/microshift-image.yaml and the custom hotstack-microshift element from dib/elements/.
    • microshift_convert: Converts the image to the format specified by MICROSHIFT_IMAGE_FORMAT (in-place conversion if raw).
    • microshift_clean: Removes the MicroShift image, virtual environment, and build artifacts.
  • blank: A target that creates a blank image file of the specified size in the format specified by BLANK_IMAGE_FORMAT.
  • blank_clean: A target that removes the blank image file.
  • uefi-netboot: Builds HotPxeChain.efi in a container (edk2 SDK), then creates a minimal raw disk image with a GPT partition table and an EFI System Partition containing the compiled EFI application as /EFI/BOOT/BOOTX64.EFI. Requires podman, sgdisk, and mtools.
    • uefi-netboot_build_efi: Builds the edk2 container image and extracts HotPxeChain.efi from it.
    • uefi-netboot_disk: Creates the GPT+ESP disk image and copies HotPxeChain.efi onto the ESP.
  • uefi-netboot_clean: Removes the uefi-netboot image, extracted EFI binary, and the build container image.
  • nat64: A target that depends on nat64_setup, nat64_build, and nat64_convert.
    • nat64_setup: A target that clones ci-framework and sets up the molecule environment with required dependencies.
    • nat64_build: A target that builds the NAT64 appliance image using the build-nat64-appliance-image.yaml playbook and the ci-framework nat64_appliance role.
    • nat64_convert: A target that converts the image to the format specified by NAT64_IMAGE_FORMAT (in-place conversion if raw).
    • nat64_clean: A target that removes the NAT64 image and build artifacts.
  • switch-host (experimental): A standalone target that depends on switch-host_download, switch-host_customize, and switch-host_convert. Not included in all or clean targets.
    • switch-host_download: A target that downloads the base image from the specified URL.
    • switch-host_customize: A target that customizes the downloaded image by installing packages (libvirt, qemu, networking tools), copying helper scripts to /usr/local/bin/, installing the systemd service, and creating necessary directories.
    • switch-host_convert: A target that converts the image to the format specified by SWITCH_HOST_IMAGE_FORMAT (in-place conversion if raw).
    • switch-host_clean: A target that removes the switch-host image file.
  • nxos: Builds the NXOS switch-host image using diskimage-builder (DIB). Depends on nxos_verify_image, nxos_dib_setup, nxos_dib_build, and nxos_convert. Not included in all or clean targets.
    • nxos_verify_image: Validates that NXOS_QCOW2_IMAGE is set and exists.
    • nxos_dib_setup: Creates a Python virtual environment and installs diskimage-builder.
    • nxos_dib_build: Builds the NXOS image using DIB with the configuration from dib/nxos-image.yaml and the custom hotstack-nxos element from dib/elements/.
    • nxos_convert: Converts the image to the format specified by NXOS_IMAGE_FORMAT (in-place conversion if raw).
    • nxos_clean: Removes the NXOS image, virtual environment, and build artifacts.
  • vjunos: Builds the vJunos switch-host image using diskimage-builder (DIB). Depends on vjunos_verify_image, vjunos_dib_setup, vjunos_dib_build, and vjunos_convert. Not included in all or clean targets.
    • vjunos_verify_image: Validates that VJUNOS_QCOW2_IMAGE is set and exists.
    • vjunos_dib_setup: Creates a Python virtual environment and installs diskimage-builder.
    • vjunos_dib_build: Builds the vJunos image using DIB with the configuration from dib/vjunos-image.yaml and the custom hotstack-vjunos element from dib/elements/.
    • vjunos_convert: Converts the image to the format specified by VJUNOS_IMAGE_FORMAT (in-place conversion if raw).
    • vjunos_clean: Removes the vJunos image, virtual environment, and build artifacts.

MicroShift Image Variables

  • MICROSHIFT_IMAGE_NAME: The name of the MicroShift image file to be created (default: microshift.qcow2).
  • MICROSHIFT_IMAGE_FORMAT: The desired format for the MicroShift image (default: raw). Set to qcow2 to skip conversion and keep the original DIB output format.
  • MICROSHIFT_DIB_VENV: Path to the Python virtual environment for diskimage-builder (default: ~/microshift-dib-venv).
  • MICROSHIFT_DIB_WORKDIR: Working directory for DIB build artifacts and cache (default: .microshift-build).
  • MICROSHIFT_VERSION: MicroShift major.minor version for dependency resolution (default: 4.20). This determines which OpenShift mirror repository to enable and which GitHub release to auto-discover.
  • DIB_MICROSHIFT_RPM_ARCHIVE: Optional. Direct URL to the MicroShift RPM archive (tgz file) from a GitHub release. When not set, the latest release matching MICROSHIFT_VERSION is auto-discovered from the microshift-io/microshift GitHub releases. Example: https://github.com/microshift-io/microshift/releases/download/4.20.0_g153ff0ca9_4.20.0_okd_scos.16/microshift-rpms-x86_64.tgz

Examples

Cleanup

make clean

Building and uploading the controller image to glance

  1. Build the controller image (using diskimage-builder):

    make controller

    This will create a Python virtual environment, install diskimage-builder, build the image using the configuration from dib/controller-image.yaml, and convert it to raw format (default).

  2. Upload the controller image to Glance:

    openstack image create hotstack-controller \
  --disk-format raw \
  --file controller.qcow2

    Note: To build as qcow2 without conversion:

    make controller CONTROLLER_IMAGE_FORMAT=qcow2
openstack image create hotstack-controller \
  --disk-format qcow2 \
  --file controller.qcow2

Building and uploading the MicroShift image to glance

  1. Build the MicroShift image (using diskimage-builder):

    The latest RPM archive for the configured version is auto-discovered from GitHub releases:

    make microshift \
  MICROSHIFT_VERSION=4.20 \
  MICROSHIFT_IMAGE_FORMAT=raw  # Optional, defaults to raw

    To pin a specific release, set DIB_MICROSHIFT_RPM_ARCHIVE explicitly:

    make microshift \
  DIB_MICROSHIFT_RPM_ARCHIVE=https://github.com/microshift-io/microshift/releases/download/4.20.0_g153ff0ca9_4.20.0_okd_scos.16/microshift-rpms-x86_64.tgz \
  MICROSHIFT_VERSION=4.20

    This will create a Python virtual environment, install diskimage-builder, build the image using the configuration from dib/microshift-image.yaml, and convert it to raw format (default).

  2. Upload the MicroShift image to Glance:

    openstack image create hotstack-microshift \
  --disk-format qcow2 \
  --file microshift.qcow2 \
  --property hw_firmware_type=uefi \
  --property hw_machine_type=q35

  3. See dib/elements/hotstack-microshift/README.rst for detailed runtime configuration instructions, including firewall setup, LVM configuration for TopoLVM, kubeconfig setup, and service enablement.

Building and uploading the blank image to glance

  1. Create the blank image:

    make blank

  2. Upload the blank image to Glance:

    openstack image create sushy-tools-blank-image \
  --disk-format raw \
  --file blank-image.qcow2 \
  --property hw_firmware_type=uefi \
  --property hw_machine_type=q35 \
  --property os_shutdown_timeout=5

Building and uploading the uefi-netboot image to glance

  1. Create the uefi-netboot image:

    make uefi-netboot

    This will:

    • Build a container image with the edk2 SDK and compile HotPxeChain.efi (cached after first run via podman build)
    • Extract the compiled EFI application from the container
    • Create a 10MB GPT+ESP disk image
    • Copy HotPxeChain.efi as /EFI/BOOT/BOOTX64.EFI onto the ESP

  2. Upload the uefi-netboot image to Glance:

    openstack image create uefi-netboot \
  --disk-format raw \
  --file uefi-netboot.img \
  --tag hotstack \
  --property hw_firmware_type=uefi \
  --property hw_machine_type=q35 \
  --property hw_rescue_device=disk

    Note: The hw_firmware_type=uefi and hw_machine_type=q35 properties are required so that OVMF firmware is used when booting the rescue image.

  3. Rescue an instance and watch the console log:

    openstack server rescue --image uefi-netboot <instance-id>
openstack console log show <instance-id>

    In the console log, look for HotPxeChain: status messages showing controller connection, PXE handle discovery, DHCP progress, and NBP chainloading. The boot sequence is:

    HotPxeChain.efi (DHCP with PXEClient:Arch:00007)
  → snponly.efi (DHCP with iPXE user-class)
    → iPXE script → kernel + ramdisk

Building and uploading the NAT64 appliance image to glance

  1. Build the NAT64 appliance image:

    make nat64

    This will:

    • Clone the ci-framework repository (if not already present)
    • Setup the molecule environment with required dependencies
    • Run the build-nat64-appliance-image.yaml playbook using the nat64_appliance role
    • Copy the resulting image to nat64-appliance.qcow2

  2. Upload the NAT64 appliance image to Glance:

    openstack image create nat64-appliance \
  --disk-format raw \
  --file nat64-appliance.qcow2 \
  --property hw_firmware_type=uefi \
  --property hw_machine_type=q35

    Note: The NAT64 image build process requires the ci-framework and its dependencies. The build artifacts are stored in .nat64-build, the ci-framework clone is stored in .ci-framework, and a Python virtual environment is created at ~/test-python. All of these can be cleaned up with make nat64_clean.

Building and uploading the switch-host image to glance

  1. Build the switch-host image:

    make switch-host

    Or with vendor switch images pre-installed:

    make switch-host \
  FORCE10_10_IMAGE=/path/to/OS10_Virtualization_10.6.0.2.74V.zip

    This will:

    • Download CentOS 9 Stream image
    • Install libvirt, qemu-kvm, and networking tools
    • Copy helper scripts to /usr/local/bin/
    • Install systemd service for managing nested switch VMs
    • Create directories for each switch model under /opt/
    • Copy any provided vendor switch images to their respective directories

  2. Upload the switch-host image to Glance:

    openstack image create hotstack-switch-host \
  --disk-format raw \
  --file switch-host.qcow2 \
  --property hw_firmware_type=uefi \
  --property hw_machine_type=q35

    See switch-host-scripts/README.md for details on switch image requirements and configuration.

Building and uploading the SONiC Virtual Switch image to glance

  1. Build the SONiC image (using diskimage-builder):

    make sonic

    This will create a Python virtual environment, install diskimage-builder, build the image using the configuration from dib/sonic-image.yaml, and keep it in qcow2 format (default).

  2. Upload the SONiC image to Glance:

    openstack image create hotstack-sonic \
  --disk-format qcow2 \
  --file sonic-vs-switch-host.qcow2 \
  --property hw_firmware_type=uefi \
  --property hw_machine_type=q35 \
  --property hw_vif_model=e1000

    Note: The hw_vif_model=e1000 property is recommended for SONiC switches to avoid virtio checksum offloading issues when connected via OVS/OVN bridge networks.

    Note: To convert to raw format (required for Ceph backends):

    make sonic SONIC_IMAGE_FORMAT=raw

openstack image create hotstack-sonic \
  --disk-format raw \
  --file sonic-vs-switch-host.qcow2 \
  --property hw_firmware_type=uefi \
  --property hw_machine_type=q35 \
  --property hw_vif_model=e1000

  3. See dib/elements/hotstack-sonic-vs/README.rst for details on runtime configuration via cloud-init, including network interface setup and SONiC configuration.

Building and uploading the cEOS image to glance

  1. Build the cEOS image (using diskimage-builder):

    make ceos CEOS_TAR_IMAGE=/path/to/cEOS64-lab-4.35.3F.tar.xz

    This will create a Python virtual environment, install diskimage-builder, build the image using the configuration from dib/ceos-image.yaml, and keep it in qcow2 format (default).

  2. Upload the cEOS image to Glance:

    openstack image create hotstack-ceos \
  --disk-format qcow2 \
  --file ceos-switch-host.qcow2 \
  --property hw_firmware_type=uefi \
  --property hw_machine_type=q35 \
  --property hw_vif_model=e1000

    Note: The hw_vif_model=e1000 property is recommended for switch images to avoid virtio checksum offloading issues when connected via OVS/OVN bridge networks.

    Note: To convert to raw format (required for Ceph backends):

    make ceos \
  CEOS_TAR_IMAGE=/path/to/cEOS64-lab-4.35.3F.tar.xz \
  CEOS_IMAGE_FORMAT=raw

openstack image create hotstack-ceos \
  --disk-format raw \
  --file ceos-switch-host.qcow2 \
  --property hw_firmware_type=uefi \
  --property hw_machine_type=q35 \
  --property hw_vif_model=e1000

  3. See dib/elements/hotstack-ceos/README.rst for details on runtime configuration via cloud-init, including network interface setup and startup configuration.

Building and uploading the NXOS image to glance

  1. Build the NXOS image (using diskimage-builder):

    make nxos NXOS_QCOW2_IMAGE=/path/to/nexus9300v64.10.5.3.F.qcow2

    This will create a Python virtual environment, install diskimage-builder, download the GNS3 OVMF firmware, build the image using the configuration from dib/nxos-image.yaml, and keep it in qcow2 format (default).

  2. Upload the NXOS image to Glance:

    openstack image create hotstack-nxos \
  --disk-format qcow2 \
  --file nxos-switch-host.qcow2 \
  --property hw_firmware_type=uefi \
  --property hw_machine_type=q35 \
  --property hw_vif_model=e1000

    Note: The hw_vif_model=e1000 property is recommended for switch images to avoid virtio checksum offloading issues when connected via OVS/OVN bridge networks.

    Note: To convert to raw format (required for Ceph backends):

    make nxos \
  NXOS_QCOW2_IMAGE=/path/to/nexus9300v64.10.5.3.F.qcow2 \
  NXOS_IMAGE_FORMAT=raw

openstack image create hotstack-nxos \
  --disk-format raw \
  --file nxos-switch-host.qcow2 \
  --property hw_firmware_type=uefi \
  --property hw_machine_type=q35 \
  --property hw_vif_model=e1000

  3. See dib/elements/hotstack-nxos/README.rst for details on runtime configuration via cloud-init, including macvtap passthrough networking, NXOS interface mapping, and POAP bootstrap support.

Building and uploading the vJunos image to glance

  1. Build the vJunos image (using diskimage-builder):

    make vjunos VJUNOS_QCOW2_IMAGE=/path/to/vJunos-switch-26.2R1.7.qcow2

    This will create a Python virtual environment, install diskimage-builder, build the image using the configuration from dib/vjunos-image.yaml, and keep it in qcow2 format (default).

  2. Upload the vJunos image to Glance:

    openstack image create hotstack-vjunos \
  --disk-format qcow2 \
  --file vjunos-switch-host.qcow2 \
  --property hw_firmware_type=uefi \
  --property hw_machine_type=q35 \
  --property hw_vif_model=e1000

    Note: To convert to raw format (required for Ceph backends):

    make vjunos \
  VJUNOS_QCOW2_IMAGE=/path/to/vJunos-switch-26.2R1.7.qcow2 \
  VJUNOS_IMAGE_FORMAT=raw

openstack image create hotstack-vjunos \
  --disk-format raw \
  --file vjunos-switch-host.qcow2 \
  --property hw_firmware_type=uefi \
  --property hw_machine_type=q35 \
  --property hw_vif_model=e1000

  3. See dib/elements/hotstack-vjunos/README.rst for details on runtime configuration via cloud-init, including macvtap passthrough networking, vJunos interface mapping, and vmm-data config disk injection.