A management plane for running GPU Kubernetes clusters on bare metal. NVIDIA Kubernetes® Engine (NVKE) provisions and operates isolated Kubernetes clusters on NVIDIA infrastructure, so platform teams can offer managed Kubernetes to their customers without building and running a control plane themselves.
Who this is for: NVIDIA Cloud Partners and platform operators who have GPU bare metal and want to hand out Kubernetes clusters on it, and engineers integrating against its management plane.
Preview: NVKE is available as a preview. APIs, configuration and behavior can change between releases, and we do not recommend NVKE for production workloads. See Preview status and known limitations.
The diagram shows the NVKE product family, how it sits on NICo and your GPU machines, and which components talk to each other.
NVKE is not a Kubernetes distribution and not a Cluster API flavour. It is a self-hosting factory for Kubernetes clusters: a throwaway seed cluster stands up a highly available management cluster, and the management plane on it then provisions and lifecycle-manages many isolated workload clusters on demand.
The loop is seed → pivot → mint:
- Seed — an ephemeral single-node cluster runs the controller, Temporal, and Postgres.
- Bring up — a durable workflow runs
kubeadm initagainst pre-provisioned hosts. - Pivot — the new cluster registers itself into its own database. Installing the controller onto it is a separate step today; it does not yet upgrade, cycle or scale itself.
- Tear down the seed.
- Mint tenant clusters on demand.
What distinguishes it from a generic CAPI distribution:
- Hard tenant isolation. Every tenant gets a dedicated
kubeadmcluster with its own control plane and etcd, not a namespace in a shared one. - Secure by default. TPM-attested node identity through SPIFFE/SPIRE, full disk encryption with TPM unlock, and etcd encryption at rest.
- Built for egress-only sites. Durable state, identity, and image serving are all local, so a site that cannot call out still works. Air-gapped installation is a first-class mode, not a workaround.
- Pluggable infrastructure. Bare metal is reached through the Cluster API contract, so the same control plane drives different backends.
| Directory | What it is |
|---|---|
controller/ |
The management plane. REST API, durable Temporal workflows, cluster and node inventory, and the infrastructure provider integrations. |
cluster-api-provider-provisioned/ |
Cluster API infrastructure provider for hosts that NVKE provisions before CAPI bootstraps them. |
agent/ |
The on-node daemon. Exposes a gRPC API the controller drives to run kubeadm, install a CNI, and configure disk and etcd encryption. |
cli/ |
nvkectl, the command-line interface. Validates a site spec and drives bootstrap. |
controller/nico-sim/ |
nico-sim, the NICo-compatible VM and Docker-container simulator used by development and tests. |
build/ |
Build and publish scripts per component. |
docs/ |
Architecture, getting started, and admin guides. |
os-image/ |
Reference Ubuntu node OS image (spec + Packer) and nvke-os-image-check. |
nktp/ |
NVKE telemetry pipeline (NKTP): a Helm chart that deploys an OpenTelemetry pipeline for cluster metrics, logs and traces. Versioned and released on its own. |
Twelve Go modules, tied together by a workspace at the repository root, each
independently buildable. make build links seven binaries: agent, updater,
controller, nvkectl, capd-reboot-controller, nvke-os-image-check and
management-engine.
See docs/go-workspace.md for workspace and dependency setup.
What each binary is, and where it ends up running:
| Binary | Module | What it does | Runs on |
|---|---|---|---|
management-engine |
management-engine/ |
Controller management engine for NVKE resources, communicating directly with the management Kubernetes API | An operator-owned management cluster, or a development host with kubeconfig |
controller |
controller/ |
The control plane: REST API, Temporal workflows, and the Kubernetes reconcilers | The management cluster |
agent |
agent/ |
gRPC server the control plane drives to run kubeadm, install a CNI, and configure encryption |
Every managed node |
updater |
agent/ |
Independent helper that replaces the node agent without depending on the agent process | Every managed node |
nvkectl |
cli/ |
Validates a site spec and drives bootstrap | Your machine |
nvke-os-image-check |
os-image/ |
Checks a node image against the contract in os-image/SPEC.md |
Your machine, or an image build |
capd-reboot-controller |
controller/hack/capd-reboot-controller/ |
Test helper that reboots CAPD machines | Test clusters |
Development works on Linux and on Apple Silicon. Most of CI is Linux amd64, so
that is the platform with the most coverage. Packaging and image builds also run
on Linux arm64 runners: deb.yml and agent-deb-publish.yml build the Debian
packages on ubuntu-24.04-arm, and image-nvcr.yml builds images on
linux-arm64-cpu16.
There is no macOS runner, so nothing in CI exercises macOS. The Flox environment
targets x86_64-linux, aarch64-linux and aarch64-darwin, which leaves Intel
macOS out.
The agent is built for Linux on both architectures, linux/amd64 and
linux/arm64, because that is where the agent runs. It is not built for macOS at
all.
The end-to-end path is the part that is amd64 only. backend-e2e.yml runs on
linux-amd64-cpu16, and the bundles controller/hack/ci/prepare-capd-bundles.sh
fetches are amd64 Linux binaries, so the CAPD end-to-end path does not run on
Apple Silicon.
Flox is offered, not required. Every target runs from whatever is on your PATH.
git clone https://github.com/dsx-ai-factory/nvke.git
cd nvkeBuild the repository from a clean checkout with access to public Go modules:
make buildThis compiles the workspace modules and writes the binaries to bin/.
Build and test the agent:
cd agent
make build
make test
make lintThe agent runs on cluster nodes, so make build cross-compiles it for Linux on
amd64 and arm64 and writes to agent/bin/linux_*. The tests and the lint run
against your own machine, but on macOS the binaries it produces will not.
Build and test the controller:
cd controller
make build # → ./bin/controller
make test # unit tests
make lintThe repository pins its toolchain with Flox in an
environment at the root that covers every module. flox activate gives you the
versions CI uses, including the GNU Make 4 that macOS lacks. It is optional, and
every target runs from whatever is on your PATH without it.
For building, testing and what to run before opening a pull request, see docs/development.md. For the end-to-end suites and the wider local setup, see docs/getting-started.md. docs/README.md indexes everything else.
The controller has its own guide in
controller/README.md. For a worked nvkectl input, see
cli/examples/sitespec.yaml, which describes a
site, and cli/examples/capd-cluster.yaml,
which requests a cluster.
| Tool | Version |
|---|---|
| Go | 1.26+ |
| Docker | 24+ |
| GNU Make | 4.0+ |
| Flox | optional, for a reproducible repository toolchain |
Running NVKE against real infrastructure additionally requires hosts already booted on an NVKE-compatible OS image with the agent installed. NVKE does not publish OS images; it publishes the recipes for building them.
| Document | Contents |
|---|---|
| docs/first-cluster.md | Three paths to a running cluster, driven by nvkectl |
| docs/architecture.md | System overview — modules, data flows, deployment model |
| docs/getting-started.md | Local development setup for the controller and the agent |
Module-specific documentation lives alongside the code, in controller/docs/,
agent/docs/ and the AGENTS.md files.
Start with CONTRIBUTING.md. Contributions are accepted by pull request; please open an issue first for anything substantial so the approach can be agreed before you write code.
Development quickstart:
git clone https://github.com/dsx-ai-factory/nvke.git && cd nvke
(cd agent && make build test)
(cd controller && make build test)- Code of Conduct: CODE_OF_CONDUCT.md
- Governance: GOVERNANCE.md
- Maintainers: MAINTAINERS.md
- Changelog: CHANGELOG.md
- Release process: RELEASE.md
Report vulnerabilities as described in SECURITY.md. Do not open a public issue for a security report.
See SUPPORT.md for support channels and expectations.
Questions and discussion: GitHub Discussions. Bugs, feature requests and documentation gaps: GitHub Issues.
Kubernetes® is a registered trademark of The Linux Foundation in the United States and other countries. NVIDIA and the NVIDIA logo are trademarks and/or registered trademarks of NVIDIA Corporation in the U.S. and other countries. Other company and product names may be trademarks of the respective companies with which they are associated.
Apache License 2.0 — see LICENSE.
Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
Copies of LICENSE are kept inside the two Helm charts so that the packaged charts
ship them. They are identical to the root file, are regenerated with
make licenses, and are verified by make licenses-check:
| Directory | License | Path | Why it is here |
|---|---|---|---|
controller/deploy/charts/controller/ (Helm chart) |
Apache-2.0 | controller/deploy/charts/controller/LICENSE | Helm packages only files inside the chart directory. |
nktp/deploy/charts/nktp/ (Helm chart) |
Apache-2.0 | nktp/deploy/charts/nktp/LICENSE | Helm packages only files inside the chart directory. This chart keeps its own NOTICE. |