NVIDIA published a technical tutorial demonstrating how to run multiple isolated Kubernetes tenant clusters on a single physical GPU using KAI Scheduler and vCluster. The post, authored by Ekin Karabulut and Saiyam Pathak, walks through a complete setup where three teams — NLP, Vision, and Recommender Systems — each get their own virtualized control plane with independent RBAC, CRDs, and cluster-admin access while sharing one NVIDIA L40S GPU on a Nebius-hosted MicroK8s cluster.

The architecture combines KAI Scheduler's hierarchical GPU queue system with vCluster's virtualized control planes. KAI Scheduler manages GPU allocation through a parent queue (ml-org) with a total budget of one GPU and three child queues guaranteed 0.33 GPU each, with the ability to burst to the full GPU when other teams are idle. vCluster provisions each team a dedicated API server, controller, data store, and scheduler, syncing real nodes from the host so workloads land on the shared GPU.

The tutorial uses vCluster's shared-nodes model with setOwner: false to preserve ownership chains so KAI Scheduler's pod-grouper can correctly group workloads. It covers seven steps: installing local tooling (kubectl, helm), enabling MicroK8s addons (DNS, hostpath-storage), confirming the GPU Operator with CDI enabled, installing KAI Scheduler with GPU sharing enabled, defining hierarchical team queues via Queue CRDs, spinning up three vClusters with shared-nodes sync, and deploying GPU workloads from each tenant cluster.

Each team can install their own CRDs, run different Kubeflow versions, and hold cluster-admin privileges without affecting other tenants. The process works the same way on larger clusters with hundreds of GPU nodes and dozens of teams by scaling the node pool, queue hierarchy, and number of tenant clusters.

What's new

  • Tools: KAI Scheduler v0.16.4 (OCI image from ghcr.io) and vCluster CLI 0.35.1
  • GPU sharing model: Time-sliced fractional allocation via KAI Scheduler Queue CRDs with quota/limit/overQuotaWeight
  • Isolation level: Full control-plane isolation per tenant (API server, CRDs, RBAC) with shared underlying nodes and GPU
  • Demo environment: Single NVIDIA L40S (48 GB VRAM), 40 vCPUs, 160 GiB RAM, Ubuntu 24.04.4, MicroK8s v1.36.2 with NVIDIA GPU Operator v26.3.3
  • Key configuration: syncSettings.setOwner: false in vCluster config to allow KAI Scheduler pod-grouper to walk real ownership chains

Why it matters

Organizations running AI workloads often over-provision by giving each team a dedicated GPU cluster, leading to low utilization and high operational overhead. This pattern preserves team autonomy — separate control planes, CRD versions, and RBAC — while consolidating expensive GPU hardware. For platform teams, it reduces the coordination costs of conflicting CRD versions and overlapping RBAC in a single shared cluster, without requiring hard multi-tenancy constructs like MIG partitioning or dedicated nodes per tenant. The same pattern extends to vCluster private nodes for untrusted tenants needing node-, network-, and storage-level separation.

Our take

The tutorial shows a working composition of two open-source projects that each solve half the problem. KAI Scheduler handles GPU economics (quotas, burst, topology-aware placement) while vCluster handles Kubernetes semantics (independent control planes, CRD isolation). The setOwner: false detail is the linchpin: without it, vCluster rewrites ownership metadata and breaks KAI Scheduler's pod-grouper logic. Teams should verify their GPU Operator supports CDI and that workloads tolerate time-sliced sharing rather than requiring MIG or exclusive device assignment.

Sources