NVIDIA released CUDA Toolkit 13.4 on September 9, 2026, adding native Windows on Arm support and early developer access to the Rubin GPU architecture. The release extends CUDA development beyond Linux on Arm platforms and introduces preview functional support for Rubin (compute capability 107), allowing developers to begin porting applications before general availability in a future toolkit release.

The toolkit also delivers Multi-Process Service V3 with a scriptable CLI, named server instances, TOML configuration, and cgroup-integrated GPU memory limits for precise GPU partitioning in containerized environments. CUDA Compute Fabric Transport adds a transport-centric API for communication-library developers to move data across NVLink fabric using named logical endpoints and asynchronous operations.

Confirmed

  • Windows on Arm support: native Arm64 compiler and build-system support; cross-compilation from Windows x86-64 toolkit; local Arm64 installer and binary archives with headers, import libraries, runtime components, and linking libraries.
  • Rubin architecture preview: functional support for compute capability 107 (SM_107) as a preview; developers can begin porting before GA in a future release.
  • MPS V3: scriptable CLI, named server instances, namespaces, TOML configuration, SM partition controls, cgroup-integrated GPU memory limits; integrates with containerized environments.
  • CUDA Compute Fabric Transport (CFT): transport-centric API for NVLink fabric; named logical endpoints with endpoint ID and offset; asynchronous put, get, and reduction operations; unicast and multicast patterns; completion and error status reporting; available only through CUDA Driver API for communication-library developers.
  • Locality domains: programmatic access to GPU portions containing SMs and device memory; allocate device memory in a locality domain and create green context with SM resources in the same domain.
  • Unified memory residency queries: cudaMemGetLocationInfo API for querying residency information for managed or system-allocated data.
  • Driver decoupling: CUDA SDK installers no longer bundle the NVIDIA driver (Windows since CUDA 13.1, Linux since CUDA 13.4); install nvidia-open driver or cuda-toolkit packages separately.
  • Coherent Driver-based Memory Management (CDMM) default on coherent platforms (Grace Hopper, Grace Blackwell, Vera Rubin); NUMA mode remains supported via kernel module parameter.
  • Host compiler compatibility: GCC 16 and Clang 22 on supported host platforms.
  • cuda.core 1.1.0: texture and surface programming (cuda.core.texture module with OpaqueArray, MipmappedArray, TextureObject, SurfaceObject); NUMA-aware managed memory (ManagedMemoryResource.allocate returns ManagedBuffer with property-based interface for CUDA memory advice; Host type for host memory, NUMA node, or calling thread's NUMA node);.pyi type stubs for every public API; CUDA graph workflow improvements (GraphBuilder.graph_definition exposes captured graph as GraphDefinition); device-specific NVLink enumeration, expanded green-context workqueue configuration, path-like inputs for Program and ObjectCode, public Buffer.size property; strengthened IPC validation, free-threaded Python correctness, CUDA process checkpoint restoration.
  • cuda.compute 1.1: ahead-of-time compilation of CCCL algorithms for multiple GPU architectures; ProxyArray and ProxyValue describe argument types without allocating device memory; serialize() creates deployable artifact; deserialize() restores algorithm without recompiling and loads build matching current GPU architecture.
  • CCCL 3.4: faster warp-specialized cub::DeviceScan on Blackwell GPUs reaching up to 92% memory-bandwidth utilization (per vendor internal testing); single-call APIs across CUB device-wide algorithms; batched warp reductions; C++ Standard Library parallel algorithms in cuda::std.
  • Component versions: CUDA Toolkit 13.4 GA is versioned 13.4.1 (supersedes 13.4.0 developer preview); cuBLAS 13.7.0.27; cuFFT 12.4.0.34; cuSOLVER 12.3.2.15; cuSPARSE 12.8.6.49; NPP 13.2.0.35; nvJPEG 13.2.2.35; Nsight Compute 2026.3.0.13; Nsight Systems 2026.3.2.313; Nsight Visual Studio Edition 2026.3.0.26187.
  • Supported platforms: Linux and Windows on x86_64, arm64-sbsa, and arm64 (Windows).

Unknown

  • Exact general availability date for Rubin architecture support beyond "future release" of CUDA Toolkit.
  • Independent replication of the 92% memory-bandwidth utilization claim for cub::DeviceScan on Blackwell GPUs (vendor-reported figure).
  • Production readiness timeline for Windows on Arm preview; known issues include CUDA transfers using pageable memory performing below expectations, some PyTorch build workflows triggering GPU timeouts or system restarts, display blanking for ~2 minutes during driver installation, and Nsight Copilot unavailable on Windows Arm64 in this release.
  • Firm RTX Spark launch date; NVIDIA developer materials point to fall 2026 launch but no firm date confirmed.
  • Whether Arm64EC adoption will be widespread enough to make the native Windows on Arm CUDA ecosystem viable at RTX Spark launch.

Our take

Shipping the Windows on Arm toolchain months before RTX Spark hardware arrives is the right sequence — previous Windows on Arm efforts stalled because apps weren't ready at launch. The Arm64EC option is pragmatic: it lets teams keep x64 dependencies while migrating incrementally. But the preview's rough edges (PyTorch build crashes, pageable-memory performance gaps) mean early adopters are effectively stress-testing the stack. The real test isn't whether CUDA compiles on Arm Windows today; it's whether the ecosystem produces enough native Arm64/Arm64EC applications by the time RTX Spark ships to make the platform credible.

Series: 1. Nvidia Commits Up to $3 Billion to Lancium, Powering the Stargate AI Campus in Texas · 2. NVIDIA ships CUDA Toolkit 13.4 with Windows on Arm support and Rubin architecture preview · NVIDIA AI Factory

Sources