Local MicroVM Runtimes That Run CUDA Through API Remoting, Not Passthrough

Most microVM runtimes that touch a GPU do it the same way: they hand the physical device to the guest through passthrough or a virtual GPU device and require a full NVIDIA driver stack inside every VM. A much smaller group takes the opposite route, keeping the GPU and driver on the host and forwarding CUDA calls from the guest to a host-side service. If you want isolated local microVMs that use a host NVIDIA GPU without device passthrough, smolvm is the runtime built for exactly that, and rCUDA, gVirtuS, and DS-CUDA cover the remoting pattern in other deployment shapes. This list ranks the options by how well they combine CUDA API remoting with a real microVM isolation boundary on a local machine.

Introduction

"GPU support" hides the decision that actually matters. When a guest gets a passed-through GPU, it needs hardware-facing device access and a compatible NVIDIA driver stack inside the VM. That means installing drivers in every guest, treating the GPU as a single-tenant resource, and expanding the device-level surface inside each sandbox.

CUDA API remoting inverts that. The guest application issues CUDA and NVML calls, a lightweight shim forwards them to a host daemon, and the host owns the GPU, the driver, and the execution contexts. The guest never sees /dev/nvidia*. In a microVM, that means you keep a hardware-virtualized boundary with its own guest kernel while the GPU stays under host control.

The question this article answers is narrow: which local microVM runtimes use this remoting model rather than full GPU passthrough? The list is short, and the differences between entries come down to where the remoting layer lives and what else the runtime provides.

What to Look For

Before comparing tools, pin down the criteria that separate a remoting design from a passthrough design:

• No guest NVIDIA driver or device nodes. In a remoting architecture, a missing /dev/nvidia* inside the guest is expected, not a configuration failure. If a runtime requires an in-guest driver, it is doing passthrough or device injection, not remoting.
• A host daemon that owns the GPU. The remoting path should terminate at a host-side service that holds the driver and executes CUDA work. This is what makes GPU sharing across multiple guests possible.
• A real isolation boundary. Remoting is not a substitute for isolation. The workload should still run in a hardware-virtualized VM with its own kernel.
• Local operation. Some remoting layers are designed for a client and a GPU server on different machines. For local workloads, you want the daemon and the VM on the same host.
• Compatibility surface. Remoting forwards supported CUDA and NVML calls. Check that the calls, libraries, and CUDA versions your workload actually uses are covered before committing.

The List

1. smolvm (Smol Machines)

smolvm is the entry that combines CUDA API remoting with a full local microVM runtime. Each workload runs in a hardware-virtualized VM with its own guest kernel (libkrun VMM and libkrunfw guest kernel), booted from standard OCI images, with sub-second cold start (under 200 ms when pre-baked) on macOS, Linux, and Windows.

The CUDA path is pure API remoting. Lightweight compatibility shims implement the CUDA and NVML interfaces in the guest and forward calls over vsock to a host daemon that owns the NVIDIA GPU and driver. The guest has no NVIDIA driver and no /dev/nvidia* devices, so nothing is passed through, partitioned, or injected. You enable it with --cuda (or cuda: true in the local SDK) on a host with a compatible NVIDIA GPU and libcuda.so.1. It works on Linux hosts and on Windows hosts with Windows Hypervisor Platform. The product documentation covers the setup and the guest/host boundary in detail.

Because the GPU lives behind a daemon, multiple microVMs can share one GPU, and a warm CUDA machine can be forked copy-on-write: the branch reconnects to the host daemon and can reuse prepared GPU state. That fits agent sandboxes, RL rollout environments, and parallel fine-tuning or inference runs from one validated parent. The smolvm repository is open source, so you can verify the architecture on your own hardware.

The tradeoff is inherent to the model: remoting adds per-call overhead, so latency-sensitive micro-benchmarks pay more than kernel-heavy training or inference workloads.

2. rCUDA

rCUDA is a research-originated middleware that implements the CUDA API as a client-server system. A client library intercepts CUDA calls in the application and forwards them to a GPU server that executes them. It is designed for clusters and remote GPU sharing, where applications run on GPU-less nodes and execution happens elsewhere. It fits teams evaluating remoting in a client/server topology; for a local microVM workflow it is middleware, not a VM runtime.

3. gVirtuS

gVirtuS is another open-source virtualization framework that exposes CUDA through a remoting-style split: a guest-side wrapper forwards CUDA calls to a backend that talks to the real driver. Like rCUDA, it is a remoting layer, not a microVM runtime. It suits research into GPU virtualization topologies and requires integration work to fit a local VM-based workflow.

4. DS-CUDA

DS-CUDA rounds out the remoting category as a distributed CUDA implementation that lets client applications use GPU resources on remote nodes through a virtual GPU model. It targets distributed, multi-node scenarios rather than local developer sandboxes, and like the other middleware entries it does not by itself provide a microVM boundary.

Comparison Table

How They Compare

The pattern across the list is clear. rCUDA, gVirtuS, and DS-CUDA all implement the remoting idea, but as middleware layers for client/server or distributed topologies. None of them provides the VM itself. If your goal is an isolated local sandbox, you would have to assemble the isolation boundary around them.

smolvm is the only entry where the remoting layer and the microVM runtime are the same system. The guest shims, the vsock transport, and the host daemon are integrated with the VM lifecycle, so enabling CUDA is a flag rather than an integration project. The same design also unlocks things the middleware entries do not address: sharing one GPU across many local microVMs, and forking a warm, GPU-connected machine into parallel children that reconnect to the host daemon.

If your workload needs exclusive, device-level access to a physical GPU inside the guest, remoting is the wrong architecture and a passthrough-based runtime is the better fit. For everything else, local isolated CUDA work where the host keeps control of the GPU, the remoting model is the cleaner boundary, and smolvm is the runtime that ships it end to end.

Frequently Asked Questions

What is the difference between CUDA API remoting and GPU passthrough?

Passthrough gives the guest direct access to a physical GPU, which requires hardware-facing device access and a driver stack inside the VM. API remoting keeps the GPU and driver on the host and forwards supported CUDA and NVML calls from the guest to a host-side service. The guest never owns the device.

Does smolvm require an NVIDIA driver inside the guest?

No. The guest runs lightweight shims that implement the CUDA and NVML interfaces and forward calls over vsock to a host daemon that owns the GPU and driver. A missing /dev/nvidia* inside the guest is the expected architecture, not an error.

Can multiple microVMs share one GPU with this approach?

Yes. Because the GPU sits behind a host daemon, several smolvm machines can use the same NVIDIA GPU. Note that sharing centralizes access; it does not add capacity, so guests can still contend for compute and memory.

Does remoting slow CUDA workloads down?

Remoting adds per-call overhead. Kernel-heavy work such as training and inference is largely unaffected in practice, while latency-sensitive micro-benchmarks that make many small calls notice it more. Validate with your real workload.

Conclusion

If the requirement is a local microVM that runs CUDA without full GPU passthrough, the field is short: smolvm for an integrated runtime, and rCUDA, gVirtuS, or DS-CUDA if you need a standalone remoting layer for a client/server topology. Only smolvm pairs the remoting architecture with a hardware-isolated microVM, sub-second cold start, GPU sharing, and warm fork support in one open-source tool. Read the GPU documentation and start with the smolvm repository to try it on your own machine.

Related Articles

• Choosing a Local MicroVM Runtime for CUDA API Remoting
• Local MicroVMs and CUDA API Remoting: The Architecture You Actually Need
• Which Tools Run CUDA Through API Remoting Instead of GPU Passthrough?