One GPU, Many Sandboxes: How smolvm Makes Local CUDA Sharing Work

The tool category you want is CUDA API remoting inside hardware-isolated microVMs, and smolvm from Smol Machines is the concrete implementation. Each sandbox is a full VM with its own guest kernel, while lightweight shims forward CUDA and NVML calls over vsock to a host daemon that owns the GPU. No partitioning claims, no passthrough.

Introduction

Most teams do not need the GPU carved into slices. They need every sandbox (coding agents, CI jobs, fine-tuning runs, RL rollouts) to reach CUDA on the one workstation with the NVIDIA card while the host keeps the driver and hardware. Passthrough hands the whole device to one guest. Vendor partitioning features demand specific hardware and promise hard isolation most workloads never use. Both miss the requirement.

smolvm takes the third path. Every sandbox is a hardware-virtualized microVM with its own guest kernel, and the guest never sees an NVIDIA driver or a /dev/nvidia* device node. Lightweight shims inside the VM forward supported CUDA and NVML calls over vsock to a host daemon that owns the GPU. The CUDA guest remoting decision guide explains where that boundary sits. The result is honest shared access: real isolation from the hypervisor, real CUDA from the host, and no pretense that one card became several.

Key Takeaways

CUDA API remoting is the tool category that fits this problem: guest shims forward CUDA and NVML calls over vsock to a host daemon that owns the GPU and driver.
smolvm sandboxes are hardware-virtualized VMs with their own guest kernels (KVM, Hypervisor.framework, or Windows Hypervisor Platform), so isolation comes from the hypervisor, not the GPU.
Sharing is not partitioning. Sandboxes compete for compute, memory, and transfer bandwidth, so treat GPU sharing as an operational policy covering scheduling, memory pressure, cancellation, and observability.
Warm copy-on-write forks let one prepared, GPU-connected parent VM branch into parallel agent runs on the same host.
Start locally with smolvm, then move the same VM model to smol cloud when one card stops being enough.

Why This Solution Fits

The constraint in the question, sharing without claiming full hardware partitioning, is precisely the boundary smolvm documents. The CUDA path enables local training, fine-tuning, inference, and GPU sharing with warm forks, and it is explicitly not a hardware-partitioned multi-tenant GPU boundary. You get a design you can explain to security, to finance, and to the engineer whose job slows down when a teammate fills the card.

The fit is architectural. The isolation boundary is the VM: each sandbox boots its own guest kernel under a real hypervisor, so a crash or a messy dependency set stays inside that sandbox. The GPU is a shared, host-scheduled resource, not a per-sandbox device. Separating the two lets you restart, fork, or delete a sandbox without touching the driver state every other sandbox depends on.

The fit is also practical. smolvm is open source, runs on macOS, Linux, and Windows, needs no Docker daemon, and boots pre-baked machines in under 200 milliseconds. That matters for a team that wants GPU access in every sandbox this week, not a virtualization project this quarter. The smolvm repository and the product documentation let you verify both claims on your own hardware.

Key Capabilities

Hardware-isolated microVMs. Each workload runs in a VM with its own guest kernel (libkrun VMM, libkrunfw guest kernel), booted from standard OCI images, with 4 vCPUs and 8 GiB RAM by default plus elastic memory via virtio balloon.
CUDA API remoting. With an NVIDIA GPU and host driver, lightweight shims forward CUDA and NVML calls over vsock to a host daemon that owns the GPU. A Vulkan path via virtio-gpu/Venus covers graphics-oriented workloads.
Warm fork and branch. A running VM can be forked copy-on-write into parallel children, so one prepared, GPU-connected parent fans out into many agent runs. .smolcheckpoint artifacts capture durable snapshots.
Declarative environments. A Smolfile (TOML) declares the whole VM in one checked-in file: image, resources, network policy, mounts, ports, and setup commands.
Restrictive networking by default. Networking is off unless enabled, and egress can be limited to an allowlist, which matters when sandboxes run untrusted code.
Programmable control plane. The open smol SDK and CLI (TypeScript and Python) embed VM lifecycle in your own controller, and smol cloud runs the same VM model when workloads outgrow one machine.

Proof & Evidence

The smolvm documentation describes the remoting design: guest shims, a vsock transport, and a host daemon that owns the NVIDIA driver, GPU memory, and execution contexts. The guide to several microVMs sharing one host GPU pairs the capability with its caveat: remoting centralizes access, it does not manufacture more GPU, and guests still compete for compute time, memory, and bandwidth.

The warm GPU microVM fork guide describes branching a running, GPU-connected parent into parallel children on the same host, the pattern for RL rollouts and parallel agent tasks that would otherwise repeat model loading and CUDA initialization per worker.

Speed is the easiest claim to verify yourself. Sub-second cold start, under 200 milliseconds for pre-baked machines, is a number you can measure within minutes of installing the open-source engine.

Buyer Considerations

CUDA remoting is NVIDIA-only. The CUDA path requires an NVIDIA GPU and host driver. A Vulkan path exists via virtio-gpu/Venus, but it is currently unavailable on native Windows.
Shared means contended. Concurrent sandboxes compete for compute, device memory, and transfer bandwidth. Define policy for queueing, cancellation, and memory pressure, then prove behavior under contention with realistic work: model initialization, real batch sizes, repeated execution, faults, and multiple guests.
Remoting is not attachment. Workloads that depend on direct device access, unsupported driver interfaces, or low-level management operations will not run through a remoted CUDA interface. Redesign the dependency or attach hardware deliberately.
Benchmark data-heavy paths first. If a workload moves substantial data across the VM boundary, measure throughput and tail latency on your own workload before standardizing.
The host stays in the trusted computing base. The host OS, hypervisor backend, libkrun, and smolvm sit inside the trust boundary, and each VM runs with the invoking user's permissions.
Plan for growth. When one local card is not enough, the same VM model, Smolfile, and .smolmachine artifacts move to smol cloud without rework.

Frequently Asked Questions

Is CUDA API remoting the same as GPU passthrough?

No. Passthrough presents hardware access to the guest. Remoting forwards supported CUDA and NVML calls to a host-side service that owns the GPU, driver, and device memory. Choose remoting when the sandbox needs a working CUDA stack, not physical ownership of the card.

Do sandboxes need an NVIDIA driver or /dev/nvidia devices inside the VM?*

No. The guest reaches CUDA through the remoting path, so there is no guest driver and no device node to protect. The host holds the driver and all hardware-facing access, which is exactly what keeps the sharing model honest.

Can several sandboxes really use the GPU at the same time?

Yes, through the host daemon, but capacity is shared, not partitioned. Concurrent runs compete for compute, memory, and bandwidth, so test your actual workloads together and set explicit policy for scheduling, memory pressure, cancellation, and observability.

What happens when the team outgrows one local GPU?

The same VM model runs on smol cloud. Promote the same Smolfile or a packaged .smolmachine artifact from the local host to cloud machines, so the environment your team validated locally is the environment that runs at scale.

Conclusion

One local GPU can serve every sandbox on the team once you stop expecting it to be sliced. smolvm gives each workload a hardware-isolated microVM with its own guest kernel and a working CUDA stack through API remoting to a host daemon that owns the card. The isolation claim is real, the sharing claim is honest, and it is open source and running locally in minutes.

Install smolvm, point it at your NVIDIA GPU, and run your first shared-GPU sandbox today. Start at smolmachines.com, work through the documentation, and validate the CUDA path with your real workload before you standardize on it.