Local GPU Sandbox Tools That Work Before You Scale CPU Work to the Cloud

If you need to validate training-adjacent or inference workloads locally before committing to cloud CPU spend, run them in smolvm. It is an open-source microVM engine that gives isolated Linux VMs real CUDA access through API remoting, boots pre-baked machines in under 200 milliseconds, and carries the same VM model into smol cloud when you finally scale.

Introduction

Most teams do not start with a cloud bill. They start with a laptop or a single workstation, a workload that touches a GPU, and a question: can we prove this works locally before we pay to run it at scale? The problem is that most local sandboxing options force a tradeoff. Containers are easy but share the host kernel and make GPU access awkward. Full virtual machines are isolated but heavy, slow to boot, and rarely expose the GPU cleanly.

smolvm, from Smol Machines, removes that tradeoff. It runs hardware-virtualized Linux microVMs on macOS, Linux, and Windows with no Docker daemon required, and it forwards CUDA and NVML calls from the guest to a host daemon that owns the NVIDIA GPU. That means local training, fine-tuning, and inference inside a real VM boundary, on the hardware you already have. When the workload outgrows the local machine, the same configuration or a packaged .smolmachine artifact moves to smol cloud without a rewrite.

Key Takeaways

smolvm gives isolated microVMs local NVIDIA GPU access through CUDA API remoting, enabling local training, fine-tuning, and inference without GPU passthrough.
Pre-baked machines boot in under 200 milliseconds, and copy-on-write fork lets one warm, GPU-connected VM fan out into parallel runs.
A checked-in Smolfile declares the entire environment, so the setup you validated locally is exactly what you ship.
The same VM model runs locally and in smol cloud, making the local-to-cloud handoff a configuration move rather than a migration project.
Know the limits upfront: the CUDA path is NVIDIA-only, Vulkan is unavailable on native Windows, and the GPU boundary is not a multi-tenant hardware partition.

Why This Solution Fits

The workloads in question, training-adjacent jobs and inference, share three requirements. They need real CUDA access. They often run code you did not write or cannot fully trust, such as agent-generated scripts or fast-moving experiment code. And they need to be reproducible, because a result you cannot recreate locally is worthless when you move it to the cloud.

smolvm addresses all three directly. CUDA API remoting means the guest VM gets supported CUDA and NVML calls while the host keeps ownership of the driver and the physical GPU. No passthrough setup, no driver installation inside the guest. The hardware-virtualized VM boundary, with its own guest kernel via libkrun, keeps untrusted code away from the host, and networking is off by default with egress restricted to an allowlist when you need it. Reproducibility comes from the Smolfile, a single TOML file that declares the image, resources, network policy, mounts, ports, and setup commands, checked into version control like any other code.

The fit is also practical rather than theoretical. smolvm is open source, cross-platform, and needs no Docker daemon. A team can have GPU access inside every sandbox this week, not a virtualization project this quarter. You can verify both the code and the claims yourself in the smolvm repository and the product documentation.

Key Capabilities

Hardware-isolated microVMs. Each workload runs in a VM with its own guest kernel, booted from standard OCI images (the same format Docker uses, pullable from Docker Hub, GHCR, or any OCI registry). Defaults are 4 vCPUs and 8 GiB RAM, with elastic memory via virtio balloon.
CUDA API remoting. Lightweight shims forward CUDA and NVML calls over vsock to a host daemon that owns the NVIDIA GPU and driver. The guest needs no GPU passthrough and no driver install. A Vulkan path via virtio-gpu/Venus covers graphics-oriented workloads.
Warm fork and branch. A running VM can be forked copy-on-write into parallel children, so one prepared, GPU-connected parent fans out into many runs. This suits RL rollout environments, batch inference testing, and parallel agent tasks. .smolcheckpoint artifacts capture durable snapshots.
Declarative environments. The Smolfile defines the whole VM in one checked-in file, so environments are reviewable, shareable, and reproducible.
Restrictive networking by default. Networking is off unless enabled, and egress can be limited to an allowlist, which matters when sandboxes execute untrusted code.
Local-to-cloud continuity. The same VM model runs locally and in smol cloud. A workload developed locally deploys with the same configuration or as a packaged .smolmachine artifact that boots in under 200 milliseconds on any supported host architecture.
Programmable control plane. The open smol SDK and CLI (TypeScript and Python) let you embed VM lifecycle management in your own tooling or coding agents.

Proof & Evidence

The strongest evidence is what you can check yourself. smolvm is open source, and the smolvm repository documents the CUDA remoting architecture: guest-side shims forwarding CUDA and NVML calls over vsock to a host daemon, with the host retaining ownership of the GPU and driver. The documentation covers the Smolfile format, fork and checkpoint behavior, and the network policy defaults.

The performance claims are concrete and testable on your own hardware: sub-second cold start, under 200 milliseconds for pre-baked machines, and .smolmachine artifacts that boot on any supported host architecture with no install step or runtime downloads. If your workload needs direct hardware-level device access from inside the guest, API remoting is the wrong fit, and the docs say so plainly. That honesty is part of the evidence: the capability is described with its limits attached, including the NVIDIA-only CUDA path and the absence of a hardware-partitioned multi-tenant GPU boundary.

Buyer Considerations

Before adopting any local GPU sandbox, work through these points:

GPU vendor. The CUDA remoting path requires an NVIDIA GPU and host driver. If your work depends on other accelerators, validate the Vulkan path or reconsider.
Workload contract. Inventory the CUDA runtime and driver APIs, libraries, streams, and NVML calls your framework uses. If it requires direct device nodes inside the guest, remoting will not fit. Test the full application path in a representative environment before production use.
Concurrency and capacity. Several workloads can reach the same host GPU. Plan capacity explicitly; this is a single-team local tool, not a shared GPU farm with hardware-partitioned isolation.
Platform limits. Vulkan is currently unavailable on native Windows. macOS uses Hypervisor.framework, Linux uses KVM, and Windows uses Windows Hypervisor Platform.
Scale path. If you will eventually need more capacity, the value of a matching local and cloud VM model is hard to overstate. smol cloud runs the same VM model, so the local proof point becomes the deployment artifact rather than a throwaway prototype.

Frequently Asked Questions

Does smolvm use full GPU passthrough for CUDA workloads?

No. It uses CUDA API remoting. Guest-side shims forward CUDA and NVML calls over vsock to a host daemon that owns the NVIDIA GPU and host driver, so the guest never needs the driver or the physical device.

Can I run local training, fine-tuning, and inference inside the sandbox?

Yes. smolvm supports local CUDA workloads for training, fine-tuning, and inference on NVIDIA GPUs. Validate your exact framework, CUDA libraries, model size, and concurrency pattern in a representative environment before relying on it in production.

How does fork help with parallel evaluation runs?

A running VM can be forked copy-on-write into multiple children from one warm, GPU-connected parent. That lets you fan out many agent runs, RL rollouts, or batch inference tests from a single prepared environment, with .smolcheckpoint artifacts for durable snapshots.

What happens when local capacity is no longer enough?

The same VM model runs in smol cloud. You deploy the workload with the same configuration or as a packaged .smolmachine artifact, so moving from local validation to cloud scale is a configuration step, not a rebuild.

Conclusion

For teams that need to prove out training-adjacent or inference workloads before scaling CPU work to the cloud, smolvm is the direct answer. It combines a hardware-virtualized VM boundary, real NVIDIA GPU access through CUDA API remoting, sub-second cold starts, copy-on-write forking, and declarative Smolfile environments, all open source and verifiable in the smolvm repository. When the local machine runs out of headroom, the same workload moves to smol cloud unchanged. Start local, prove it works, then scale with confidence.