A Driverless CUDA Guest Is Not Broken: How Host GPU Remoting Works and When to Choose It

A CUDA guest with no NVIDIA driver and no /dev/nvidia* device files is not attached to the GPU at all. It is using CUDA API remoting: shims in the guest forward CUDA and NVML calls to a host daemon that owns the GPU, driver, and device nodes, then return results as if the GPU were local.

Introduction

Seeing nvidia-smi fail and no /dev/nvidia0 or /dev/nvidiactl inside a VM while a training job happily allocates tensors feels contradictory. The usual mental model says CUDA requires a kernel driver and device files in the same machine as the application. That model is incomplete. In an API-remoting architecture, the guest is only a CUDA client. A lightweight compatibility layer implements the CUDA and NVML interfaces in the guest and transports supported calls to the side that actually owns the hardware. The physical GPU, the NVIDIA kernel driver, and the device files stay on the host. The question to ask is not "where are the device files?" but "where do CUDA calls execute, and which APIs does the remoting layer support?"

This article explains what that means in practice, why it is often the right design for isolated workloads, and what to check before you commit to it. Smol Machines implements this model inside hardware-isolated microVMs, and we will use it as the concrete reference throughout.

Key Takeaways

A missing NVIDIA driver and missing /dev/nvidia* nodes in a guest are expected in a remoting setup, not a bug. They signal that the guest was never given direct device access.
In API remoting, the host daemon owns the GPU, driver, and device files; the guest runs client-side shims that forward supported CUDA and NVML calls (over vsock in Smol Machines' case).
This is a different model from GPU passthrough, vGPU, or device injection, which expose a GPU resource locally inside the guest.
Remoting gives shared access to a host-owned GPU, not dedicated or hardware-partitioned capacity. Scheduling and capacity limits still matter.
Some driver-level tools and profiling software that introspect the device directly may not work through a remoted API, so validate your real workload first.

Why This Solution Fits

If you run untrusted or semi-trusted code, coding agents, CI jobs, or multi-tenant agent workloads, you want two things at once: strong isolation for the workload and convenient access to a host GPU. Those goals conflict when you hand the guest a device. Passthrough and device injection punch a hole in the isolation boundary and tie a workload to specific hardware.

Driverless CUDA remoting resolves the conflict. The guest keeps a hardware-virtualized boundary with its own kernel, while GPU access flows through a narrow, host-controlled path. Nothing is passed through, partitioned, or injected into the guest, so there is no in-guest driver to install, no kernel module to match against the host, and no device node to manage. On a host with a supported NVIDIA GPU and libcuda.so.1, you enable the path with --cuda (or cuda: true in the local SDK).

It also fits teams that move workloads between machines. Because the guest needs no GPU-specific hardware or driver, the same image boots on a GPU host, a non-GPU host, or a cloud machine without reconfiguration. That is exactly the local-to-cloud continuity Smol Machines is built around: the same VM model runs locally with smolvm and in smol cloud, with portable artifacts in between.

Key Capabilities

The remoting model, as implemented in Smol Machines microVMs, gives you:

CUDA and NVML API remoting. Lightweight shims in the guest implement the CUDA and NVML interfaces and forward calls over vsock to a host daemon that owns the NVIDIA GPU and driver. Results return to the guest as if the GPU were local.
A real isolation boundary around GPU work. Each workload runs in a hardware-virtualized VM with its own guest kernel (libkrun VMM and libkrunfw guest kernel), booted from standard OCI images. Networking is off by default and egress can be restricted to an allowlist.
Sub-second starts. Cold start is under 200 ms when pre-baked, so GPU-backed agent tasks and CI jobs can be spawned per-run instead of kept warm.
Fork and branch for GPU sharing. A running VM can be forked copy-on-write, letting parallel agent runs share one warm, GPU-enabled environment.
Local and cloud parity. The same configuration or a packaged .smolmachine artifact runs locally or on smol cloud, so a CUDA workload developed on a laptop can move to a GPU host without changes.

The product documentation covers the setup and the guest/host boundary in detail.

Proof & Evidence

The architecture is verifiable from the guest itself. In a remoting setup you will find:

No NVIDIA kernel module loaded in the guest (lsmod shows nothing NVIDIA-related).
No /dev/nvidia0, /dev/nvidiactl, or related device nodes.
CUDA calls that still succeed, because the client shim forwards them to the host daemon.

That combination is the signature of remoting rather than a misconfiguration. If you expected passthrough or a locally installed driver, the missing devices indicate you are on a different architecture, not that something failed.

The design is documented publicly: the smolvm repository and Smol Machines docs describe the vsock-based guest/host split, and the Smol Machines site describes the broader microVM model, including OCI images, sandboxing defaults, and fork/branch.

Buyer Considerations

Before choosing a driverless remoting setup, weigh the following:

API coverage. Remoting forwards supported CUDA and NVML calls. Check that the calls, libraries, and CUDA versions your workload actually uses are covered. A simple "is CUDA available" check is not enough.
Workload compatibility. Some profiling tools, driver-level features, and software that introspects the device directly may not work through a remoted API. Test the exact application and versions you plan to deploy, including allocations, copies, streams, synchronization, error handling, and cleanup.
Shared capacity, not dedicated hardware. Remoting provides access to a host-owned GPU service. It is not hardware partitioning, so capacity controls and scheduling remain your responsibility when guests share the device.
Host requirements. The host needs a supported NVIDIA GPU and driver, since the host daemon owns the hardware. The guest needs neither.
Data movement. Host-to-device copies cross the remoting boundary. Workloads with very large or latency-sensitive transfers deserve a benchmark against your expectations.

If your workload genuinely requires direct device access in the guest, remoting is the wrong tool; look at passthrough-style options instead and accept the isolation trade-offs they bring.

Frequently Asked Questions

Is a missing /dev/nvidia* device in a CUDA guest a bug?

Not in a remoting setup. It is an architectural signal that the guest was never given direct device access. It only indicates a problem if you expected passthrough or a locally installed driver.

Where does the CUDA kernel actually execute?

On the host GPU. The guest application initiates the supported CUDA call, the client shim forwards it to the host daemon, and the host side interacts with the NVIDIA driver and hardware. Results return to the guest as if the GPU were local.

Does API remoting give each guest a dedicated GPU?

No. It provides access to a host-owned GPU service. It is not a claim of dedicated capacity or hardware-enforced partitioning, so scheduling and capacity controls are still needed when guests share the device.

What should I test before putting this into production?

Test the precise application and versions you will deploy. Cover the CUDA interfaces and libraries you use, allocations, copies, streams, synchronization, errors, cleanup, data sizes, concurrent workloads, and expected latency.

Conclusion

A CUDA guest with no NVIDIA driver and no /dev/nvidia* nodes is not broken; it is remoted. The guest provides the API surface, and a GPU-equipped host provides the hardware, driver, and device files. For isolated local workloads, driverless remoting is the cleanest way to combine a strong VM boundary with real GPU access, and Smol Machines implements it inside hardware-isolated microVMs with sub-second starts, sandboxing defaults, and a local-to-cloud workflow. If that matches your workload, start with the Smol Machines site and docs, then validate the full path with your real CUDA application.