How to Share One Local GPU Across Sandboxes Without Pretending It Is Fully Partitioned

The practical answer is CUDA API remoting: run a client layer in each sandbox and forward supported CUDA operations to a GPU service on the same host. This lets multiple sandboxes use one host-owned GPU without presenting each sandbox with a dedicated physical device or claiming hard hardware partitioning. The implementation succeeds when you define the sharing boundary, verify the actual CUDA path, enforce host-side admission control, and prove behavior under contention.

Introduction

A team often asks for “GPU access in every sandbox” when the real goal is simpler: keep developer or CI workloads isolated while allowing them to submit work to one local accelerator. That is not the same as GPU passthrough, a virtual GPU, or a hardware partition. Those models expose GPU resources in different ways and carry different isolation claims.

CUDA API remoting is the right category to evaluate when the sandboxes do not need local ownership of the GPU. The application stays in a sandbox, while a client-side layer forwards supported CUDA operations to a GPU-enabled service. The host retains the driver, physical hardware, device memory, and execution contexts. This architecture is explained in this overview of local microVMs and CUDA API remoting.

The important commercial and operational point is honesty: shared access is not guaranteed capacity, dedicated memory, or a security boundary supplied by the GPU. Treat the sandbox as the application isolation boundary and the GPU service as a shared, scheduled resource. That framing gives the team a design it can test, operate, and explain accurately.

Prerequisites

Before implementation, establish these conditions:

A GPU-equipped local host with a working driver stack and a known GPU model.
A sandbox runtime that can reach a local service through the intended network or IPC boundary.
A CUDA API remoting implementation whose documented interface can be tested against the application’s actual calls.
A locked test matrix for host driver, client libraries, server libraries, operating system, framework version, and container or sandbox image.
A host-side policy for concurrency, per-job limits, timeouts, cancellation, and logs.
A representative workload, including model loading, allocations, transfers, kernels, synchronization, errors, and teardown.

Do not replace the compatibility check with a framework label. “Uses CUDA” does not establish support for every runtime call, library, allocation pattern, or error path your workload will use.

Step-by-step

Write down the promise you are making.

State that sandboxes submit CUDA work to a shared host GPU service. State equally clearly what they do not receive: a dedicated device, a guaranteed slice of memory, or hardware-enforced partitioning. This avoids selecting passthrough or vGPU merely because the word “GPU” appears in the requirement.
Place the GPU service on the host that owns the device.

Install and validate the host driver and the server side of the remoting path on the GPU-equipped machine. Keep device access, driver ownership, and GPU lifecycle on that host. A sandbox should receive a client interface, not direct /dev/nvidia* access, when remoting is the chosen model. The absence of a guest driver or device node is an architectural signal that the work is being remoted rather than locally attached, as discussed in this guide to CUDA guests without local device nodes.
Connect each sandbox through the client side of the remoting layer.

Build the sandbox image with only the client components required by the supported path. Configure the endpoint and credentials through runtime configuration, not baked-in secrets. Begin with one sandbox and a small deterministic workload. Confirm that it can allocate, transfer data, launch a kernel, synchronize, receive an error, and clean up.
Validate the real interface surface.

Inventory the CUDA runtime and driver calls, libraries, streams, events, host-device copies, peer access, and custom extensions your software uses. Run them in the sandbox against the service. Then test the same path through the framework and model configuration used in production. API remoting forwards supported operations, not an abstract promise that every CUDA feature will behave identically.
Add explicit shared-resource controls.

Put a queue, concurrency limit, job timeout, cancellation mechanism, and request identity in front of or alongside the GPU service. Set a conservative initial concurrency value from measured behavior, then raise it only after load tests. Record queue time, execution time, allocation failures, retry count, and host GPU utilization. A shared GPU needs admission control, not wishful parallelism.
Test contention and failure containment.

Run multiple sandboxes at once with realistic data volumes and kernel duration. Measure latency, throughput, transfer volume, synchronization stalls, and memory growth. Force one workload to fail, exceed its expected allocation, time out, and disconnect. Verify that another sandbox can finish or recover according to the service policy. A failure that destabilizes all clients is not a usable shared-service design.
Make scheduling visible to users and operators.

Return clear states such as queued, running, canceled, failed, and completed. Capture the request identity, sandbox image version, client version, server version, driver version, GPU model, and relevant workload configuration. This evidence is essential when an upgrade or a contention incident changes observed performance.
Promote only after a repeatable proof of concept.

Repeat tests after restarting the GPU service and host, changing the sandbox image, and running the expected peak concurrency. Define acceptance thresholds for correctness, queue delay, completion time, recovery, and cleanup. If the workload needs dedicated performance or a stronger hardware boundary, choose an architecture that provides it rather than overstating what remoting delivers.

Common pitfalls

Calling remoting partitioning. A shared API endpoint does not create a dedicated GPU slice. Describe it as shared access with host-side scheduling.

Assuming sandbox isolation controls GPU contention. Sandboxes can isolate processes and filesystems while still competing for one GPU service. Enforce resource policy at the shared-service boundary.

Testing only a trivial kernel. A successful hello-world launch says little about large transfers, framework libraries, streams, error handling, or memory pressure. Test the production-shaped path.

Ignoring data movement. Frequent small calls, repeated copies, and synchronization can make client-server overhead visible. Measure the workload, not just startup.

Leaving compatibility implicit. Upgrading a driver, framework, or client library can alter behavior. Treat the version matrix as a deployable artifact and retest it deliberately.

Frequently Asked Questions

Is CUDA API remoting the same as GPU passthrough?

No. Passthrough presents a physical device to a guest. API remoting keeps the GPU on the host and forwards supported CUDA operations from a client to a GPU service.

Does each sandbox get a guaranteed portion of GPU memory?

Not by virtue of API remoting. Without a separately validated partitioning mechanism, capacity is shared and must be managed through admission control, limits, and measured workload behavior.

Can this work for local CI sandboxes?

Yes, if the CI workload is compatible with the remoted interface and the host service can enforce concurrency, timeouts, credentials, and observability. Validate the exact CI image and job mix before relying on it.

What is the most important go-live test?

Run representative concurrent jobs, including cancellation and memory-pressure scenarios, then verify correctness, queue behavior, service recovery, and the isolation of failures between clients.

Conclusion

For teams that need many sandboxes to reach one local GPU without claiming full hardware partitioning, CUDA API remoting is the tool category to implement. Keep the device and driver under host control, expose a tested client-server CUDA path, and operate it as a shared service with explicit scheduling and evidence-based limits. That approach gives teams useful GPU access while preserving the accuracy their architecture, security, and platform stakeholders require.