Why we open source our moat

smolvm, the runtime under our paid cloud, and smol, the tool we use to manage machines, are fully open source. Anyone can run them, deploy them, or build on them.

For most infrastructure companies, this code is the moat. We gave ours away on purpose, because we believe compute is about to change a lot, and we want to build that change in the open with the developers who will use it.

What customers pay for

We discussed this on a panel with Reflex and General Translation about building businesses around open source.

If developers can take the code and run it themselves, why would they pay us?

Our answer is that we sell “responsibility or blame.” Customers pay us to run their machines, keep the service up, and own its performance and reliability. Naturally, the people who build the open sourced runtime are the best people to run the distributed system around it.

Where smolvm came from

smolvm grew out of four years of building container infrastructure at AWS on ECS, Fargate, and Fargate on Firecracker. That work gave us a close look at the assumptions built into cloud compute and the necessary expertise to bring innovation to this field.

The first assumption we questioned was boot time. Hardware has gotten more than a thousand times faster since the 90s, but a Linux VM still boots about as slowly as it did then. This is because the kernel and the boot process kept collecting unnecessary parts over the years, so we removed parts until the machine only holds the bare minimum code to run, successfully bringing the boot time to under one second.

We build the rest of smolvm with the same mindset, keeping the essentials and being selective with additions.

Less time spent on resource management. smolvm’s elastic memory lets the host commit only what the guest actually uses. A running machine can grow its CPU, memory, or disk without restarting its workload. Developers make fewer sizing decisions, and the same hardware does more work.

Running machines can be saved and branched. Branching and checkpoints let you save a running machine, come back to it, and explore several directions from the same point. An agent that makes a mistake can return to a checkpoint and continue from there. We wrote more about this in Giving AI an undo button.

The capabilities of VMs are further explored. A VM that runs CUDA usually needs a GPU of its own and an NVIDIA driver installed inside it. With GPU remoting, a smolvm VM forwards its CUDA calls to the host, so several VMs can run CUDA work on one GPU without a driver of their own. We used this for LoRA fine-tuning. We started one GPU machine with the base model already loaded, then branched it into several experiments. Every branch reads the same copy of the base model and trains its own small adapter on top of it, so each experiment stays isolated and none of them has to load the base model again.

Why we chose to open source

The code that branches machines and the tools that manage them are public. You can read them, adapt them to your infrastructure, or build a cloud of your own on top of them. We want companies to run smolvm themselves, and we help them do it.

Publishing the code also tests our ideas. People run workloads we never planned for, read the implementation, and push back on the design, and this shows us which parts hold up and which need more work.

Where can smolvm run

Some companies already have an infrastructure team, their own hardware, and their own rules for deploying software. We help them run smolvm on premises, connect it to the systems they already use, and set it up for unusual workloads. They can run it entirely themselves or bring us in for the parts they want help with.

Other teams would rather we run the machines so they can spend their time on their product, and that’s what smol cloud is for. We handle capacity planning, upgrades, monitoring, and failures, and our job is to make this worth paying for.

Companies running smolvm on their own hardware are part of the same goal, because they bring this way of running compute to places our cloud may never reach.

You can start with smolvm today, talk to us about an on-premises or custom deployment, or use smol cloud if you'd like us to run it for you.