Runloop vs exe.dev
Runloop is a sandbox platform for coding agents, with per-second billing, hostname egress rules and devboxes that live up to 48 hours unless an idle policy takes over. exe.dev rents long-lived Linux VMs from a flat monthly pool, in nine regions, with no time limit.
The companies and products
Runloop
Runloop is a US platform of sandboxes, called devboxes, for building, running and testing AI coding agents.
You get Linux devboxes built from Dockerfile-style blueprints, disk snapshots, HTTPS tunnels, agent gateways that hide API keys, and Axons, event streams that drive agents such as Claude Code. Each devbox runs in a micro-VM on Runloop's own bare-metal hypervisor, with a container layer inside. It is built for teams shipping coding agents, plus evaluation and RL fine-tuning work on the Enterprise plan.
- Compliance
- SOC 2 Type 2
- Architectures
- x86_64 and arm64 base blueprints
- Own-cloud deploy
- Deploy to VPC on AWS, GCP and Azure (Enterprise)
- Regions
- Not documented or selectable
- Company
- Runloop AI, Inc.
exe.dev
exe.dev, made by Bold Software, Inc., rents Linux VMs for developers and AI agents that you reach over SSH and HTTPS.
You reserve a pool of vCPUs and memory, and your VMs share it; you can also start standalone VMs billed by the second. Each VM is KVM-isolated (they use Cloud Hypervisor), boots from a container image in about two seconds, and has root, systemd and Docker. It comes with Claude Code, Codex, Pi and exe.dev's own Shelley agent, and is aimed at developers who want a persistent dev box or a sandbox for their agents.
- Company
- Bold Software, Inc.
- Preinstalled agents
- claude, codex, pi and the Shelley web agent
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “20th cheapest of 367” means 19 of the other 366 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | Runloop: price rank | exe.dev: price rank | bochka: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month | 20th cheapest of 367 $8,152 a month | 253rd of 367, with limits $3,701 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB Both fit always-on assistants; exe.dev keeps a VM running with no time limit from a flat pool, while Runloop needs an idle policy to run past 48 hours. | 32nd cheapest of 366 $10,194 a month | 15th cheapest of 366 $3,833 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month Runloop ranks higher for many short code runs; exe.dev's 50-VM cap and bochka's start-rate limits both hold back very large bursts. | 44th cheapest of 345 $525 a month | 235th of 345, with limits $175 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Runloop fits sandboxes for your users at 300 at once, exe.dev Personal does not, and bochka ranks first with token-protected URLs. | 26th cheapest of 361 $3,301 a month | 257th of 361, with limits $1,226 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month Runloop ranks near the top for RL rollouts because it starts large waves with no published start limits, while exe.dev Personal stops at 50 VMs at once. | 4th cheapest of 360 $15,922 a month | 265th of 360, with limits $5,250 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 81st cheapest of 364 $2,562 a month | 283rd of 364, with limits $1,050 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month bochka ranks highest for CI with the fastest measured CPUs; Runloop bills per second, and exe.dev pools bill whether jobs run or not. | 121st cheapest of 377 $1,554 a month | 288th of 377, with limits $504 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB exe.dev is built for 24/7 dev boxes with SSH and a disk that lasts, and Runloop ranks lower here because of its 48-hour default lifetime cap and no public IP. | 113th cheapest of 368 $410 a month | 25th cheapest of 368 $35 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Runloop ships a desktop and browser your agent can control, exe.dev VMs are headless, and bochka streams a 60fps desktop you can embed. | 39th cheapest of 369 $2,306 a month | 240th of 369, with limits $1,055 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 40th cheapest of 367 $1,153 a month | 130th of 367, with limits $530 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 68th cheapest of 346 $781 a month | 124th of 346, with limits $278 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 266th of 338, with limits $15,611 a month | 166th of 338, with limits $5,490 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month Runloop can lock each devbox to a hostname allowlist, which exe.dev does not offer, and bochka gives each sandbox a full kernel with /dev/kvm. | 10th cheapest of 360 $770 a month | 60th of 360, with limits $263 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month | Can't run it | Can't run it | Can't run it |
Key features
| Feature | Runloop | exe.dev | bochka |
|---|---|---|---|
| Isolation | Micro-VM on a custom bare-metal hypervisor, plus a container layer | KVM VM with Cloud Hypervisor on rented bare metal; you can't choose the kernel | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts |
| How you pay | Per second for allocated CPU and memory; disk billed for the devbox's whole life | Flat monthly pool shared by your VMs, plus standalone VMs billed per second | Per second while a sandbox runs; the plan fee is all usage credit |
| Docker inside | Yes, on the Docker-in-Docker blueprints, as root | Yes, with root and systemd | Yes, with root and systemd |
| Nested virtualization | Not documented | Enterprise only | Yes: /dev/kvm on every plan, Android emulator, VMs inside |
| Custom images | Blueprints built from Dockerfiles, plus disk snapshots | Boot from your own Docker image | Templates from named snapshots; no Dockerfile or OCI images |
| Saved state | Disk kept through suspend (Pro) or manual disk snapshots; shutdown ends the devbox | Disk persists and is streamed to a second server; no snapshots | Snapshot every minute and on stop, on every plan; resume with the same disk |
| How long it runs | Lifetime 1 hour by default, 48 hours max; an idle policy replaces that cap | No cap; VMs run until you stop them | No session cap on paid plans; you set the TTL |
| Idle sleep and wake | Suspend after idle, wake on an HTTP request to the tunnel (Pro) | No idle auto-stop; pools bill whether VMs are busy or idle | No idle timer; TTL based, and a stopped sandbox costs nothing |
| Desktop and computer use | Desktop and browser your agent can control | Headless; no desktop or screen streaming | 60fps desktop and Chromium, computer use, browser control, embeddable stream |
| Inbound traffic | One HTTPS tunnel per devbox; SSH through a TLS proxy; no public IP | HTTPS proxy for ports 3000 to 9999, one of them public; SSH; no public IP or raw TCP | HTTPS URLs for up to 50 ports, token-protected by default; raw TCP; dedicated IPv6 or IPv4 |
| Egress rules | Hostname allowlist with wildcards, or block all | No provider-side allowlist | Open internet, no provider-side allowlist |
| CPU architecture | x86_64 and arm64 | x86_64 only | x86_64 only |
| Where it runs | Regions not documented; Deploy to VPC on AWS, GCP or Azure (Enterprise) | 9 regions in the US, Europe, Asia and Australia | EU only (Germany, Finland, France), EU data residency |
| Agent API | Claude Code, Codex, OpenCode, Gemini CLI, DeepAgents, driven through Axons and Broker | Prompts to its own Shelley agent from the CLI or HTTPS API; other agents through shell commands | Seven agent harnesses with a prompt, steer and events API, TypeScript and Python SDKs |
Every feature Battleships tracks (43)
| Feature | Runloop | exe.dev | bochka |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Yes | Yes |
| Nested virtualization | No | Yes | Yes |
| Root | Yes | Yes | Yes |
| Systemd | Not published | Yes | Yes |
| Desktop GUI | Yes | No | Yes |
| Browser | Yes | Yes | Yes |
| Computer-use API | Yes | No | Yes |
| SSH | Yes | Yes | Yes |
| Persistent disk | Yes | Yes | Yes |
| Pause/resume | Yes | Yes | Yes |
| Automatic snapshots | Yes | No | Yes |
| Fork/clone | Yes | Yes | Yes |
| Live fork (no pause) | Yes | Yes | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | Yes | Yes | Yes |
| ≥24 h sessions | Yes | Yes | Yes |
| Idle auto-stop | Yes | No | No |
| Wake on request | Yes | No | No |
| HTTPS preview URLs | Yes | Yes | Yes |
| Raw TCP inbound | No | No | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | Yes | No | No |
| Secret proxy | Yes | Yes | Yes |
| Secret proxy for any API | Yes | Yes | Yes |
| Preinstalled agents | Yes | Yes | Yes |
| Agent harness API | Yes | Yes | Yes |
| MCP server | Yes | Yes | No |
| GPU | No | No | No |
| Windows | No | No | No |
| Arm64 | Yes | No | No |
| Self-hosting | No | No | No |
| BYOC | Yes | Yes | No |
| Open source | No | No | No |
| Scoped API keys | Yes | Yes | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | No | Yes | Yes |
| Audit logs | Not published | Not published | No |
| SSO | Yes | Yes | No |
| SOC 2 | Yes | No | No |
| HIPAA | Not published | No | No |
| EU data residency | No | Yes | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
On the hpc-sandbox-benchmarks Node.js build-and-test loop at 4 vCPU / 8 GB (run 36498291449, 2026-09-29), bochka scored 26.69 runs a second and Runloop 8.29, about 3.2 times apart. exe.dev is not in that run, so there is no comparable CPU measurement for it, and we don't guess how it would score.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. exe.dev has no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | Runloop, a month | exe.dev, a month | bochka, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $113 · Basic | $35 · Personal | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $429 · Basic | $35 · Personal | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $5,268 · Basic | $1,848 · Personal | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $1,441 · Basic | $530 · Personal · with limits | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $3,184 · Basic | $1,050 · Personal · with limits | $2,000 · $2000/mo · with limits |
Runloop: Runloop bills per second for the CPU and memory you allocate while a devbox starts, runs, suspends or resumes, and bills its disk for the devbox's whole life. Basic has no plan fee; Pro adds a monthly platform fee that is not usage credit and unlocks suspend and resume.
exe.dev: exe.dev Personal is a flat monthly pool your VMs share, from $15 for 2 vCPU to $155 for 16 vCPU, billed whether VMs are busy or idle. Standalone VMs bill per second at $0.105 per 2 vCPU / 4 GiB per hour, and Work is $75 a month for a whole team.
Limits in those scenarios
exe.dev Personal allows one pool of up to 16 vCPU and 50 VMs at once, so the 300-at-once and 1,000-at-once scenarios don't fit there. Runloop publishes no concurrency cap for paid plans and fits both, but its fixed lifetime tops out at 48 hours unless an idle policy replaces it, and it publishes no egress price. On bochka, start-rate limits mean even the $2,000 plan can't launch 1,000 short jobs at once without queueing.
| Scenario | Runloop | exe.dev | bochka |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
| None that bite | None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use Runloop when
you launch large waves of short agent jobs or RL rollouts, or need hostname allowlists, arm64, Dockerfile builds or devboxes inside your own cloud.
- Lock down the network. Useful if a sandbox must only reach a fixed list of hosts: Runloop allows listed hostnames, with wildcards, or blocks everything.
- Big waves of short jobs. For RL rollouts and large bursts, Runloop bills per second and publishes no start limits, so 1,000 jobs at once fits.
- Dockerfiles, arm64 and your own cloud. Runloop builds images from Dockerfiles, offers arm64, and on Enterprise runs devboxes inside your own AWS, GCP or Azure account.
Use exe.dev when
you want a few long-lived Linux VMs at a flat monthly price, close to users outside Europe, booted from your own Docker image.
- One flat price for a box that never stops. For a dev box running all month, an exe.dev 4 vCPU pool is a flat $35 with no lifetime cap to manage.
- Regions around the world. Useful if your users are far from Europe: exe.dev runs in nine regions across the US, Europe, Asia and Australia.
- Your own Docker image and team controls. exe.dev boots VMs from your own Docker image, and its Work plan covers a whole team with SSO and roles.
Where bochka fits
Runloop is a short-lived agent sandbox and exe.dev is a long-lived dev box; bochka gives you both in one machine. Each sandbox is a full VM that can run for days, saves its disk every minute, costs nothing while stopped, and ran about three times faster than Runloop on the same CPU benchmark.
VMs and emulators inside
Runloop documents Docker-in-Docker but not /dev/kvm, and exe.dev offers nested virtualization only on Enterprise at custom pricing. Without it an Android emulator or a VM does not start. bochka exposes /dev/kvm on every plan.
State saved every minute
On Runloop a devbox's disk is gone at shutdown unless you pay for the Pro plan to suspend it, or write snapshot code. exe.dev keeps the disk but has no snapshots, so one bad agent step can cost a session's work.
Hosting with raw TCP
Runloop reaches a devbox through one HTTPS tunnel and exe.dev through an HTTPS proxy with one public port; neither gives a public IP or raw TCP. bochka serves up to 50 HTTPS ports, raw TCP and a dedicated IPv6 or IPv4.
Use bochka when your agents do real work for hours: builds, tests, Docker, emulators, a desktop and a browser, on faster CPUs, with state saved every minute. Compare it one to one: bochka vs Runloop and bochka vs exe.dev.
Questions people ask
Is Runloop cheaper than exe.dev?
Usually not. One coding agent on workdays costs $113 a month on Runloop and $35 on exe.dev, and 50 agents in parallel cost $5,268 versus $1,848. Runloop fits 300 or 1,000 sandboxes at once, which exe.dev Personal's 50-VM cap does not. bochka costs $20 and $317 for the first two.
Which is faster, Runloop or exe.dev?
We can't say: exe.dev was not in the benchmark run. On the same Node.js build-and-test loop at 4 vCPU / 8 GB, Runloop scored 8.29 runs a second and bochka 26.69.
Can Runloop and exe.dev run Docker?
Yes. Runloop runs Docker as root on its Docker-in-Docker blueprints, and exe.dev VMs come with root, systemd and Docker. bochka runs Docker with root and systemd too, and adds /dev/kvm for VMs and emulators inside.
How long can a Runloop devbox or exe.dev VM run?
A Runloop devbox has a 1-hour lifetime by default and 48 hours at most, unless you set an idle policy, which has no documented cap. exe.dev VMs run until you stop them. bochka has no session cap on paid plans.
Do Runloop or exe.dev have a desktop for computer use?
Runloop ships a desktop and a browser your agent can control. exe.dev VMs are headless. bochka streams a 60fps desktop with Chromium that you can embed in your own app.
Where do Runloop and exe.dev run?
exe.dev has nine regions in the US, Europe, Asia and Australia. Runloop does not document its regions, but on Enterprise it deploys into your own AWS, GCP or Azure account. bochka runs only in the EU, with EU data residency.
Try bochka
Install the CLI, then run bochka new.
Product preview