exe.dev vs Upstash Box
exe.dev gives each developer or agent a full KVM VM with root, systemd and Docker, from a reserved pool that bills whether it is busy or not. Upstash Box gives each agent a light Docker container in the US that pauses when idle, bills only the CPU it uses and wakes on the next request.
The companies and products
exe.dev
exe.dev, made by Bold Software, Inc., rents Linux VMs for developers and AI agents that you reach over SSH and HTTPS.
You reserve a pool of vCPUs and memory, and your VMs share it; you can also start standalone VMs billed by the second. Each VM is KVM-isolated (they use Cloud Hypervisor), boots from a container image in about two seconds, and has root, systemd and Docker. It comes with Claude Code, Codex, Pi and exe.dev's own Shelley agent, and is aimed at developers who want a persistent dev box or a sandbox for their agents.
- Company
- Bold Software, Inc.
- Preinstalled agents
- claude, codex, pi and the Shelley web agent
Upstash Box
Upstash Box is a cloud computer for AI agents from Upstash, the company known for serverless Redis, Vector and QStash.
Each box is a Docker container with its own filesystem, shell, git and an optional coding agent such as Claude Code, Codex or OpenCode, which you drive from TypeScript or Python SDKs. A box pauses when it sits idle, keeps its files, and you pay for the CPU it actually uses. Upstash built it to run its own Context7 service and opened it to everyone in March 2026.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “15th cheapest of 366” means 14 of the other 365 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | exe.dev: price rank | Upstash Box: price rank | bochka: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month A software factory needs Docker for databases and test services, which exe.dev VMs run and an Upstash box, itself a container, does not; bochka runs Docker and adds a desktop and snapshots every minute. | 253rd of 367, with limits $3,701 a month | 61st of 367, with limits $464 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB exe.dev fits always-on assistants fully with a VM that runs until you stop it, while Upstash keeps an agent on as a flat-price keep-alive container. | 15th cheapest of 366 $3,833 a month | 103rd of 366, with limits $475 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month Upstash suits short code runs: an idle box costs no CPU and a paused box wakes when work arrives, while exe.dev pools bill whether jobs run or not. | 235th of 345, with limits $175 a month | 10th cheapest of 345 $119 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month For sandboxes you hand to your users, Upstash allows 1,000 boxes at once and bochka fits too, while exe.dev's Personal plan stops at 50 VMs at once. | 257th of 361, with limits $1,226 a month | 2nd cheapest of 361 $222 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month | 265th of 360, with limits $5,250 a month | 352nd of 360, with limits $8,068 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 283rd of 364, with limits $1,050 a month | 52nd cheapest of 364 $1,011 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month exe.dev VMs run Docker for container builds but pools bill when idle; Upstash bills only active CPU but has no Docker inside; bochka has both Docker and per-second billing. | 288th of 377, with limits $504 a month | 120th cheapest of 377 $1,520 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB All three make good 24/7 dev boxes; exe.dev offers nine regions and a bigger pooled disk, Upstash a cheap flat keep-alive box, and bochka a full VM with a browser desktop. | 25th cheapest of 368 $35 a month | 10th cheapest of 368 $20 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Computer-use agents need a screen; neither exe.dev nor Upstash has a desktop, while bochka streams a 60fps Linux desktop. | 240th of 369, with limits $1,055 a month | 73rd of 369, with limits $76 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month Upstash ships a managed headless Chromium with AI page actions, on exe.dev you install a browser yourself, and bochka gives the agent a visible desktop browser. | 130th of 367, with limits $530 a month | 49th of 367, with limits $38 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 124th of 346, with limits $278 a month | 37th cheapest of 346 $400 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 166th of 338, with limits $5,490 a month | 200th of 338, with limits $8,000 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month | 60th of 360, with limits $263 a month | 225th of 360, with limits $259 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month | Can't run it | Can't run it | Can't run it |
Key features
| Feature | exe.dev | Upstash Box | bochka |
|---|---|---|---|
| Isolation | KVM VM with Cloud Hypervisor on rented servers; you can't choose the kernel | Docker container per box on AWS | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts |
| Docker inside | Yes, with root and systemd | Not documented; the box is itself a Docker container | Yes, Docker and systemd run as on any Linux server |
| Nested virtualization | Enterprise only | No | Yes, /dev/kvm on every plan |
| Custom images | Boot each VM from your own Docker image | Coming soon; reuse setups through filesystem snapshots | Templates from named snapshots; no Dockerfile or OCI images |
| Saved state | Disk persists and is streamed to a second server; no snapshots | Snapshot a running box on demand; paused boxes keep their files | Snapshots every minute while running and on stop |
| Fork | cp copies a VM's disk and config into a new VM | Start many new boxes from a snapshot of a running box | Fork a running sandbox; named snapshots as templates |
| Idle and wake | VMs run until you stop them; no wake on request | Pauses when idle; a request to its URL wakes it | Stopped sandboxes cost nothing and are started by API |
| Desktop and browser | Headless; install a browser yourself | Managed headless Chromium with AI page actions and CDP; no desktop | 60fps Linux desktop with Chromium, computer use and browser control |
| Coding agents | Claude Code, Codex, Pi and Shelley; only Shelley takes prompts through the API | Claude Code, Codex and OpenCode with run and stream APIs, webhooks and scheduled prompts | Seven agents preinstalled, with one API to prompt, steer and stream events |
| Inbound traffic | HTTPS proxy for ports 3000 to 9999, SSH; no public IP or raw TCP | HTTPS URL per port, public unless you add a token; SSH gateway; no raw TCP | HTTPS URLs for up to 50 ports, token-protected by default; raw TCP |
| Outbound rules | Open internet, no allowlist | Allow or deny by domain and CIDR, changeable on a running box | Open internet, no provider-side allowlist |
| Secrets | Integrations inject secrets at the network edge for any HTTP host | Host proxy injects headers into outbound HTTPS, set when the box is created | Credentials proxy for model providers and any HTTPS API; switch accounts when credits run out |
| Machine size and disk | Pools up to 16 vCPU on Personal; disk pooled, 200 GB at 4 vCPU | Up to 8 vCPU / 16 GB; disk 5, 10 or 20 GB by size | Up to 16 vCPU / 32 GB; 50 GB disk at 4 vCPU / 8 GB |
| Regions | 9 regions in the US, Europe, Asia and Australia | US only (AWS us-east-1) | EU only: Germany, Finland, France |
Every feature Battleships tracks (43)
| Feature | exe.dev | Upstash Box | bochka |
|---|---|---|---|
| Full VM (own kernel) | Yes | No | Yes |
| Docker inside | Yes | No | Yes |
| Nested virtualization | Yes | No | Yes |
| Root | Yes | Not published | Yes |
| Systemd | Yes | Not published | Yes |
| Desktop GUI | No | No | Yes |
| Browser | Yes | Yes | Yes |
| Computer-use API | No | No | Yes |
| SSH | Yes | Yes | Yes |
| Persistent disk | Yes | Yes | Yes |
| Pause/resume | Yes | Yes | Yes |
| Automatic snapshots | No | No | Yes |
| Fork/clone | Yes | Yes | Yes |
| Live fork (no pause) | Yes | Yes | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | Yes | Yes | Yes |
| ≥24 h sessions | Yes | Yes | Yes |
| Idle auto-stop | No | Yes | No |
| Wake on request | No | Yes | No |
| HTTPS preview URLs | Yes | Yes | Yes |
| Raw TCP inbound | No | No | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | No | Yes | No |
| Secret proxy | Yes | Yes | Yes |
| Secret proxy for any API | Yes | Yes | Yes |
| Preinstalled agents | Yes | Yes | Yes |
| Agent harness API | Yes | Yes | Yes |
| MCP server | Yes | Yes | No |
| GPU | No | No | No |
| Windows | No | No | No |
| Arm64 | No | Yes | No |
| Self-hosting | No | No | No |
| BYOC | Yes | No | No |
| Open source | No | No | No |
| Scoped API keys | Yes | Not published | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | Yes | Yes | Yes |
| Audit logs | Not published | Not published | No |
| SSO | Yes | No | No |
| SOC 2 | No | Yes | No |
| HIPAA | No | No | No |
| EU data residency | Yes | No | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
Neither exe.dev nor Upstash Box is in the hpc-sandbox-benchmarks run, so there is no comparable CPU measurement for them. bochka did 26.69 runs per second on that Node.js build-and-test loop at 4 vCPU / 8 GB, on dedicated AMD Ryzen 9 9950X hosts. We don't guess how the other two would score.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. exe.dev and Upstash Box have no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | exe.dev, a month | Upstash Box, a month | bochka, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $35 · Personal | $21 · Pay as You Go | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $35 · Personal | $21 · Pay as You Go | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $1,848 · Personal | $900 · Pay as You Go · with limits | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $530 · Personal · with limits | $306 · Pay as You Go | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $1,050 · Personal · with limits | $1,822 · Pay as You Go | $2,000 · $2000/mo · with limits |
exe.dev: Personal pools are a flat monthly fee for reserved vCPU and memory that your VMs share, billed whether they are busy or idle. Standalone VMs bill per second at $0.105 per 2 vCPU / 4 GiB per hour.
Upstash Box: A standard box bills only the CPU it actually uses, at $0.10, $0.20 or $0.40 per active CPU hour by size, plus $0.10 per GB-month of storage, and RAM is free. A keep-alive box costs a flat $8, $16 or $32 a month.
Upstash Box: Plans start at $20 a month, and the whole fee is usage credit. Sandboxes bill per second while they run, and a stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
exe.dev's Personal plan stops at 50 VMs at once, so the 300-at-once and 1,000-at-once scenarios don't fit there. Upstash has no Docker inside a box, so the 50-agent scenario does not fit, and its egress price is not published. bochka's limit is start rate: even the top plan starts at most 90 sandboxes a minute, so 1,000 short jobs at once are queued, while Upstash starts them.
| Scenario | exe.dev | Upstash Box | bochka |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel | None that bite |
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use exe.dev when
you want full VMs with Docker near users outside Europe, booted from your own Docker image, with SSO and roles for a team.
- A real VM with Docker. Agents that run Docker, systemd services or container builds get a full KVM VM with root on exe.dev.
- Regions around the world. Useful if your users are spread out: exe.dev runs in nine regions across the US, Europe, Asia and Australia.
- Your own Docker image and team controls. exe.dev boots VMs from your own Docker image, and its Work plan gives a team SSO and roles.
Use Upstash Box when
your agents mostly wait, should wake on an incoming request, need a strict outbound allowlist, or must burst to 1,000 at once in the US.
- Idle costs no CPU. Good for agents that wait a lot: Upstash bills only the CPU a box uses, and RAM is free.
- Wakes on the next request. A paused box wakes when a request reaches its URL, which suits user-facing agents that are quiet most of the day.
- Big bursts and outbound rules. It runs 1,000 boxes at once by default and can limit outbound traffic to listed domains and IP ranges.
Where bochka fits
If your agents do long, real work like builds, tests, Docker and checking UIs, bochka gives each one a full VM like exe.dev, but with a desktop, snapshots every minute and per-second billing that stops when the sandbox stops. It also keeps Docker inside, which Upstash Box lacks.
A screen agents can use
Neither exe.dev nor Upstash has a desktop: exe.dev VMs are headless, and Upstash offers only a headless Chromium over CDP. bochka streams a 60fps Linux desktop with computer-use and browser-control APIs.
Docker and /dev/kvm in every sandbox
Upstash boxes are containers with no Docker inside, so the 50-agent scenario does not fit there. exe.dev runs Docker but keeps nested VMs for Enterprise. bochka has both on every plan, so emulators and nested VMs work.
State saved every minute
exe.dev has no snapshots to roll back to, and Upstash snapshots only when you ask. bochka snapshots a running sandbox every minute and on stop, so a bad agent step loses at most a minute, and you can fork it while it runs.
Use bochka when your agents need a real machine for hours: Docker, /dev/kvm, a desktop and a browser, with state saved every minute and one API to drive agents inside from outside. Compare it one to one: bochka vs exe.dev and bochka vs Upstash Box.
Questions people ask
Is exe.dev cheaper than Upstash Box?
For one agent, no: in our scenarios exe.dev's Personal pool is $35 a month against $21 on Upstash, both for workdays and for running all month. For 50 agents in parallel exe.dev is $1,848 a month, and Upstash does not fit because it has no Docker inside. bochka is $317 a month for that fleet.
Can Upstash Box run Docker?
Its docs do not offer Docker inside a box: each box is itself a Docker container, and custom images are listed as coming soon. exe.dev VMs and bochka sandboxes are full VMs where Docker and systemd run as on any Linux server.
Which is faster, exe.dev or Upstash Box?
We can't say: neither is in the hpc-sandbox-benchmarks CPU test. bochka was measured at 26.69 runs per second on that test at 4 vCPU / 8 GB.
Do exe.dev or Upstash Box have a desktop for computer use?
No. exe.dev VMs are headless, and Upstash gives a headless Chromium with AI page actions and CDP but no desktop. bochka streams a full Linux desktop at 60fps with computer-use tools.
Where do exe.dev and Upstash Box run?
exe.dev has nine regions in the US, Europe, Asia and Australia. Upstash Box runs only in AWS us-east-1, with more regions planned. bochka runs in the EU, in Germany, Finland and France.
Can I run many sandboxes at once?
Upstash Pay as You Go allows 1,000 boxes at once by default and raises it on request. exe.dev Personal allows 50 VMs in a pool plus 50 standalone VMs. bochka goes from 100 to 2,000 at once by plan, with a limit on how fast new ones start.
Try bochka
Install the CLI, then run bochka new.
Product preview