GitHub Actions vs Blacksmith

Blacksmith does not replace GitHub Actions: it runs your GitHub Actions jobs on its own runners once you change the runner label. So the real choice is between GitHub's hosted runners and Blacksmith's Firecracker microVMs with sticky disks and no published concurrency limit.

The companies and products

GitHub Actions

GitHub Actions is GitHub's CI/CD service: workflows in your repository run as jobs on runners that GitHub hosts or that you host yourself.

GitHub Actions is a CI service, not a sandbox API. Each job on a GitHub-hosted runner gets a new VM, Linux and Windows on Azure and macOS on GitHub's own Mac fleet, with passwordless sudo, and the VM is gone when the job ends. It is built for building, testing and deploying the code in a repository, and GitHub's own Copilot cloud agent and the Claude and Codex agents on GitHub also run on Actions runners.

Kind of product: CI/CD service; jobs are defined in workflow files in a GitHub repository
Isolation: A new VM for each job; Linux and Windows on Azure, macOS on GitHub's own Mac cloud
Job length: Each job can run for up to 6 hours
Concurrency: 20 jobs on Free, 40 on Pro, 60 on Team, 500 on Enterprise; macOS 5, or 50 on Enterprise
Allowed use: GitHub-hosted runners are for work on the repository's software project; reselling Actions as a service is not allowed
Self-hosted runners: Open-source runner under MIT; usage on self-hosted runners is free
Compliance: SOC 1 and SOC 2 Type 2 reports, ISO/IEC 27001:2022

Blacksmith

Blacksmith is a CI service: it runs your GitHub Actions jobs on its own faster, cheaper runners instead of GitHub's.

You change the runner label in your workflow file, and each job runs in a fresh Firecracker microVM that boots in under 3 seconds and is destroyed when the job ends. It adds Docker layer caching and sticky disks that carry chosen folders from one run to the next, plus a Testbox CLI in early beta that lets a local coding agent run commands inside a CI job. It is built for teams that want quicker, cheaper GitHub Actions, not as a general sandbox you drive by API; a separate Sandboxes product is marked coming soon.

Kind of product: Drop-in GitHub Actions runners; jobs start from workflows
Isolation: Firecracker microVM per job on Blacksmith's own bare-metal hardware, destroyed with its state after the job
Operating systems: Ubuntu on x64 and ARM, Windows Server 2025 (public beta), macOS on Apple Silicon
Regions: us-east (Ashburn), us-west (Phoenix), eu-west (Amsterdam), eu-central (Frankfurt)
Concurrency: Docs say Blacksmith does not impose any concurrency limits
Agent products: Testbox CLI in early beta; Sandboxes for coding agents marked coming soon, with no published price
Compliance: SOC 2 Type 2, GDPR compliant

Which use cases each one fits

For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “100th cheapest of 377” means 99 of the other 376 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.

Use case and its workload	GitHub Actions: price rank	Blacksmith: price rank	bochka: price rank
Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month A software factory needs agents that keep their checkout and services and can show what they built; both CI products only cover the build and test step.	298th of 367, with limits $5,853 a month	248th of 367, with limits $3,607 a month	3rd cheapest of 367 $634 a month
Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB An assistant that lives on one machine all month does not fit either, because both start a fresh machine per job, while bochka keeps one VM running as long as you want.	333rd of 366, with limits $10,692 a month	78th of 366, with limits $5,463 a month	5th cheapest of 366 $657 a month
Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month Each code run would be a whole workflow job on either service, which suits builds better than many tiny runs.	294th of 345, with limits $721 a month	281st of 345, with limits $488 a month	186th of 345, with limits $2,000 a month
Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Neither is built to give your users a sandbox: machines start only from GitHub workflows, and GitHub's terms rule out offering Actions as a service; bochka starts them by API.	308th of 361, with limits $2,892 a month	268th of 361, with limits $1,563 a month	Cheapest of 361 $209 a month
RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month	304th of 360, with limits $14,721 a month	276th of 360, with limits $7,488 a month	34th of 360, with limits $2,000 a month
Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month	303rd of 364, with limits $1,962 a month	354th of 364, with limits $1,188 a month	10th cheapest of 364 $200 a month
CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month Both fit, and Blacksmith runs the same workflows for less in this table ($708 against $1,012 a month); bochka also fits, as a self-hosted runner you keep warm or CI you drive by API.	100th cheapest of 377 $1,012 a month	78th cheapest of 377 $708 a month	4th cheapest of 377 $86 a month
24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB A dev box needs a disk that stays and a way in; GitHub runners keep nothing, Blacksmith keeps only sticky disk folders, and bochka keeps the whole disk with SSH and open ports.	279th of 368, with limits $159 a month	180th of 368, with limits $207 a month	18th cheapest of 368 $26 a month
Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Both are headless runners, while bochka streams each agent a 60fps Linux desktop with computer use built in.	292nd of 369, with limits $1,768 a month	243rd of 369, with limits $1,088 a month	6th cheapest of 369 $200 a month
Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month	135th of 367, with limits $942 a month	131st of 367, with limits $588 a month	7th cheapest of 367 $200 a month
Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month	134th of 346, with limits $1,184 a month	130th of 346, with limits $719 a month	6th cheapest of 346 $90 a month
Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month	301st of 338, with limits $28,971 a month	262nd of 338, with limits $14,613 a month	71st of 338, with limits $1,800 a month
Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month	70th of 360, with limits $467 a month	64th of 360, with limits $350 a month	2nd cheapest of 360 $200 a month
GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month	Can't run it	Can't run it	Can't run it
macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month Both have hosted macOS runners on Apple silicon, which bochka does not offer, as it runs Linux only.	25th of 36, with limits $1,178 a month	28th of 36, with limits $1,508 a month	Can't run it

Key features

Feature	GitHub Actions	Blacksmith	bochka
What it is	GitHub's CI/CD service: workflows in your repository run as jobs on runners GitHub hosts	Drop-in runners for GitHub Actions: you change the runner label and jobs run on Blacksmith's machines	Sandbox API: create, stop, fork and reach full VMs from your code
Isolation	New VM per job on Azure (Linux, Windows) or GitHub's Mac cloud, with passwordless sudo	Firecracker microVM per job on its own hardware with gaming CPUs, destroyed after the job	Full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts, with root, Docker and systemd
How a machine starts	A workflow job in a GitHub repository	A GitHub Actions workflow job; Testbox warmup also dispatches a workflow	One API call, SDK or CLI, from anywhere
How long it lives	Up to 6 hours per job	Until the job ends, then the microVM is destroyed; Testbox stops after 30 idle minutes by default	Until you stop it; no session cap on paid plans
Files between runs	Fresh VM every job; files carry over only through cache, artifacts or custom images	Sticky disks: up to 5 folders per job, evicted after 7 days unused, plus a Docker layer cache	The whole disk, saved every minute and on stop; a stopped sandbox costs nothing
Fork	No; custom images are a template for new runners, not a copy of a live one	No; every machine starts fresh from a job	Fork a running sandbox with its disk
Jobs at once	20 on Free, 40 on Pro, 60 on Team, 500 on Enterprise; macOS 5, or 50 on Enterprise	Docs say Blacksmith does not impose any concurrency limits	Many sandboxes at once; the $2,000 plan queues starts above 90 a minute
Operating systems	Linux x64 and arm64, Windows x64 and arm64, macOS	Ubuntu x64 and ARM, Windows Server 2025 (public beta), macOS on Apple Silicon	Linux x64
GPUs	Linux and Windows larger runners with an NVIDIA T4	None	None
Nested virtualization	Yes on Linux runners, after adding the runner user to the KVM group	Yes on x64 Linux runners, not on ARM	Yes, /dev/kvm for Android emulators and VMs
Inbound access	None; GitHub says hosted runners never need inbound connections	Opt-in debug SSH, only for the user who triggered the job; no ports or URLs	SSH, HTTPS URLs for up to 50 ports, raw TCP, dedicated IP
Desktop and computer use	Headless runners; browsers are preinstalled for tests	Headless runners	Linux desktop streamed at 60fps, Chromium, computer use API
Coding agents	Copilot cloud agent, plus Claude and Codex in public preview, on issues and pull requests	Its own [code]smith agent from Slack, GitHub or a web UI; Testbox CLI (early beta) for local agents	Seven agents preinstalled, driven from outside with a prompt, steer and events API; credentials proxy keeps real keys out
Regions and self-hosting	Azure, with larger runners able to join your Azure network in the US, EU and APAC; free open-source self-hosted runner	US east, US west, Amsterdam and Frankfurt; managed only	EU only (Germany, Finland, France), managed only

Every feature Battleships tracks (43)

Feature	GitHub Actions	Blacksmith	bochka
Full VM (own kernel)	Yes	Yes	Yes
Docker inside	Yes	Yes	Yes
Nested virtualization	Yes	Yes	Yes
Root	Yes	Yes	Yes
Systemd	No	No	Yes
Desktop GUI	No	No	Yes
Browser	Yes	Yes	Yes
Computer-use API	No	No	Yes
SSH	No	Yes	Yes
Persistent disk	No	Yes	Yes
Pause/resume	No	No	Yes
Automatic snapshots	No	No	Yes
Fork/clone	No	No	Yes
Live fork (no pause)	No	No	Yes
Memory fork	No	No	No
Custom image (Docker or snapshot)	Yes	No	Yes
≥24 h sessions	No	Yes	Yes
Idle auto-stop	Yes	Yes	No
Wake on request	No	No	No
HTTPS preview URLs	No	No	Yes
Raw TCP inbound	No	No	Yes
Public IPv4	No	No	No
Egress allowlist	Yes	No	No
Secret proxy	No	No	Yes
Secret proxy for any API	No	No	Yes
Preinstalled agents	Yes	No	Yes
Agent harness API	Yes	No	Yes
MCP server	Yes	No	No
GPU	Yes	No	No
Windows	Yes	Yes	No
Arm64	Yes	Yes	No
Self-hosting	Yes	No	No
BYOC	No	No	No
Open source	Yes	No	No
Scoped API keys	Not published	Not published	Yes
Spend limits	Yes	Not published	Yes
Usage per sandbox (API)	Yes	Yes	Yes
Audit logs	Yes	Not published	No
SSO	Yes	No	No
SOC 2	Yes	Yes	No
HIPAA	No	No	No
EU data residency	Yes	Yes	Yes
Zero data retention	Not published	Not published	Yes

Performance

GitHub ActionsNot measured

BlacksmithNot measured

bochka26.69 runs/s

Neither GitHub Actions nor Blacksmith has a comparable measurement in the hpc-sandbox-benchmarks CPU test, so we cannot say which is faster. Blacksmith says its runners use fast gaming CPUs. bochka did 26.69 runs per second on that test at 4 vCPU / 8 GB, on dedicated AMD Ryzen 9 9950X hosts.

Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. GitHub Actions and Blacksmith have no published run in this suite yet, so there is no like-for-like number to compare.

Pricing in five scenarios

Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.

Scenario	GitHub Actions, a month	Blacksmith, a month	bochka, a month
One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days	$80 · Team · with limits	$36 · Pay-as-you-go · with limits	$20 · $20/mo
One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month	$354 · Team · with limits	$207 · Pay-as-you-go · with limits	$26 · $20/mo
50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside	$3,814 · Team · with limits	$2,364 · Pay-as-you-go · with limits	$317 · $20/mo × 3 seats
Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once	$1,092 · Enterprise Cloud · with limits	$663 · Pay-as-you-go · with limits	$200 · $100/mo × 2 seats
1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month	$2,721 · Enterprise Cloud · with limits	$1,488 · Pay-as-you-go · with limits	$2,000 · $2000/mo · with limits

GitHub Actions: GitHub bills per minute per job, rounding each job up to the next minute, from $0.006 a minute for 2-core Linux. Plans include monthly minutes for standard runners only; larger and GPU runners are always billed.

Blacksmith: Blacksmith bills per minute of job time by vCPU and OS: $0.004 a minute for 2 vCPU Ubuntu x64, $0.0025 on ARM, with 3,000 free minutes a month. Sticky disks cost $0.50 per GB a month and a static IP $100 a month.

Blacksmith: Plans start at $20 a month, and the whole fee is usage credit. You pay per second while a sandbox runs, and a stopped sandbox costs nothing and keeps its disk.

Limits in those scenarios

Neither CI service fits the five agent scenarios cleanly, because a machine exists only while a workflow job runs. GitHub jobs end after 6 hours and keep nothing on disk, and Enterprise Cloud stops at 500 concurrent jobs, short of the 1,000 short jobs at once. Blacksmith keeps only sticky disk folders, so packages installed elsewhere reset, and its egress price is not published. bochka fits four of the five; for 1,000 short jobs at once its $2,000 plan queues starts above 90 a minute.

Scenario	GitHub Actions	Blacksmith	bochka
One coding agent, workdays	Team: sessions capped at 6 h missing persistent disk different kind of product: CI runner	different kind of product: CI runner files survive a stop only in sticky disk paths: packages installed elsewhere reset egress price not published	None that bite
One agent that never stops	Team: sessions capped at 6 h missing persistent disk different kind of product: CI runner	different kind of product: CI runner files survive a stop only in sticky disk paths: packages installed elsewhere reset egress price not published	None that bite
50 agents in parallel	Team: sessions capped at 6 h missing persistent disk different kind of product: CI runner	different kind of product: CI runner files survive a stop only in sticky disk paths: packages installed elsewhere reset egress price not published how many machines can run at once: limit not published, probably fine at 50 (the typical published limit is 50)	None that bite
Sandboxes for your users	different kind of product: CI runner	different kind of product: CI runner not confirmed: how many machines can run at once (you need 300; the typical published limit is 50) egress price not published	None that bite
1,000 short jobs at once	Enterprise Cloud: max 500 concurrent different kind of product: CI runner	different kind of product: CI runner not confirmed: how many machines can run at once (you need 1,000; the typical published limit is 50) egress price not published	max 2,400 starts/day (needs ~6,667) max 600 starts/hour (needs ~1,000) bursts above 90 starts/min are queued (a burst of 333 takes ~4 min to start)

Which one should you use?

Use GitHub Actions when

you want CI with nothing extra to set up, need GPU runners, Windows arm64 or your own Azure network, or want to self-host the runner for free.

Nothing extra to set up. For a GitHub repository, hosted runners are already there, with included minutes on paid plans and free standard runners for public repos.
GPUs and your own network. Teams that need a T4 GPU in CI, or runners inside their Azure network with static IPs, get both from GitHub.
Run it on your own machines. The runner is open source, and jobs on self-hosted runners cost nothing in GitHub usage.

Use Blacksmith when

your CI already lives in GitHub Actions and you want cheaper minutes, warm caches through sticky disks and no concurrency cap by changing one runner label.

Same workflows, cheaper minutes. Teams already on GitHub Actions switch by changing one runner label, and per-minute rates start below GitHub's.
Caches that stay warm. Sticky disks and a managed Docker layer cache carry chosen folders between runs, so builds skip work a fresh VM repeats.
No concurrency cap. Its docs say it imposes no concurrency limits, so a big test matrix can start all at once instead of waiting in line.

Where bochka fits

GitHub Actions and Blacksmith both run CI jobs, and Blacksmith runs GitHub's own workflows, so for CI the choice is between them. If what you are really running is agents, both give you a machine that exists only while a workflow job runs. bochka gives each agent a full VM you start by API, keep as long as you want and reach from outside.

A machine that stays

On GitHub runners every job starts clean and ends within 6 hours; on Blacksmith the microVM is destroyed and only sticky disk folders survive. bochka saves the whole disk every minute, stops for free and can fork a running sandbox.

Reach it while it runs

GitHub runners take no inbound traffic and Blacksmith offers only debug SSH for the person who started the job, so previews need a tunnel. bochka gives HTTPS URLs on 50 ports, raw TCP, SSH and a 60fps desktop.

Agents driven from outside

On both, agents start from workflows, issues, Slack or a web UI, with real secrets inside the job. bochka has seven agents preinstalled, driven by one API from your backend, with keys kept in a credentials proxy outside the sandbox.

Use bochka when what you are running is agents: each needs its own full VM that you start by API, that keeps its disk, hosts what it builds, shows a desktop and keeps real API keys out of reach. Compare it one to one: bochka vs GitHub Actions and bochka vs Blacksmith.

Questions people ask

Is Blacksmith cheaper than GitHub Actions?

At list rates, yes: 2 vCPU Ubuntu x64 is $0.004 a minute on Blacksmith against $0.006 for 2-core Linux on GitHub. In this page's CI runners use case it comes to $708 against $1,012 a month. GitHub's included minutes and free public-repo runners can still make GitHub cheaper for small or open-source projects.

Do I have to leave GitHub Actions to use Blacksmith?

No. Blacksmith runs your existing GitHub Actions workflows: you change the runner label in the workflow file and the job runs on a Blacksmith microVM instead of a GitHub-hosted runner.

Which is faster?

We have no CPU measurement for either one made the same way, so we cannot rank them. Blacksmith says its runners use fast gaming CPUs, and its sticky disks and Docker layer cache let builds skip repeated work.

Can GitHub Actions and Blacksmith run Docker?

Both run Docker on Linux, and both expose KVM for Android emulators: GitHub on Linux runners once you add the runner user to the KVM group, Blacksmith on x64 Linux but not on ARM. bochka sandboxes are full VMs with Docker, systemd and /dev/kvm.

Can I run AI agents on GitHub Actions or Blacksmith?

For work on a repository, yes: GitHub runs the Copilot, Claude and Codex agents on Actions runners, and Blacksmith has its [code]smith agent and the Testbox CLI in early beta. Both still start a fresh machine per job with no inbound traffic, and Blacksmith's Sandboxes product is marked coming soon.

Do files persist between runs?

Not on GitHub runners, where files carry over only through cache, artifacts or custom images. Blacksmith keeps up to 5 sticky disk folders per job, evicted after 7 days unused. A bochka sandbox keeps its whole disk and saves it every minute.

Try bochka

Install the CLI, then run bochka new.

Product preview