Benchmarks
249× faster than LocalStack · 143× faster than Moto
Verified benchmarks using hey. Rust-accelerated DynamoDB. Endpoints fingerprinted via response headers (x-localstack: true, Server: Werkzeug).
Two Modes
Section titled “Two Modes”CloudMock ships in two deployment modes. Pick the one that fits your language and architecture.
In-Process Mode
The mock runs inside your Go test binary. Zero network hops, zero Docker, zero startup cost. AWS SDK calls are intercepted before they hit the wire.
- DynamoDB GetItem: 0.020ms
- S3 PutObject: 0.030ms
- SQS SendMessage: 0.015ms
- Startup: ~1ms
- Memory: 0 extra (in-process)
- Annual cost (1K builds/day): $4.32
HTTP Mode
A native Go binary exposes a real HTTP endpoint. Drop it in as a Docker sidecar or run the binary directly. Any AWS SDK in any language works against it.
- DynamoDB GetItem: 0.44ms
- S3 PutObject: 1.01ms
- SQS SendMessage: 0.73ms
- Startup: 65ms
- Memory: 67MB
- Annual cost (1K builds/day): $191
Latency by Service
Section titled “Latency by Service”P50 latency per key operation. Each service group shows all four providers. Lower is better.
Per-Service P50 Latency (lower is better)
In-Process mode is Go-only. SNS, IAM, and STS figures for In-Process are not applicable because those services are typically called through the HTTP endpoint even in Go tests. SQS LocalStack latency reflects its synchronous message routing overhead — ReceiveMessage with long-polling is excluded from all averages.
Throughput by Service
Section titled “Throughput by Service”Requests per second across 8 core AWS operations. Higher is better. Measured with hey at 100 concurrent connections (CloudMock) and 20 concurrent (LocalStack/Moto).
| Operation | CloudMock | LocalStack | Moto | CM / LS | CM / Moto |
|---|---|---|---|---|---|
| DynamoDB ListTables | 98,150 best | 662 | 1,526 | 148× | 64× |
| DynamoDB PutItem | 128,164 best | 449 | 1,492 | 285× | 85× |
| DynamoDB GetItem | 134,714 best | 723 | 1,569 | 186× | 85× |
| S3 ListBuckets | 102,010 best | 506 | 1,129 | 201× | 90× |
| SQS ListQueues | 102,880 best | 554 | 387 | 185× | 265× |
| IAM ListUsers | 115,341 best | 294 | 355 | 392× | 325× |
| Lambda ListFunctions | 117,876 best | 601 | 1,330 | 196× | 88× |
| EC2 DescribeInstances | 117,161 best | 179 | 217 | 653× | 539× |
Geometric mean: CloudMock 113,901 req/s — 249× faster than LocalStack (457 req/s), 143× faster than Moto (795 req/s).
Startup Time
Section titled “Startup Time”How long each provider takes to be ready for the first request.
Startup Time — lower is better
CI Cost at Scale
Section titled “CI Cost at Scale”Annual cost for a team running 1,000 CI builds per day, with 5,000 AWS SDK calls per build. Includes GitHub Actions compute ($0.008/min) and developer wait time ($75/hr loaded).
Time per CI build (5,000 SDK calls) — lower is better
Annual Savings by Switching to CloudMock
Section titled “Annual Savings by Switching to CloudMock”| SDK calls / build | vs LocalStack | vs Moto |
|---|---|---|
| 500 | $99,406 / yr | $26,971 / yr |
| 2,000 | $124,420 / yr | $41,306 / yr |
| 5,000 | $174,446 / yr | $69,975 / yr |
| 10,000 | $257,824 / yr | $117,757 / yr |
Developer Time Saved
Section titled “Developer Time Saved”At 5,000 SDK calls per build, 1,000 builds/day:
| Metric | vs LocalStack | vs Moto |
|---|---|---|
| Hours saved / year | 2,311 | 927 |
| Engineer-years saved | 1.1 | 0.4 |
| Working days saved | 289 | 116 |
| Cost saved ($75/hr loaded) | $173,325 | $69,525 |
Savings include GitHub Actions compute and developer wait time. CloudMock finishes 5,000 SDK calls in 0.1 seconds where LocalStack takes 22.9 seconds (12s Docker startup + 10.9s request processing) and Moto takes 9.3 seconds (3s Python startup + 6.3s request processing). That 23-second gap, repeated 1,000 times per day, adds up to 6.4 hours of cumulative wait time every day — 1.1 engineer-years annually.
Methodology
Section titled “Methodology”Harness: A Go binary using AWS SDK v2 exercises all providers with identical code paths. No provider-specific workarounds or tuning.
Iterations: 50 warm iterations per operation after a cold-start measurement. Concurrency tested at 5 goroutines.
In-Process measurement: AWS SDK calls are intercepted by CloudMock’s transport hook before leaving the process. Latency is measured from client.GetItem(ctx, input) call to response return — pure Go function call overhead.
HTTP measurement: Wall-clock round-trip from SDK call dispatch through localhost TCP to the CloudMock HTTP binary and back.
Metrics:
- P50 latency measured over warm iterations
- Startup time — wall clock from process/container start to first successful health check (HTTP) or first successful SDK call (In-Process)
- Memory — peak RSS after all operations complete; In-Process adds 0 extra memory since the mock runs in the same heap
- Cost — EC2
c5.largeon-demand pricing ($0.085/hr), 30-second mock lifetime per build, 1,000 builds/day, 365 days
Environment: Apple Silicon (darwin/arm64). CloudMock HTTP and LocalStack run as containers. Moto runs via moto_server in a container. CloudMock In-Process runs directly in the test binary. All containers use the same CPU and memory limits.
Date: April 1, 2026 — Go 1.26.1, darwin/arm64.
Reproduce it yourself:
# In-Process benchmark (Go tests only)go test ./benchmarks/... -run BenchmarkInProcess -bench=. -benchtime=50x
# HTTP mode benchmark (requires Docker)go run ./benchmarks/cmd/bench --target=all --mode=docker
# Quick run, specific servicesgo run ./benchmarks/cmd/bench --target=cloudmock --services=s3,dynamodb,sqs --quickResults are written to benchmarks/results/ as JSON.