November 12, 2026
How accurate is cloud simulation? Measured results on AWS and GCP
The short answer
A simulator is only useful if you know how far to trust it. So we ran our own load tests on AWS and GCP and compared them with what Cloud World Model predicted.
Here's what we found:
- AWS: Cloud World Model scores 93.8% on /accuracy against company-owned load tests of one architecture in us-east-2. The four traffic scenarios score 94.6% (idle, 10 RPS), 95.5% (normal, 100 RPS), 99.8% (peak, 500 RPS) and 85.5% (burst, 1,000 RPS).
- GCP: for one topology in us-central1 at 300–500 RPS, 5 of 6 P95 holdout runs were within 20% of the engine's predictions, made before the runs, and 1 was within 35%. All 6 P50 runs were within 20%.
- Where it misses: measured GCP P95 was slower than predicted at every load level. On AWS, most of the gap to 100% is in the 1,000 RPS burst scenario, which was held out of tuning. There, modeled latency was faster than measured and modeled app CPU was higher.
These results cover two specific architectures. They aren't a guarantee for every stack, and the rest of this post is about where the boundaries are.
What we measured
Both benchmarks use the same three-tier shape: a load balancer, two app servers and a MySQL database.
| AWS | GCP | |
|---|---|---|
| Region | us-east-2 | us-central1 |
| Load balancer | Application Load Balancer | External HTTP(S) load balancer |
| App servers | 2 × m5.large | 2 × n2-standard-2 managed instance group, running Node 20 |
| Database | RDS db.r5.large, MySQL, Single-AZ | Cloud SQL MySQL 8.0, 2 vCPU / 13 GiB, max_connections 500 |
| Measured loads | 10, 100, 500 and 1,000 RPS | 300, 400 and 500 RPS, two runs each |
| Used for tuning | 10, 100 and 500 RPS | Holdout against frozen predictions |
| Held out | 1,000 RPS burst | All six runs |
Every measurement comes from company-owned cwm-bench load tests, not provider documentation. On AWS, latency is measured in-VPC through an internal load balancer, so internet travel is excluded.
AWS: 93.8% across four scenarios
The headline score averages four scenarios, each scored on latency, app-server CPU, throughput, error rate and hourly cost. These figures come from /accuracy:
| Scenario | Scenario score | P50 latency (measured → modeled) | P95 latency (measured → modeled) | App-server CPU (measured → modeled) |
|---|---|---|---|---|
| Idle, 10 RPS (fit) | 94.6% | 2.63 → 2.5 ms | 4.53 → 4.4 ms | 0.48% → 0.39% |
| Normal, 100 RPS (fit) | 95.5% | 2.19 → 2.4 ms | 4.03 → 4.25 ms | 1.79% → 1.9% |
| Peak, 500 RPS (fit) | 99.8% | 2 → 2 ms | 3.76 → 3.75 ms | 8.63% → 8.61% |
| Burst, 1,000 RPS (holdout) | 85.5% | 1.93 → 1.45 ms | 3.71 → 3.1 ms | 13.36% → 17% |
Throughput and error rate scored 100% in all four scenarios. At 1,000 RPS, for example, the test measured 875.12 req/s of goodput, the model gave 875.12 req/s, and the run recorded one failed request in 1,050,142.
Where the remaining gap comes from. These are real misses, shown without retuning the engine:
- Burst latency: modeled P50 was 24.9% faster than measured, and modeled P95 was 16.5% faster.
- Burst CPU: modeled app-server CPU was 27.2% higher than measured.
- Idle CPU: modeled CPU was 18.3% lower than measured, a small absolute gap (0.39% vs 0.48%).
The weakest scenario is the burst, which was held out of tuning. That's the one that tells you most about loads the model hasn't been fitted to.
GCP: one topology, frozen predictions
For GCP, the engine's P95 and P50 predictions were frozen before the load tests ran. Then each load level was run twice. From the GCP holdout section of /accuracy:
| Offered load | Predicted P95 | Predicted P50 | P95 error (run 1 / run 2) | P50 error (run 1 / run 2) |
|---|---|---|---|---|
| 300 RPS | 68.7 ms | 11.2 ms | 11.8% / 3.1% | 3.8% / 4.8% |
| 400 RPS | 86.2 ms | 11.4 ms | 9.6% / 6.1% | 5.8% / 1.4% |
| 500 RPS | 103.7 ms | 11.6 ms | 20.4% / 5.9% | 15.9% / 6.0% |
What this shows:
- 5 of 6 P95 runs were within 20%. The sixth, the first 500 RPS run, was 20.4% off, which is within 35%.
- All 6 P50 runs were within 20%. The largest gap was 15.9%, also on the first 500 RPS run.
- No load level saturated, which matched the prediction.
What it doesn't show:
- Measured P95 was slower than predicted at every load level. The errors all point the same way. If you use these modeled P95 figures for a similar stack, treat them as a floor rather than a midpoint.
- This isn't an out-of-range test. The holdout loads were close to the range used for fitting.
- It covers one topology, one region and 300–500 RPS only. It says nothing about other GCP machine types, databases, regions, or loads above 500 RPS.
- Cost accuracy isn't scored for GCP yet, and P99/tail latency and connection counts aren't claimed.
What the two results have in common
On both clouds, the misses at the highest measured load lean the same way. The modeled latency was faster than the measurement. On AWS that showed up in the 1,000 RPS burst. On GCP, measured P95 was slower than predicted at every rung, and the biggest gap was at 500 RPS.
The practical takeaway is to leave headroom on modeled latency as load rises, and to load-test the architecture you actually plan to run. Simulation helps you narrow the options before deploying or changing infrastructure. It doesn't replace a final load test.
What these results don't cover
We'd rather you know the limits up front:
- Other architectures and providers. The measured evidence applies to the two architectures above. The multi-cloud burst report is an illustrative simulation, not a measured benchmark, and its GCP configuration (e2-standard-2, Cloud SQL db-n1-standard-2) isn't the measured GCP topology. Azure, OCI and DigitalOcean figures there are uncalibrated engine estimates.
- Cost. AWS hourly cost on /accuracy comes from the us-east-2 public price list, not a measured bill. GCP cost isn't scored. Neither result is a claim about cost accuracy.
- Tail latency. Neither benchmark makes P99 claims.
- Internet latency. AWS latency was measured in-VPC, so it excludes the network path to your users.
FAQ
How accurate is Cloud World Model on AWS? It scores 93.8% on /accuracy against company-owned load tests of an ALB → 2 × m5.large → db.r5.large MySQL stack in us-east-2. The scenario scores are 94.6% idle, 95.5% normal, 99.8% peak and 85.5% burst.
Has Cloud World Model been validated on GCP? For one topology only. In us-central1, an external HTTP(S) load balancer → 2 × n2-standard-2 managed instance group → Cloud SQL MySQL 8.0 (2 vCPU / 13 GiB) at 300–500 RPS landed 5 of 6 P95 holdout runs within 20% and 1 within 35%. All 6 P50 runs were within 20%. Other GCP topologies, regions and loads above 500 RPS haven't been validated.
Does the simulator over- or under-predict latency? In these tests it leaned optimistic at higher load. Measured GCP P95 was slower than predicted at every load level, and in the AWS 1,000 RPS burst, modeled P50 and P95 were faster than measured.
Does the accuracy score cover cost? No. AWS hourly cost is checked against the public price list, not a measured bill, and GCP cost isn't scored yet. Don't treat these results as a cost-accuracy claim.
Can my AI agent run these simulations? Yes. Cloud World Model is an MCP server at https://www.cloudworldmodel.ai/mcp. Coding agents can connect keyless to the demo tools and simulate an architecture before deploying or changing infrastructure. Simulations don't create resources in your cloud account.
Methodology and sources
- AWS results: Simulation Accuracy Benchmark, fetched November 12, 2026.
- Company-owned cwm-bench measurements at 10, 100, 500 and 1,000 offered RPS.
- 10, 100 and 500 RPS are fit runs; 1,000 RPS is a holdout.
- Latency is owned k6 median and P95, measured in-VPC through an internal load balancer.
- Hourly cost uses the AWS us-east-2 public price list, not a measured bill.
- The page links the pinned results export, the cwm-bench repository and the run code.
- GCP results: GCP holdout section of /accuracy, fetched November 12, 2026.
- Engine predictions were frozen before the runs, with two runs per load level at 300, 400 and 500 RPS.
- Predicted latencies are shown as the page rounds them.
- Multi-cloud burst report: /benchmark, fetched November 12, 2026. It's an illustrative simulation, not a measured benchmark.
Simulate the cloud without the cloud
Cloud World Model lets you simulate cloud infrastructure, keyless, before deploying or changing infrastructure. You can model traffic, failures and modeled cost, and you can check /accuracy to see how far the model has been measured. Connect your coding agent to the hosted MCP server at https://www.cloudworldmodel.ai/mcp.