Simulation Accuracy Benchmark

Name: Cloud World Model Accuracy Benchmark
Creator: Cloud World Model

Company-owned cwm-bench measurements for ALB → 2 × m5.large → db.r5.large MySQL in us-east-2 across 10, 100, 500, and 1,000 RPS, with engine comparison status shown honestly.

Download the Accuracy dataset (JSON)

Lean overall benchmark

This page publishes a reproducible, company-owned cwm-bench measurement of the canonical AWS architecture and clearly separates it from engine predictions and provider-documentation references.

What this benchmark demonstrates

The canonical AWS ALB → 2 × m5.large → db.r5.large MySQL setup in us-east-2 is compared against company-owned cwm-bench measurements at 10, 100, 500, and 1,000 offered RPS. App CPU, in-VPC load-balancer latency, goodput, and CRUD errors use those measurements; hourly cost uses the public AWS price list, not a measured bill. The 10 / 100 / 500 RPS runs inform tuning, while the 1,000 RPS run and later-day and us-west-2 runs test transfer outside that fit. This is evidence for this architecture and workload, not a universal accuracy guarantee. Other providers' reference comparisons are documentation-based, not owned load tests or a cross-provider ranking.

AWS scored app CPU, in-VPC internal-load-balancer latency, goodput and CRUD errors come from the pinned owned cwm-bench campaign. Cost uses the us-east-2 price list. Tuning uses 10 / 100 / 500 RPS; 1,000 RPS, later-day and us-west-2 are holdouts. See the Methodology section below for full source citations.

Overall score: 93.8%. The accuracy formula and weighted metric scores are listed below, and the four-scenario results are generated by the same API used by the client.

Scoring basis: measured_cwm_bench; cost reference: public_price_list. Canonical AWS compares seeded engine app-server CPU and modeled in-VPC internal-load-balancer latency with owned cwm-bench measurements. Goodput/errors are owned; billed cost uses the us-east-2 price list. CPU and latency misses are shown without engine retuning.

Architecture: {"name":"AWS 3-Tier Web App","description":"Standard three-tier web application: Application Load Balancer distributing traffic across two m5.large EC2 instances backed by a single db.r5.large RDS MySQL instance (Single-AZ, us-east-2).","provider":"aws","region":"us-east-2","referenceRegion":"us-east-2","components":[{"role":"Load Balancer","resourceType":"network","instanceType":"ALB (Application Load Balancer)","count":1,"officialHourlyCost":0.0225,"notes":"AWS Application Load Balancer base charge (us-east-2). Scales automatically."},{"role":"Web / App Server","resourceType":"compute","instanceType":"m5.large","count":2,"officialHourlyCost":0.096,"notes":"General-purpose instance: 2 vCPU, 8 GiB RAM, up to 10 Gbps network bandwidth."},{"role":"Database","resourceType":"database","instanceType":"db.r5.large (MySQL 8.0, Single-AZ)","count":1,"officialHourlyCost":0.24,"notes":"Memory-optimized: 2 vCPU, 16 GiB RAM. gp2 storage with 3,000 IOPS burst. ~500 max connections."}]}

Idle — 10 RPS

Minimal background traffic — keep-alive checks, health probes, and occasional real requests.

Scenario score: 94.6%. Comparison status: complete.

Idle reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 in-VPC LB latency	2.63 ms	2.5 ms	-4.9%	95.1%
P95 in-VPC LB latency	4.53 ms	4.4 ms	-2.9%	97.1%
App-server CPU	0.48 %	0.39 %	-18.3%	81.7%
Throughput	8.87 req/s	8.87 req/s	0%	100%
Error Rate	0 %	0 %	0%	100%
Cost / Hour	0.4545 $/hr	0.4545 $/hr	0%	100%

latencyP50Ms / latencyP95Ms: Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), idle fit run. Source
cpuPct: Owned app-server CPU mean of two application hosts; database CPU measured separately, idle fit run. Source
throughputRps / errorRatePct: Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 8.874155959479843 RPS goodput and 0% errors at a 10 RPS target. Source
costPerHour: AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr. Source

Owned fit measurement: target 10 RPS, goodput 8.9 RPS, errors 0%, app CPU 0.48%, DB CPU 3.34%, blended CPU 1.43%, maximum DB connections 2. Fit rung; goodput is the campaign's five-minute warmup plus fifteen-minute steady-window average.

Normal — 100 RPS

Typical business-hours traffic with steady sustained load.

Scenario score: 95.5%. Comparison status: complete.

Normal reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 in-VPC LB latency	2.19 ms	2.4 ms	9.7%	90.3%
P95 in-VPC LB latency	4.03 ms	4.25 ms	5.5%	94.5%
App-server CPU	1.79 %	1.9 %	6%	94%
Throughput	87.62 req/s	87.62 req/s	0%	100%
Error Rate	0 %	0 %	0%	100%
Cost / Hour	0.4545 $/hr	0.4545 $/hr	0%	100%

latencyP50Ms / latencyP95Ms: Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), normal fit run. Source
cpuPct: Owned app-server CPU mean of two application hosts; database CPU measured separately, normal fit run. Source
throughputRps / errorRatePct: Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 87.62477401512365 RPS goodput and 0% errors at a 100 RPS target. Source
costPerHour: AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr. Source

Owned fit measurement: target 100 RPS, goodput 87.6 RPS, errors 0%, app CPU 1.79%, DB CPU 4.43%, blended CPU 2.67%, maximum DB connections 8. Fit rung; goodput is the campaign's five-minute warmup plus fifteen-minute steady-window average.

Peak — 500 RPS

Peak business load — product launch, end-of-day batch, or marketing campaign spike.

Scenario score: 99.8%. Comparison status: complete.

Peak reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 in-VPC LB latency	2 ms	2 ms	0.2%	99.8%
P95 in-VPC LB latency	3.76 ms	3.75 ms	-0.3%	99.7%
App-server CPU	8.63 %	8.61 %	-0.2%	99.8%
Throughput	437.62 req/s	437.62 req/s	0%	100%
Error Rate	0 %	0 %	0%	100%
Cost / Hour	0.4545 $/hr	0.4545 $/hr	0%	100%

latencyP50Ms / latencyP95Ms: Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), peak fit run. Source
cpuPct: Owned approximate app-server CPU; database CPU measured separately, peak fit run. Source
throughputRps / errorRatePct: Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 437.62281962650434 RPS goodput and 0% errors at a 500 RPS target. Source
costPerHour: AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr. Source

Owned fit measurement: target 500 RPS, goodput 437.6 RPS, errors 0%, app CPU 8.63%, DB CPU 8.22%, blended CPU 8.49%, maximum DB connections 43. Fit rung; app CPU and DB CPU were reported as approximate campaign values.

Burst — 1000 RPS

Traffic burst exceeding normal peak — flash sale, viral event, or coordinated load test.

Scenario score: 85.5%. Comparison status: complete.

Burst reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 in-VPC LB latency	1.93 ms	1.45 ms	-24.9%	75.1%
P95 in-VPC LB latency	3.71 ms	3.1 ms	-16.5%	83.5%
App-server CPU	13.36 %	17 %	27.2%	72.8%
Throughput	875.12 req/s	875.12 req/s	0%	100%
Error Rate	0.000095225 %	0 %	-100%	100%
Cost / Hour	0.4545 $/hr	0.4545 $/hr	0%	100%

latencyP50Ms / latencyP95Ms: Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), burst holdout run. Source
cpuPct: Owned approximate app-server CPU; database CPU measured separately, burst holdout run. Source
throughputRps / errorRatePct: Owned Burst holdout measured 875.1162528375527 RPS whole-run goodput and one failure in 1,050,142 (0.0000952%). Five-minute ramp plus fifteen-minute steady window; no dropped requests implied. Source
costPerHour: AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr. Source

Owned holdout measurement: target 1000 RPS, goodput 875.1 RPS, errors 0.00009522521716110772%, app CPU 13.36%, DB CPU 10.91%, blended CPU 12.54%, maximum DB connections 55. Holdout rung; one failure in 1,050,142 requests (raw fraction 9.522521716110773e-07, or 0.0000952%). Goodput includes a five-minute ramp and fifteen-minute steady window; no dropped requests are implied.

Sources: cwm-bench repository · Pinned results export · Run code · AWS public price list.

Lean burst holdout (not used for tuning)

85.5% accurate · 3 tuning runs + 1 holdout.

This 93.8% overall result and burst holdout measure lean app weight. Self-serve and the GitHub App default to typical; its separate owned 300 RPS holdout below is not included in the lean overall score.

Throughput largely follows the prescribed workload; cost uses the AWS public price list, not a measured bill.

Target 1000 req/s in us-east-2; goodput 875.1 req/s; errors 0.00009522521716110772%; app CPU 13.36%; DB CPU 10.91%; blended CPU 12.54%; maximum DB connections 55. RDS BurstBalance was not exhausted. This holdout has no separate prediction and is not a capacity claim.

Holdout rung; one failure in 1,050,142 requests (raw fraction 9.522521716110773e-07, or 0.0000952%). Goodput includes a five-minute ramp and fifteen-minute steady window; no dropped requests are implied.

Typical app weight (owned benchmark)

300 RPS typical holdout: 27.0 → 82.4 without cost (24.3 → 84.2 on page cost basis; before/after cost bases differ). Later-day and us-west-2 holdouts: 76.0 and 81.1. Separate frozen before- and after-fit comparison for the exact owned typical setup; not part of the lean overall score.

300 RPS holdout — before fit: 24.3/100 (27.0 without cost)

300 RPS independent holdout — after fit: 84.2/100 (82.4 without cost)

84.2 is the original typical-v1 after-fit 300 RPS holdout. Later-day (us-east-2) and second-region (us-west-2) runs at 300 RPS scored 76.0 and 81.1 on the page cost basis (73.3 and 79.0 without cost, versus 82.4). These are observed scores from three single runs, not an expected accuracy range. Day and region holdouts

The main weakness is P95: the model predicts 12.78 ms, but the three runs measured 26.9 ms (original), 32.9 ms (us-west-2), and 51.1 ms (later-day us-east-2), a consistent 2× to 4× underprediction.

100 RPS repeatability: 95.6 (later-day us-east-2) and 91.4 (us-west-2).

Predictions frozen on engine 1.2.5; reproduced exactly on engine 1.2.13. aws-crud/typical:typical-v1-20260927c:6aa574d7ff9d3080b88b221bcd59f7d218ae37f0. Only typical-fit-20, typical-fit-100, typical-fit-200 were used for fitting. Neither 300 nor 500 RPS was used to tune the fit. The matching workload/topology is measured through the 300 RPS holdout; the 301–500 RPS predictions are diagnostic extrapolations, not validated measurements.

Independent cwm-bench P50 readings: 5.27 ms at 20 RPS, 4.68 ms at 100 RPS, 4.61 ms at 200 RPS, 4.62 ms at the 300 RPS holdout, and 5.19 ms at 500 RPS. The fit-rung P50 falls as load rises; these single-campaign readings do not show that higher load makes an app faster. Above 300 RPS, the existing owned fit is extrapolated for diagnostic use only; it is not measured or validated evidence of critical behavior or saturation. The 500 RPS holdout is not used to fit, and this boundary does not change the fit rungs or the 300 RPS holdout score. P50 is held at its 300 RPS projection rather than extending the negative fit slope.

Typical-v1 300 RPS after-fit score metrics
Metric	Predicted	Measured / reference	Metric score	Weight	Weighted contribution
P50	4.17 ms	4.62 ms	90.1	0.20	18.0
P95	12.78 ms	26.92 ms	47.5	0.25	11.9
App CPU	14.74 %	15.30 %	96.3	0.20	19.3
Throughput	262.60 RPS	262.61 RPS	100.0	0.15	15.0
Error rate	0.0000 %	0.0044 %	100.0	0.10	10.0
Cost	0.454581 USD/hour	0.4545 USD/hour (reference)	100.0	0.10	10.0

Cost: Cost was not measured. USD 0.4545/hour is a fixed-fee public list-price reference, not a bill. The frozen post-fit scored prediction predates modeled LB variable processing. It excludes modeled egress; the historical before-fit prediction included it, so their cost scores are not on identical bases.

500 RPS diagnostic comparison — 73.8/100 (70.9 without cost)

Predictions frozen on engine 1.2.5; reproduced exactly on engine 1.2.13. Owned fit aws-crud/typical:typical-v1-20260927c:6aa574d7ff9d3080b88b221bcd59f7d218ae37f0, using only typical-fit-20, typical-fit-100, typical-fit-200. Diagnostic extrapolation beyond the calibrated and validated range (fit at 20, 100, and 200 RPS; validated at 300 RPS), compared with a measured 500 RPS diagnostic run; excluded from headline accuracy and not a validated saturation finding.

Typical-v1 500 RPS diagnostic comparison
Metric	Predicted	Measured / reference	Metric score
P50	4.165 ms	5.188 ms	80.3
P95	16.333 ms	88.190 ms	18.5
App CPU	24.264 %	26.342 %	92.1
Throughput	437.588 RPS	437.577 RPS	100.0
Error rate	0.000 %	0.010 %	97.2
Cost	0.454500 USD/hour	0.4545 USD/hour (reference)	100.0

Throughput: Measured throughput is whole-run goodput averaged over the 5-minute warmup plus 15-minute steady window; engine throughput is a post-step point rate, not a duration-weighted average. The post-fit model was fit to whole-run goodput at 20, 100, and 200 RPS. Cost: USD 0.4545/hour is a fixed-fee public list-price reference, not a measured bill. Fixed-fee public price-list comparison, not a measured bill: scored prediction uses itemized resource costs before display rounding and excludes modeled egress and load-balancer variable data processing/ALB LCUs. The load-balancer base hourly fee remains included; other resource charges are not removed.

One campaign, one day, us-east-2, one attempt per rung; engine 1.2.3 predictions frozen before measurement (seed 20240601, step 6). Measured cost missing. Internal ALB + 2 × m5.large + db.r5.large MySQL 8.0; two workers and pool 250 per server.

Typical-v1 300 RPS before-fit score metrics
Metric	Predicted	Measured / reference	Metric score	Weight	Weighted contribution
P50	17.00 ms	4.62 ms	0.0	0.20	0.0
P95	42.00 ms	26.92 ms	44.0	0.25	11.0
App CPU	59.55 %	15.30 %	0.0	0.20	0.0
Throughput	299.00 RPS	262.61 RPS	86.1	0.15	12.9
Error rate	0.2840 %	0.0044 %	3.6	0.10	0.4
Cost	0.940000 USD/hour	0.4545 USD/hour (reference)	0.0	0.10	0.0

P99 (unweighted context, not part of the score): 236 vs 97.5 ms; score 0.

Throughput: Measured throughput is whole-run goodput averaged over the 5-minute warmup plus 15-minute steady window; engine throughput is a post-step point rate, not a duration-weighted average.

Cost: Cost was not measured. Score uses the list-price reference of USD 0.4545 per hour (ALB 0.0225 + 2 × 0.096 + 0.24, no egress, no generator). Reference, not a bill.

Optional, unscored steady-window context: 300.0 RPS gives throughput 99.7 and total 26.3.

Historical pre-fit context — fit data, not independent holdout accuracy; the 500 RPS observation remains separate from fitting.

fit-20 (fit data, not holdout): 23.0 (15.3 without cost)
fit-100 (fit data, not holdout): 19.8 (14.9 without cost)
fit-200 (fit data, not holdout): 16.1 (14.8 without cost)
500 RPS historical diagnostic (diagnostic only, not validated saturation): 25.6 (28.5 without cost)

Sources: cwm-bench typical report · summary export · scores · frozen predictions · measurement commit 6aa574d7ff9d.

Adding or removing app servers (typical-scale-v1)

Nine configurations across one, two, and three m5.large app servers at 100, 200, and 300 total RPS, with three independent repetitions per configuration (27 runs). Predictions were frozen on engine 1.2.14 before any AWS run.

In this benchmark, CWM predicts app CPU changes but fails to predict the measured P95 latency changes under the preregistered rules. CPU matched within a few percent in every configuration.

At 300 RPS, measured P95 was 95.0 ms with one server, 41.3 ms with two, and 20.7 ms with three; CWM predicted 18.1, 12.8, and 11.0 ms.

2→1 servers: Does not predict the change. 2→3 servers: Does not predict the change.

Measured and predicted P95 latency and the total no-egress page-style score for each server-count and traffic configuration.
App servers	Total RPS	Measured P95 (median)	Predicted P95	Total no-egress score (median; min–max)
1	100	9.1 ms	11.0 ms	89.1/100 (81.5–93.6)
1	200	37.5 ms	14.6 ms	82.1/100 (79.8–83.8)
1	300	95.0 ms	18.1 ms	74.5/100 (74.0–75.0)
2	100	8.4 ms	9.2 ms	95.7/100 (94.4–97.1)
2	200	15.1 ms	11.0 ms	89.8/100 (81.1–95.8)
2	300	41.3 ms	12.8 ms	77.7/100 (77.5–80.3)
3	100	8.7 ms	8.6 ms	98.4/100 (96.6–98.4)
3	200	12.1 ms	9.8 ms	94.7/100 (90.4–94.9)
3	300	20.7 ms	11.0 ms	86.8/100 (84.9–89.2)

Validated scope: us-east-2, m5.large app servers (1–3), 100–300 total RPS, internal load balancer, and db.r5.large MySQL. Measurement date: 2026-10-06.

Measured cell metrics are medians across three repetitions. The total no-egress score is the median of the three per-rep page-style totals; the range in parentheses is the min–max of those totals. This is distinct from the report’s separate no-cost metric, which excludes the cost component.

No connection-limit errors occurred.

RDS version caveat: sessions 1–2 reported 8.0.46; session 3 reported 8.0.46-rds.20260908. The report flags this as a possible confound limited to session 3 and does not attribute performance changes to it.

A later typical-p95-v1 candidate was evaluated on a separate, preregistered holdout; those results appear below. They do not change this historical engine 1.2.14 comparison or its frozen scores.

Sources: cwm-bench PR #18 · Campaign report

Typical P95 model: independent holdout (typical-p95-v1)

Campaign typical-p95-v1-20261010: 9 valid applies and 27 traffic rungs across three UTC days, with three independent repetitions for each configuration. Predictions were frozen before measurement; the model was not refit on these runs.

Both preregistered verdicts passed: all 9 cell medians were inside the candidate’s acceptance tolerances, with a median P95-only score of 87.6; the server-count change verdict also passed. The candidate beat the frozen engine 1.2.18 baseline at all three 300 RPS cells.

The 87.6 is the median of nine P95-only cell scores, not a replacement for the Accuracy page’s overall score. Each cell score is computed against that cell’s measured median; the preregistered ×0.666–×1.432 bands are acceptance tolerances, not predictive or confidence intervals.

Independent typical-p95-v1 holdout: measured median P95, frozen candidate and engine 1.2.18 predictions, and P95-only scores for each server-count and traffic cell.
App servers	Total RPS	Measured P95 (median)	Candidate P95	Frozen engine 1.2.18 P95	P95-only score (candidate / engine)
1	100	8.6 ms	9.6 ms	11.0 ms	89.0 / 72.6
1	200	40.3 ms	32.0 ms	14.6 ms	79.3 / 36.1
1	300	94.7 ms	106.5 ms	18.1 ms	87.6 / 19.1
2	100	9.2 ms	8.0 ms	9.2 ms	87.1 / 99.5
2	200	15.7 ms	15.0 ms	11.0 ms	95.7 / 70.2
2	300	38.2 ms	34.3 ms	12.8 ms	89.7 / 33.5
3	100	7.9 ms	8.0 ms	8.6 ms	99.1 / 91.0
3	200	9.0 ms	11.7 ms	9.8 ms	71.2 / 91.4
3	300	17.9 ms	23.5 ms	11.0 ms	68.8 / 61.5

Measurement dates: 2026-10-10, 2026-10-12. Validated scope: us-east-2, m5.large app servers (1–3), 100–300 total RPS, internal load balancer, and db.r5.large MySQL. RDS was not saturated: CPU stayed at or below about 28%, and no connection-limit errors occurred. This does not validate database saturation or traffic above 300 RPS. All nine valid applies used RDS 8.0.46-rds.20260908. Observed request errors were at most 0.025% (0.017% at 1×/300 RPS), all unclassified.

Aggregation: each measured P95 is the median of three independent repetitions for that cell. It is k6 client-side http_req_duration p(95) over the complete rung, including the 5-minute warmup ramp and 15-minute steady window; the per-cell score uses the measured median, and 87.6 is the median of those nine scores. The candidate column is the frozen typical-p95-v1 model output; engine 1.2.18 is the frozen baseline comparator, not the current engine.

The server-change verdict passed for both 300 RPS transitions and all four transitions at 100 and 200 RPS. For the 1→2 and 2→3 P95 ratios at 300 RPS, predicted ratios were 3.11 and 1.46; measured ratios of cell medians were 2.48 and 2.14.

The candidate still overpredicts at three servers: its P95-only scores are 71.2 at 200 RPS and 68.8 at 300 RPS. At 200 RPS, that score is below the frozen engine baseline’s 91.4, although both cells are within the preregistered tolerance.

This independent result is separate from the original typical-v1 two-server holdout and the earlier typical-scale-v1 engine 1.2.14 server-count comparison above. Those historical predictions and scores are unchanged; the older scale measurements informed model development and are not this holdout.

Sources: cwm-bench PR #21 · Holdout report · Preregistration

AWS canonical reference comparison

Overall score: 93.8%. Scoring basis: measured_cwm_bench; cost reference: public_price_list. Canonical AWS compares seeded engine app-server CPU and modeled in-VPC internal-load-balancer latency with owned cwm-bench measurements. Goodput/errors are owned; billed cost uses the us-east-2 price list. CPU and latency misses are shown without engine retuning.

Idle — 10 RPS

Minimal background traffic — keep-alive checks, health probes, and occasional real requests.

Scenario score: 94.6%. Comparison status: complete.

Idle reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 in-VPC LB latency	2.63 ms	2.5 ms	-4.9%	95.1%
P95 in-VPC LB latency	4.53 ms	4.4 ms	-2.9%	97.1%
App-server CPU	0.48 %	0.39 %	-18.3%	81.7%
Throughput	8.87 req/s	8.87 req/s	0%	100%
Error Rate	0 %	0 %	0%	100%
Cost / Hour	0.4545 $/hr	0.4545 $/hr	0%	100%

latencyP50Ms / latencyP95Ms: Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), idle fit run. Source
cpuPct: Owned app-server CPU mean of two application hosts; database CPU measured separately, idle fit run. Source
throughputRps / errorRatePct: Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 8.874155959479843 RPS goodput and 0% errors at a 10 RPS target. Source
costPerHour: AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr. Source

Normal — 100 RPS

Typical business-hours traffic with steady sustained load.

Scenario score: 95.5%. Comparison status: complete.

Normal reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 in-VPC LB latency	2.19 ms	2.4 ms	9.7%	90.3%
P95 in-VPC LB latency	4.03 ms	4.25 ms	5.5%	94.5%
App-server CPU	1.79 %	1.9 %	6%	94%
Throughput	87.62 req/s	87.62 req/s	0%	100%
Error Rate	0 %	0 %	0%	100%
Cost / Hour	0.4545 $/hr	0.4545 $/hr	0%	100%

latencyP50Ms / latencyP95Ms: Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), normal fit run. Source
cpuPct: Owned app-server CPU mean of two application hosts; database CPU measured separately, normal fit run. Source
throughputRps / errorRatePct: Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 87.62477401512365 RPS goodput and 0% errors at a 100 RPS target. Source
costPerHour: AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr. Source

Peak — 500 RPS

Peak business load — product launch, end-of-day batch, or marketing campaign spike.

Scenario score: 99.8%. Comparison status: complete.

Peak reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 in-VPC LB latency	2 ms	2 ms	0.2%	99.8%
P95 in-VPC LB latency	3.76 ms	3.75 ms	-0.3%	99.7%
App-server CPU	8.63 %	8.61 %	-0.2%	99.8%
Throughput	437.62 req/s	437.62 req/s	0%	100%
Error Rate	0 %	0 %	0%	100%
Cost / Hour	0.4545 $/hr	0.4545 $/hr	0%	100%

latencyP50Ms / latencyP95Ms: Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), peak fit run. Source
cpuPct: Owned approximate app-server CPU; database CPU measured separately, peak fit run. Source
throughputRps / errorRatePct: Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 437.62281962650434 RPS goodput and 0% errors at a 500 RPS target. Source
costPerHour: AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr. Source

Burst — 1000 RPS

Traffic burst exceeding normal peak — flash sale, viral event, or coordinated load test.

Scenario score: 85.5%. Comparison status: complete.

Burst reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 in-VPC LB latency	1.93 ms	1.45 ms	-24.9%	75.1%
P95 in-VPC LB latency	3.71 ms	3.1 ms	-16.5%	83.5%
App-server CPU	13.36 %	17 %	27.2%	72.8%
Throughput	875.12 req/s	875.12 req/s	0%	100%
Error Rate	0.000095225 %	0 %	-100%	100%
Cost / Hour	0.4545 $/hr	0.4545 $/hr	0%	100%

latencyP50Ms / latencyP95Ms: Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), burst holdout run. Source
cpuPct: Owned approximate app-server CPU; database CPU measured separately, burst holdout run. Source
throughputRps / errorRatePct: Owned Burst holdout measured 875.1162528375527 RPS whole-run goodput and one failure in 1,050,142 (0.0000952%). Five-minute ramp plus fifteen-minute steady window; no dropped requests implied. Source
costPerHour: AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr. Source

Fidelity components: cost price_list, latency owned, CPU owned, throughput owned, error rate owned.

GCP canonical reference comparison

Overall score: 98%. Scoring basis: documentation_reference; cost reference: documentation_reference. GCP uses its own latency and error coefficients fit to e2-standard-2 / db-n1-standard-2 reference data, plus a +5 ms idle base-latency offset (Cloud LB forwarding over AWS ALB) so idle P50 reproduces the documented floor. Latency, error rate (including Burst) and low/mid-load CPU track references closely; the main remaining gap is the shared (AWS-calibrated) database CPU curve, which slightly over-predicts CPU at Burst.

Architecture: {"name":"Custom GCP 3-Tier Web App","description":"Cloud Load Balancing distributing traffic across 2× e2-standard-2 VM instances backed by a single db-n1-standard-2 Cloud SQL (us-central1).","provider":"gcp","region":"us-central1","referenceRegion":"us-central1","components":[{"role":"Load Balancer","resourceType":"network","instanceType":"Cloud Load Balancing (forwarding rule)","count":1,"officialHourlyCost":0.025,"notes":"GCP Cloud Load Balancing per forwarding-rule charge (us-central1)."},{"role":"Web / App Server","resourceType":"compute","instanceType":"e2-standard-2","count":2,"officialHourlyCost":0.067,"notes":"General-purpose: 2 vCPU (dedicated Intel Ice Lake), 8 GiB RAM. Comparable tier to AWS m5.large."},{"role":"Database","resourceType":"database","instanceType":"db-n1-standard-2 (Cloud SQL)","count":1,"officialHourlyCost":0.1917,"notes":"Standard: 2 vCPU, 7.5 GiB RAM. ~300 max connections, SSD storage."}]}

Idle — 10 RPS

Minimal background traffic — keep-alive checks, health probes, and occasional real requests.

Scenario score: 98.3%. Comparison status: complete.

Idle reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	18 ms	18 ms	0%	100%
P95 Latency	42 ms	43 ms	2.4%	97.6%
CPU Utilization	6 %	6.3 %	5%	95%
Throughput	10 req/s	9.99 req/s	-0.1%	99.9%
Error Rate	0.1 %	0.1 %	-1%	100%
Cost / Hour	0.3507 $/hr	0.3484 $/hr	-0.7%	99.3%

latencyP50Ms / latencyP95Ms: GCP e2-standard-2 (dedicated-core) + Cloud LB: idle P50 ~16–20 ms, P95 ~38–46 ms. Cloud LB adds ~2–4 ms more overhead than AWS ALB; e2-standard-2 dedicated vCPUs behave similarly to AWS m5.large with comparable baseline latency. Source: GCP Load Balancing and e2 dedicated-core documentation. Source
cpuPct / errorRatePct: GCP e2-standard-2 at idle: ~5–7% CPU (OS + runtime baseline, similar to AWS dedicated-core at equivalent load). db-n1-standard-2 (300 max connections) at 10 RPS: ~2 active connections = 0.7% pool utilisation; transient errors ~0.08–0.12%. Source: GCP Compute and Cloud SQL n1-standard-2 documentation. Source
costPerHour: GCP Pricing (us-central1, August 2026): Cloud LB $0.025/hr + 2× e2-standard-2 $0.067/hr + db-n1-standard-2 $0.1917/hr = $0.3507/hr. Source

Normal — 100 RPS

Typical business-hours traffic with steady sustained load.

Scenario score: 99%. Comparison status: complete.

Normal reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	22 ms	23 ms	4.5%	95.5%
P95 Latency	52 ms	52 ms	0%	100%
CPU Utilization	22 %	22 %	0%	100%
Throughput	100 req/s	99.8 req/s	-0.2%	99.8%
Error Rate	0.2 %	0.2 %	0%	100%
Cost / Hour	0.3507 $/hr	0.3484 $/hr	-0.7%	99.3%

latencyP50Ms / latencyP95Ms: GCP e2-standard-2 (dedicated 2 vCPU) at ~50 RPS/instance: real-world P50 ~20–24 ms, P95 ~48–56 ms. The AWS-calibrated simulator under-predicts by ~10–15% because the queuing-theory latency formula does not model GCP LB's additional forwarding overhead. Source: GCP e2-standard performance docs and Cloud Monitoring latency guidance. Source
cpuPct: GCP e2-standard-2 (dedicated Intel core) at ~50 RPS/instance: effective reported CPU ~19–25%. Dedicated vCPUs report similarly to AWS m5.large at comparable utilisation. Source: GCP Cloud Monitoring CPU utilisation guidance for e2 standard machine types. Source
errorRatePct: GCP db-n1-standard-2 (300 max connections): at 100 RPS, ~20 concurrent connections = 6.7% pool utilisation. Well below saturation; real-world error rate ~0.15–0.25% from transient connection overhead. Source: Cloud SQL connection management documentation. Source

Peak — 500 RPS

Peak business load — product launch, end-of-day batch, or marketing campaign spike.

Scenario score: 97.9%. Comparison status: complete.

Peak reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	46 ms	46 ms	0%	100%
P95 Latency	105 ms	98 ms	-6.7%	93.3%
CPU Utilization	58 %	57.3 %	-1.2%	98.8%
Throughput	493 req/s	497 req/s	0.8%	99.2%
Error Rate	0.6 %	0.6 %	0%	100%
Cost / Hour	0.3507 $/hr	0.3484 $/hr	-0.7%	99.3%

errorRatePct / throughputRps: GCP db-n1-standard-2 (300 max connections) at 500 RPS: ~100 concurrent connections = 33% pool utilisation. At 33% utilisation with a dedicated 2-vCPU DB the connection queue rarely backs up; real-world error rate ~0.4–0.8%. Effective throughput ~493 RPS. Source: Cloud SQL operational guidance and GCP performance best practices. Source
latencyP50Ms / latencyP95Ms: GCP e2-standard-2 (dedicated vCPU) at ~250 RPS/instance: real-world P50 ~42–50 ms, P95 ~96–114 ms. The AWS-calibrated queuing-theory formula slightly under-predicts due to GCP LB forwarding overhead. Source: GCP Compute and Cloud Monitoring latency guidance. Source
cpuPct: GCP e2-standard-2 at 250 RPS/instance: CPU ~54–62%. Dedicated Intel vCPU utilisation profile is similar to AWS m5.large at equivalent per-instance load. Source: GCP e2 standard machine type performance documentation. Source

Burst — 1000 RPS

Traffic burst exceeding normal peak — flash sale, viral event, or coordinated load test.

Scenario score: 96.8%. Comparison status: complete.

Burst reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	82 ms	82 ms	0%	100%
P95 Latency	185 ms	169 ms	-8.6%	91.4%
CPU Utilization	84 %	88.1 %	4.9%	95.1%
Throughput	972 req/s	975 req/s	0.3%	99.7%
Error Rate	2.5 %	2.5 %	0%	100%
Cost / Hour	0.3507 $/hr	0.3484 $/hr	-0.7%	99.3%

errorRatePct / throughputRps: GCP db-n1-standard-2 (300 max connections) at 1000 RPS: ~200 concurrent connections = 67% pool utilisation. At 67% utilisation the connection pool begins to queue intermittently; real-world error rate ~2–3% from connection wait timeouts — comparable to AWS db.r5.large behaviour at similar relative utilisation. Source: Cloud SQL best practices. Source
latencyP50Ms / latencyP95Ms: GCP e2-standard-2 at ~500 RPS/instance under light DB connection pressure: real-world P50 ~76–88 ms, P95 ~174–196 ms. GCP hypervisor and LB overhead add ~10–15% vs AWS at equivalent load. Source: GCP load testing guidance and Cloud Monitoring latency documentation. Source
cpuPct: GCP e2-standard-2 at ~500 RPS/instance: CPU ~80–88% under sustained burst. Dedicated vCPU utilisation curve mirrors AWS m5.large at equivalent per-core RPS. Source: GCP Compute performance documentation. Source

Fidelity components: cost estimated, latency estimated, CPU estimated, throughput estimated, error rate estimated.

AZURE canonical reference comparison

Overall score: 98%. Scoring basis: documentation_reference; cost reference: documentation_reference. Azure uses its own latency and error coefficients fit to Standard_D2s_v3 / SQL Standard S2 reference data, plus a +3.75 ms idle base-latency offset so idle P50 reproduces the documented floor. Latency and error rate (including Burst) track references closely; the largest remaining divergence is at Burst, where the shared (AWS-calibrated) database CPU saturation curve over-predicts CPU for the 200-connection S2 tier.

Architecture: {"name":"Custom AZURE 3-Tier Web App","description":"Azure Load Balancer distributing traffic across 2× Standard_D2s_v3 VMs backed by a single sql-s2 SQL Database (eastus).","provider":"azure","region":"eastus","referenceRegion":"eastus","components":[{"role":"Load Balancer","resourceType":"network","instanceType":"Standard Load Balancer","count":1,"officialHourlyCost":0.025,"notes":"Azure Standard Load Balancer per-rule charge (East US)."},{"role":"Web / App Server","resourceType":"compute","instanceType":"Standard_D2s_v3","count":2,"officialHourlyCost":0.096,"notes":"General-purpose: 2 vCPU (non-burstable, dedicated Intel Broadwell), 8 GiB RAM. Comparable tier to AWS m5.large."},{"role":"Database","resourceType":"database","instanceType":"sql-s2 (SQL Database)","count":1,"officialHourlyCost":0.1008,"notes":"Standard S2: 50 DTUs, 250 GiB max size. Medium-traffic production apps."}]}

Idle — 10 RPS

Minimal background traffic — keep-alive checks, health probes, and occasional real requests.

Scenario score: 95.8%. Comparison status: complete.

Idle reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	16 ms	16 ms	0%	100%
P95 Latency	38 ms	42 ms	10.5%	89.5%
CPU Utilization	5 %	5.4 %	8%	92%
Throughput	10 req/s	9.99 req/s	-0.1%	99.9%
Error Rate	0.1 %	0.1 %	-1%	100%
Cost / Hour	0.3178 $/hr	0.3175 $/hr	-0.1%	99.9%

latencyP50Ms / latencyP95Ms: Azure Standard_D2s_v3 (non-burstable, dedicated Intel Broadwell) + Azure Standard LB: idle P50 ~14–18 ms, P95 ~34–42 ms. Azure LB adds ~2–3 ms more overhead than AWS ALB; D2s_v3 dedicated cores behave similarly to AWS m5.large at equivalent idle load. Source: Azure VM performance docs and Standard Load Balancer documentation. Source
cpuPct / errorRatePct: Azure Standard_D2s_v3 at idle: ~4–6% CPU (OS + runtime baseline, similar to AWS m5.large). Azure SQL S2 (200 max connections) at 10 RPS: ~2 active connections = 1% pool utilisation; transient errors ~0.08–0.12%. Source: Azure VM sizes documentation and Azure SQL S2 resource limits. Source
costPerHour: Azure Pricing (East US, August 2026): Standard LB $0.025/hr + 2× Standard_D2s_v3 $0.096/hr + SQL Standard S2 $0.1008/hr = $0.3178/hr. Source

Normal — 100 RPS

Typical business-hours traffic with steady sustained load.

Scenario score: 98.4%. Comparison status: complete.

Normal reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	21 ms	21 ms	0%	100%
P95 Latency	50 ms	52 ms	4%	96%
CPU Utilization	19 %	18.5 %	-2.6%	97.4%
Throughput	100 req/s	99.8 req/s	-0.2%	99.8%
Error Rate	0.2 %	0.2 %	0.5%	100%
Cost / Hour	0.3178 $/hr	0.3175 $/hr	-0.1%	99.9%

latencyP50Ms / latencyP95Ms: Azure Standard_D2s_v3 at ~50 RPS/instance: real-world P50 ~19–23 ms, P95 ~46–54 ms. The AWS-calibrated simulator slightly under-predicts by ~10% because the queuing-theory latency formula does not model Azure LB overhead. D2s_v3 dedicated vCPUs perform comparably to AWS m5.large per-core. Source: Azure VM performance benchmarks. Source
cpuPct: Azure Standard_D2s_v3 (non-burstable, dedicated 2-vCPU) at ~50 RPS/instance: effective CPU ~17–21%. Dedicated core metrics behave identically to AWS m5.large monitoring — no B-series credit model complexity. Source: Azure Monitor CPU metrics for Dv3 series documentation. Source
errorRatePct: Azure SQL Standard S2 (200 max connections, 50 DTUs) at 100 RPS: ~20 concurrent connections = 10% pool utilisation. Well below saturation; real-world error rate ~0.15–0.25%. Source: Azure SQL Database Standard S2 resource limits documentation. Source

Peak — 500 RPS

Peak business load — product launch, end-of-day batch, or marketing campaign spike.

Scenario score: 99.3%. Comparison status: complete.

Peak reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	42 ms	42 ms	0%	100%
P95 Latency	98 ms	98 ms	0%	100%
CPU Utilization	54 %	52.3 %	-3.1%	96.9%
Throughput	492 req/s	496 req/s	0.8%	99.2%
Error Rate	0.8 %	0.8 %	0%	100%
Cost / Hour	0.3178 $/hr	0.3175 $/hr	-0.1%	99.9%

errorRatePct / throughputRps: Azure SQL S2 (200 max connections, 50 DTUs) at 500 RPS: ~100 concurrent connections = 50% pool utilisation. At 50% utilisation with dedicated DTU capacity, connection queuing is minimal; real-world error rate ~0.5–1.1%. Effective throughput ~492 RPS. Source: Azure SQL DTU service tier documentation. Source
latencyP50Ms / latencyP95Ms: Azure Standard_D2s_v3 (dedicated vCPU) at ~250 RPS/instance: real-world P50 ~38–46 ms, P95 ~90–106 ms. The AWS-calibrated model slightly under-predicts due to Azure LB overhead; dedicated non-burstable vCPUs perform similarly to AWS m5.large at peak. Source: Azure VM benchmark documentation. Source
cpuPct: Azure Standard_D2s_v3 at 250 RPS/instance: CPU ~50–58%. Non-burstable dedicated cores remain in a predictable linear utilisation range, unlike B-series. Source: Azure Monitor D-series VM performance documentation. Source

Burst — 1000 RPS

Traffic burst exceeding normal peak — flash sale, viral event, or coordinated load test.

Scenario score: 98.3%. Comparison status: complete.

Burst reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	76 ms	76 ms	0%	100%
P95 Latency	174 ms	171 ms	-1.7%	98.3%
CPU Utilization	81 %	86.1 %	6.3%	93.7%
Throughput	962 req/s	962 req/s	0%	100%
Error Rate	3.8 %	3.8 %	0%	100%
Cost / Hour	0.3178 $/hr	0.3175 $/hr	-0.1%	99.9%

errorRatePct / throughputRps: Azure SQL S2 (200 max connections) at 1000 RPS: ~200 concurrent connections = 100% pool utilisation. At full pool saturation, connection wait timeouts begin; real-world error rate ~3–4.5%. Effective throughput ~962 RPS (1000 × 0.962). Source: Azure SQL performance guidance. Source
latencyP50Ms / latencyP95Ms: Azure Standard_D2s_v3 at ~500 RPS/instance with moderate DB connection pressure: real-world P50 ~70–82 ms, P95 ~162–186 ms. Non-burstable dedicated cores maintain predictable latency curves at high load. Source: Azure load testing guidance. Source
cpuPct: Azure Standard_D2s_v3 at ~500 RPS/instance: CPU ~77–85% under burst. Non-burstable cores do not throttle at high utilisation (unlike B-series), so CPU scales linearly. Source: Azure D-series VM performance documentation. Source

Fidelity components: cost estimated, latency estimated, CPU estimated, throughput estimated, error rate estimated.

OCI canonical reference comparison

Overall score: 96.2%. Scoring basis: documentation_reference; cost reference: documentation_reference. OCI uses its own latency, error, CPU, tail and throughput-shed coefficients fit to VM.Standard.E4.Flex 2-OCPU / MySQL HeatWave reference data, plus a +6.25 ms idle base-latency offset (EPYC + Flexible LB overhead) so idle P50 reproduces the documented floor. HeatWave's low in-memory error rates are captured by the fitted polynomial (Burst error matches closely), and a connection-pool goodput-shed term reproduces the documented Peak/Burst effective throughput (which sits below traffic × (1 − errorRate) due to admission back-pressure). The residual is in CPU: OCI's reference curve is unusually steep (low Normal 12 % yet high Burst 78 %), which the shared √-shaped compute curve plus the shared database CPU curve cannot fully track — leaving CPU slightly over-predicted at Normal and under-predicted at Peak/Burst.

Architecture: {"name":"Custom OCI 3-Tier Web App","description":"OCI Flexible Load Balancer distributing traffic across 2× VM.Standard.E4.Flex-2OCPU VM instances backed by a single mysql-heatwave Database (us-ashburn-1).","provider":"oci","region":"us-ashburn-1","referenceRegion":"us-ashburn-1","components":[{"role":"Load Balancer","resourceType":"network","instanceType":"Flexible Load Balancer (10 Mbps–8 Gbps)","count":1,"officialHourlyCost":0.0224,"notes":"OCI Flexible Load Balancer base charge (us-ashburn-1). Bandwidth charged separately."},{"role":"Web / App Server","resourceType":"compute","instanceType":"VM.Standard.E4.Flex-2OCPU","count":2,"officialHourlyCost":0.074,"notes":"Flexible: 2 OCPU (4 AMD EPYC vCPUs), 16 GiB RAM. Comparable tier to AWS m5.large (us-ashburn-1)."},{"role":"Database","resourceType":"database","instanceType":"mysql-heatwave (Database)","count":1,"officialHourlyCost":0.074,"notes":"MySQL HeatWave DB System: 2 OCPU, 16 GiB RAM (E4.Flex). OLTP + in-database analytics (us-ashburn-1)."}]}

Idle — 10 RPS

Minimal background traffic — keep-alive checks, health probes, and occasional real requests.

Scenario score: 94.1%. Comparison status: complete.

Idle reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	20 ms	21 ms	5%	95%
P95 Latency	46 ms	46 ms	0%	100%
CPU Utilization	4 %	4.3 %	7.5%	92.5%
Throughput	10 req/s	9.99 req/s	-0.1%	99.9%
Error Rate	0.15 %	0.1 %	-31.3%	68.7%
Cost / Hour	0.2444 $/hr	0.25 $/hr	2.3%	97.7%

latencyP50Ms / latencyP95Ms: OCI VM.Standard.E4.Flex 2-OCPU (4 AMD EPYC vCPUs) + OCI Flexible LB: idle P50 ~18–22 ms, P95 ~42–50 ms. OCI Flexible LB adds ~4–6 ms more round-trip overhead than AWS ALB; EPYC vCPU scheduling latency is similar to x86 at idle. Source: OCI Load Balancer and Compute performance documentation. Source
cpuPct / errorRatePct: OCI E4.Flex 2-OCPU at idle: ~3–5% CPU (OS + runtime on 4 EPYC vCPUs — lower relative utilisation than 1-OCPU due to more available cores). MySQL HeatWave (300 max connections) at 10 RPS: ~2 active connections; OCI network fabric transient errors ~0.12–0.18%. Source: OCI MySQL HeatWave documentation. Source
costPerHour: OCI Pricing (us-ashburn-1, August 2026): Flexible LB $0.0224/hr + 2× E4.Flex 2-OCPU ($0.074/hr each: 2 OCPU × $0.025 + 16 GB × $0.0015) + MySQL HeatWave $0.074/hr = $0.2444/hr. Source

Normal — 100 RPS

Typical business-hours traffic with steady sustained load.

Scenario score: 94.1%. Comparison status: complete.

Normal reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	34 ms	32 ms	-5.9%	94.1%
P95 Latency	76 ms	67 ms	-11.8%	88.2%
CPU Utilization	12 %	12.1 %	0.8%	99.2%
Throughput	100 req/s	99.8 req/s	-0.2%	99.8%
Error Rate	0.18 %	0.2 %	13.3%	86.7%
Cost / Hour	0.2444 $/hr	0.25 $/hr	2.3%	97.7%

latencyP50Ms / latencyP95Ms: OCI E4.Flex 2-OCPU at ~50 RPS/instance: real-world P50 ~30–38 ms, P95 ~70–82 ms. The AWS-calibrated simulator under-predicts by ~20–25% because the latency formula does not model OCI's EPYC scheduling overhead or OCI LB network latency. With 4 vCPUs per instance, queuing at 50 RPS is minimal. Source: OCI Compute performance documentation. Source
cpuPct: OCI E4.Flex 2-OCPU (4 EPYC vCPUs) at ~50 RPS/instance: effective CPU ~10–14%. With 4 vCPUs handling 50 RPS, per-core utilisation is low; the AWS-calibrated sqrt-of-utilisation formula over-predicts because it was tuned to 2-vCPU m5.large scaling characteristics. Source: OCI Compute performance documentation. Source
errorRatePct: MySQL HeatWave (300 max connections) at 100 RPS: ~20 concurrent connections = 6.7% pool utilisation. HeatWave in-memory query acceleration keeps connection hold times short; real-world error rate ~0.14–0.22%. Source: OCI MySQL HeatWave documentation. Source

Peak — 500 RPS

Peak business load — product launch, end-of-day batch, or marketing campaign spike.

Scenario score: 98%. Comparison status: complete.

Peak reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	82 ms	83 ms	1.2%	98.8%
P95 Latency	162 ms	162 ms	0%	100%
CPU Utilization	48 %	44.4 %	-7.5%	92.5%
Throughput	487 req/s	487 req/s	0%	100%
Error Rate	0.35 %	0.35 %	0%	100%
Cost / Hour	0.2444 $/hr	0.25 $/hr	2.3%	97.7%

latencyP50Ms / latencyP95Ms: OCI E4.Flex 2-OCPU at ~250 RPS/instance: real-world P50 ~75–89 ms, P95 ~150–174 ms. The AWS-calibrated queuing-theory formula under-predicts OCI latency by ~20–25% at peak due to EPYC scheduling and OCI LB overhead. With 4 vCPUs at 250 RPS, the instance has meaningful headroom. Source: OCI performance best practices. Source
cpuPct: OCI E4.Flex 2-OCPU at 250 RPS/instance: real-world CPU ~44–52%. 4 EPYC vCPUs at 250 RPS load produce roughly half the per-core utilisation compared to a 2-vCPU AWS m5.large. Source: OCI Compute performance documentation. Source
errorRatePct / throughputRps: MySQL HeatWave at 500 RPS: ~100 concurrent connections = 33% pool utilisation (300 max). HeatWave in-memory acceleration returns query results before connection hold time causes pool pressure; real-world error rate ~0.25–0.45%. Effective throughput ~487 RPS. Source: OCI MySQL HeatWave benchmarks. Source

Burst — 1000 RPS

Traffic burst exceeding normal peak — flash sale, viral event, or coordinated load test.

Scenario score: 98.5%. Comparison status: complete.

Burst reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	155 ms	155 ms	0%	100%
P95 Latency	298 ms	297 ms	-0.3%	99.7%
CPU Utilization	78 %	82.6 %	5.9%	94.1%
Throughput	882 req/s	883 req/s	0.1%	99.9%
Error Rate	2.8 %	2.8 %	0%	100%
Cost / Hour	0.2444 $/hr	0.25 $/hr	2.3%	97.7%

latencyP50Ms / latencyP95Ms: OCI E4.Flex 2-OCPU at ~500 RPS/instance: real-world P50 ~144–166 ms, P95 ~280–316 ms. OCI LB + EPYC scheduling overhead add ~15–20% vs the AWS-calibrated model at equivalent per-instance RPS. Source: OCI performance documentation. Source
cpuPct: OCI E4.Flex 2-OCPU at 500 RPS/instance: CPU ~74–82% as the 4 EPYC cores approach saturation under combined compute + network I/O load. Still below the saturation threshold a 2-vCPU instance would hit at the same RPS. Source: OCI Compute performance docs. Source
errorRatePct / throughputRps: MySQL HeatWave at 1000 RPS: ~200 connections = 67% pool utilisation (300 max). HeatWave in-memory processing keeps connection hold times low; real-world error rate ~2.2–3.4% at 67% pool utilisation — lower than a disk-based DB at equivalent utilisation. Effective throughput ~882 RPS. Source: OCI MySQL HeatWave documentation. Source

Fidelity components: cost estimated, latency estimated, CPU estimated, throughput estimated, error rate estimated.

DIGITALOCEAN canonical reference comparison

Overall score: 97.4%. Scoring basis: documentation_reference; cost reference: documentation_reference. DigitalOcean uses its own latency and error coefficients fit to s-4vcpu-8gb / db-s-2vcpu-4gb reference data, plus a +9 ms idle base-latency offset (shared-tenant network fabric) so idle P50 reproduces the documented floor. Latency and CPU track references across all tiers; the main remaining divergence is the very low documented error rates at Idle/Normal, where sub-0.5 % absolute references make small noise-floor differences a large relative delta.

Architecture: {"name":"Custom DIGITALOCEAN 3-Tier Web App","description":"DigitalOcean Load Balancer distributing traffic across 2× s-4vcpu-8gb Droplets backed by a single db-s-2vcpu-4gb Managed Database (nyc1).","provider":"digitalocean","region":"nyc1","referenceRegion":"nyc1","components":[{"role":"Load Balancer","resourceType":"network","instanceType":"Standard Load Balancer","count":1,"officialHourlyCost":0.0164,"notes":"DigitalOcean Standard Load Balancer flat charge (NYC1). $12/mo."},{"role":"Web / App Server","resourceType":"compute","instanceType":"s-4vcpu-8gb","count":2,"officialHourlyCost":0.06575342465753424,"notes":"Basic Droplet: 4 vCPU, 8 GiB RAM. Mid-tier general-purpose workload, comparable to AWS m5.large (NYC1)."},{"role":"Database","resourceType":"database","instanceType":"db-s-2vcpu-4gb (Managed Database)","count":1,"officialHourlyCost":0.08342465753424658,"notes":"Managed PostgreSQL/MySQL standard node: 2 vCPU, 4 GiB RAM. ~200 max connections (NYC1)."}]}

Idle — 10 RPS

Minimal background traffic — keep-alive checks, health probes, and occasional real requests.

Scenario score: 95.3%. Comparison status: complete.

Idle reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	24 ms	25 ms	4.2%	95.8%
P95 Latency	54 ms	54 ms	0%	100%
CPU Utilization	6 %	6.3 %	5%	95%
Throughput	10 req/s	9.98 req/s	-0.2%	99.8%
Error Rate	0.3 %	0.22 %	-27.7%	72.3%
Cost / Hour	0.2302 $/hr	0.2314 $/hr	0.5%	99.5%

latencyP50Ms / latencyP95Ms: DigitalOcean s-4vcpu-8gb Droplet + DO Standard Load Balancer: idle P50 ~22–26 ms, P95 ~50–58 ms. DO LB adds ~5–7 ms more overhead than AWS ALB; DO nyc1 network path contributes additional latency vs AWS us-east-1. Source: DigitalOcean Load Balancer and Droplet performance documentation. Source
cpuPct / errorRatePct: DO s-4vcpu-8gb at idle: ~5–7% CPU (OS + runtime baseline on 4 shared vCPUs). db-s-2vcpu-4gb (200 max connections) at 10 RPS: ~2 connections = 1% pool utilisation; DO managed DB shared infrastructure causes ~0.2–0.4% background error rate. Source: DigitalOcean Droplet and Managed Databases documentation. Source
costPerHour: DO Pricing (nyc1, August 2026): Standard LB $0.0164/hr + 2× s-4vcpu-8gb $0.0658/hr + db-s-2vcpu-4gb $0.08219/hr = $0.2302/hr. Source

Normal — 100 RPS

Typical business-hours traffic with steady sustained load.

Scenario score: 97.6%. Comparison status: complete.

Normal reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	32 ms	32 ms	0%	100%
P95 Latency	70 ms	68 ms	-2.9%	97.1%
CPU Utilization	15 %	15.4 %	2.7%	97.3%
Throughput	100 req/s	99.56 req/s	-0.4%	99.6%
Error Rate	0.4 %	0.44 %	10.5%	89.5%
Cost / Hour	0.2302 $/hr	0.2314 $/hr	0.5%	99.5%

latencyP50Ms / latencyP95Ms: DO s-4vcpu-8gb at ~50 RPS/instance: real-world P50 ~29–35 ms, P95 ~64–76 ms. The AWS-calibrated simulator under-predicts because the queuing-theory latency formula was tuned to AWS m5.large network characteristics and does not model DO's network overhead. Source: DigitalOcean Droplet performance benchmarks. Source
cpuPct: DO s-4vcpu-8gb (4 shared vCPUs) at ~50 RPS/instance: real-world CPU ~12–18%. The AWS-calibrated sqrt-of-utilisation formula over-predicts CPU because it was tuned to AWS m5.large (maxThroughput=5000 RPS); at 50/3600 = 1.4% throughput utilisation, real 4-vCPU Droplets run ~12–18%. Source: DigitalOcean Droplet performance guidance. Source
errorRatePct: DO db-s-2vcpu-4gb (200 max connections) at 100 RPS: ~20 concurrent connections = 10% pool utilisation. At 10% utilisation with 2-vCPU managed DB, error rate is low; real-world ~0.3–0.5%. Source: DigitalOcean Managed Databases documentation. Source

Peak — 500 RPS

Peak business load — product launch, end-of-day batch, or marketing campaign spike.

Scenario score: 97.8%. Comparison status: complete.

Peak reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	68 ms	68 ms	0%	100%
P95 Latency	142 ms	136 ms	-4.2%	95.8%
CPU Utilization	44 %	41.9 %	-4.8%	95.2%
Throughput	488 req/s	492.5 req/s	0.9%	99.1%
Error Rate	1.5 %	1.5 %	-0.1%	100%
Cost / Hour	0.2302 $/hr	0.2314 $/hr	0.5%	99.5%

cpuPct: DO s-4vcpu-8gb at ~250 RPS/instance (6.9% of 3600-RPS capacity): real-world CPU ~40–48% on 4-vCPU Droplets at this absolute RPS level. The AWS-calibrated sqrt-of-utilisation formula still over-predicts somewhat due to capacity-basis mismatch but is more accurate than s-2vcpu-4gb was. Source: DigitalOcean Droplet performance guidance. Source
latencyP50Ms / latencyP95Ms: DO s-4vcpu-8gb at 250 RPS/instance with moderate DB connection pressure: real-world P50 ~62–74 ms, P95 ~132–152 ms. More headroom than s-2vcpu-4gb at the same load, with less queuing pressure. Source: DigitalOcean Managed Databases operational guidance. Source
errorRatePct / throughputRps: DO db-s-2vcpu-4gb (200 max connections) at 500 RPS: ~100 concurrent connections = 50% pool utilisation. At 50% utilisation error rate is modest; real-world ~1.2–1.8%. Effective throughput ~488 RPS. Source: DigitalOcean Managed Databases operational guidance. Source

Burst — 1000 RPS

Traffic burst exceeding normal peak — flash sale, viral event, or coordinated load test.

Scenario score: 99%. Comparison status: complete.

Burst reference and simulated metric comparisons
Metric	Reference	Simulated	Delta	Accuracy
P50 Latency	118 ms	118 ms	0%	100%
P95 Latency	228 ms	230 ms	0.9%	99.1%
CPU Utilization	68 %	70.3 %	3.4%	96.6%
Throughput	952 req/s	955 req/s	0.3%	99.7%
Error Rate	4.5 %	4.5 %	0%	100%
Cost / Hour	0.2302 $/hr	0.2314 $/hr	0.5%	99.5%

cpuPct: DO s-4vcpu-8gb at 500 RPS/instance (13.9% of 3600-RPS capacity): real-world CPU ~64–72% on 4-vCPU Droplets. The AWS-calibrated formula over-predicts toward ~85% at this load due to capacity-basis mismatch, but less severely than with the smaller s-2vcpu-4gb. Source: DigitalOcean Droplet performance guidance. Source
latencyP50Ms / latencyP95Ms: DO s-4vcpu-8gb under burst with DB approaching saturation: real-world P50 ~110–126 ms, P95 ~212–244 ms. Lower than s-2vcpu-4gb at equivalent load due to more vCPU headroom and a 200-connection DB vs the old 75-connection one. Source: DigitalOcean performance guidance. Source
errorRatePct / throughputRps: DO db-s-2vcpu-4gb at 1000 RPS: ~200 concurrent connections = 100% pool utilisation, causing connection wait timeouts; real-world error rate ~4–5%. Effective throughput ~952 RPS (1000 × 0.952). Source: DigitalOcean Managed Databases documentation. Source

Fidelity components: cost estimated, latency estimated, CPU estimated, throughput estimated, error rate estimated.

Methodology & Reference Sources

How reference values are derived and what each data source covers.

Application weight, prediction evidence & uncertainty

Workspace, public/MCP self-serve simulations and the GitHub App default to typical. Weight is fixed at creation. The four-rung canonical Accuracy overall score uses lean; the separate owned typical 300 RPS holdout uses frozen pre-fit predictions and is not blended with it. Non-AWS comparisons intentionally retain a benchmark-only compatibility path: their existing outputs, references, citations and scores are not validation of the new self-serve predictions. Scoring and the 0.01 percentage-point error tolerance are unchanged.

Let T be offered application RPS, r be routed offered RPS per serving app host (not goodput or database traffic), and v be verified catalog vCPU. Reference CPU is C = 0.22333 + k × 0.033548 × r percent. Lean k = 1; typical k = (20 − 0.22333)/(50 × 0.033548); heavy k = 2 × typical k. The intercept is not multiplied by k. Thus two m5.large servers at 100 total RPS (50 per server) target about 20% CPU each for typical, versus about 1.9% for lean. The generic typical 15–25% anchor is placeholder guidance, not an AWS measurement. The separately identified owned typical Node CRUD fit uses measured CPU and latency at 20, 100, and 200 RPS on the exact Ohio graph; the 300 RPS comparison above is its independent holdout. heavy's approximately 39.8% is an unsupported product assumption, not a measured hashing workload.

Scale the whole CPU expression, including its intercept, by 2/v and clamp to 0–100%. Only lean AWS M5 sizes with verified proportional RAM (GiB/8 = v/2) qualify for same-family measured scaling. Lean m5.xlarge at 50 RPS per server is about 0.95%. Other known-vCPU families/providers use this as a broad reference estimate, not a validated conversion. More vCPU does not prove more single-process request capacity; the observed roughly 51% host CPU limit is not a universal cap.

Outside the preserved exact lean fit, internal latency baselines in milliseconds are L50 = max(1.45, 2.4833 − 0.0010466T) and L95 = max(L50, 3.10, 4.3697 − 0.0012973T), with full-precision fitted coefficients used before rounding. Typical multiplies these by 18.5/2.40 and 45/4.25, respectively; heavy uses twice those multipliers. For lean scaled predictions, the formula floors are 1.45 ms P50 and 3.10 ms P95, with no generic 5 ms floor; 0.05 ms is only a physical/non-finite safety floor and P95 cannot be below P50. The exact eligible lean path retains its 0.05-ms upward bucketing. Topology, congestion, memory, routing and active-failure penalties still apply. These are in-VPC/internal-LB baselines, not measured internet end-to-end latency for other networks. The canonical lean P99 prediction uses an owned three-rung fit; other reference configurations retain uncalibrated generic P99.

Measured describes the prediction basis only: lean, the fully eligible canonical graph, 10–1,000 RPS, no active outage. Scaled from measured covers eligible analogous lean M5 sizing and beyond-load extrapolation of the exact graph. Typical/heavy, other providers/families, incomplete or disqualified graphs are reference estimates. Active failures retain their penalties and cannot carry an unqualified healthy measured label.

Measured CPU/P50/P95 and exact lean owned P99 assumption intervals: 0.65–1.35 × center; these are not confidence intervals.
P99 is a diagnostic only: ordinary least squares on the owned 10/100/500 RPS fit readings gives max(final P95, 7.021919127633514 + 0.00027013891327780484 × offered RPS) ms. The 1,000 RPS measurement is a holdout, not a fit input. No P99 weight enters the score.
Scaled: CPU 0.40–1.60 × center; P50/P95 0.50–1.50 × center.
Reference: 0.50–2.00 × center.
Beyond 1,000 RPS: scaled 0.25–2.00 × center; reference 0.25–3.00 × center. Never measured beyond this range.

Physical floors/caps apply. These are assumption intervals, not statistical confidence intervals. The measured ±35% range covers cited holdout deviations in magnitude; later-day, region, pool-bound and app-bound holdouts motivate uncertainty, not universal guarantees. Source IDs separate the owned campaign 473f1339-f712-4096-96d6-3d4fc07cb427 (results 117051b; measurement SHA 7416cb63), SKU specifications, cwm-industry-guidance-placeholder-v1 and cwm-heavy-assumption-v1.

Residual generic estimates remain for unrecognized/custom compute without verified capacity and opaque Kubernetes pools or serverless tasks without serving-vCPU/routing denominators. GPU token inference/training latency is a separate model, not a CRUD-vCPU validation claim. Unknown-vCPU app estimates use cwm-legacy-generic-unverified, the widest reference range, and have no external reference; missing vCPU is never assumed to be two. New predictionEvidence sets legacyGeneric on each affected app CPU and at the top level if any reported CPU or latency used the generic branch; older stored evidence is not reclassified. DB-only/network-only graphs have no applicable app CPU. App-host CPU evidence is separate from the legacy aggregate CPU metric. Historical rows without saved evidence are explicitly unavailable/legacy and are not retroactively labeled measured.

Prediction-only capacity catalog additions cover GCP E2 standard (e2-standard-2/4/8/16/32: 2/8, 4/16, 8/32, 16/64, 32/128 vCPU/GB), N1 standard (n1-standard-1/2/4/8/16/32/64/96: 1/3.75, 2/7.5, 4/15, 8/30, 16/60, 32/120, 64/240, 96/360 vCPU/GB), and N2 standard (n2-standard-16/32/48/64/80/96/128: 16/64, 32/128, 48/192, 64/256, 80/320, 96/384, 128/512 vCPU/GB), using published capacity specifications. These hardware dimensions do not establish GCP-specific benchmark performance, throughput, latency, or price; existing Accuracy benchmark comparisons and scores remain unchanged. OCI x86 E4 Flex capacity requires resource.characteristics.ocpus as a positive integer from 1–64. Each OCPU maps to 2 vCPUs; memory is independently selected and cannot be inferred as a fixed amount per OCPU. The same x86 mapping applies to E3 (1–64), E5 (1–126), and Standard3 (1–32) Flex; ARM A1 uses 1 vCPU per OCPU (1–76). See GCP general-purpose machine specifications and OCI Compute Shapes specifications. The read-only generic-shape inventory lists catalog gaps and conditional fallback categories with their reasons; unknown/custom compute is an open-ended wildcard, not an enumerable SKU list. Each prediction reports its actual fallback reason in predictionEvidence.

Reproducibility & Seed

The simulation engine is deterministic when a fixed seed is provided. Scored custom and provider results use seed 20240601. The canonical AWS response is reproducible with the calibrated engine and fixed seed. Its score uses owned app-server CPU and in-VPC internal-LB latency, owned goodput/errors, and price-list cost. Engine predictions use seed 20240601.

Generated at: Sun, 25 Oct 2026 17:32:58 GMT

Reference Data Sources

Canonical AWS CPU and latency references come from the pinned company-owned cwm-bench campaign: app-server CPU (not database CPU or the derived blend) and k6 in-VPC latency through an internal load balancer, excluding internet travel. Billed cost is from the AWS us-east-2 price list, not a measurement. Other providers retain their own reference sources.

Owned campaign provenance: campaign 473f1339-f712-4096-96d6-3d4fc07cb427; 10–1,000 RPS run code 7416cb63ace3a7ab2e3486bb6f132a2dcb574c34; later-day, us-west-2 and cpu-only run code 1fdfe97572ef91a270c9d39142fb28f7a10061eb; recorded results 117051b05b253b16b4b819d326bf1b15181af124; export measurement_sha 7416cb63ace3a7ab2e3486bb6f132a2dcb574c34; primary region us-east-2. Fit: idle, normal, peak. Holdout: burst, later-day, second-region.

Sources: cwm-bench campaign repository · 10–1,000 RPS run code · Later-day, us-west-2 and cpu-only run code · Pinned results export · AWS public price list ($0.4545/hour, not a measured bill).

Scored: mean application-server CPU across two app hosts; database CPU is measured separately. (2 × app + DB) / 3 is a derived diagnostic, not app CPU. Scored: k6 in-VPC through an internal load balancer, excluding internet travel; engine latency is a modeled in-VPC LB path, not an end-to-end internet target. Prediction: SimulationEngine.simulateStep, seed 20240601; modeled in-VPC internal-LB P50/P95, mean of two compute-resource CPU predictions, whole-run goodput and fixed-fee price-list hourly cost (excluding modeled egress and LB variable processing). Holdout policy: Fit idle, normal, and peak; hold out burst, later-day, and second-region runs. Canonical AWS scores owned app CPU, internal-LB in-VPC latency, goodput and error; $0.4545/hour is from the us-east-2 public price list, not a measured bill. Later-day and second-region have no separate engine prediction and do not enter the score.

Idle (10 req/s)

latencyP50Ms / latencyP95Ms: Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), idle fit run. Source
cpuPct: Owned app-server CPU mean of two application hosts; database CPU measured separately, idle fit run. Source
throughputRps / errorRatePct: Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 8.874155959479843 RPS goodput and 0% errors at a 10 RPS target. Source
costPerHour: AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr. Source

Normal (100 req/s)

latencyP50Ms / latencyP95Ms: Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), normal fit run. Source
cpuPct: Owned app-server CPU mean of two application hosts; database CPU measured separately, normal fit run. Source
throughputRps / errorRatePct: Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 87.62477401512365 RPS goodput and 0% errors at a 100 RPS target. Source
costPerHour: AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr. Source

Peak (500 req/s)

latencyP50Ms / latencyP95Ms: Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), peak fit run. Source
cpuPct: Owned approximate app-server CPU; database CPU measured separately, peak fit run. Source
throughputRps / errorRatePct: Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 437.62281962650434 RPS goodput and 0% errors at a 500 RPS target. Source
costPerHour: AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr. Source

Burst (1000 req/s)

latencyP50Ms / latencyP95Ms: Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), burst holdout run. Source
cpuPct: Owned approximate app-server CPU; database CPU measured separately, burst holdout run. Source
throughputRps / errorRatePct: Owned Burst holdout measured 875.1162528375527 RPS whole-run goodput and one failure in 1,050,142 (0.0000952%). Five-minute ramp plus fifteen-minute steady window; no dropped requests implied. Source
costPerHour: AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr. Source

Composite Score Methodology

For metrics other than error rate, accuracy is max(0, 100 − |deltaPct|). Error rate uses the absolute difference between simulated and reference rates: at most 0.01 percentage points scores 100%. Outside that tolerance, its score is the higher of 100 × 0.01 ÷ difference (in percentage points) and the usual relative-accuracy score. The composite score is a weighted average:

P95 Latency — 25% (most user-visible SLO)
P50 Latency — 20%
CPU Utilization — 20%
Throughput — 15%
Error Rate — 10%
Cost / Hour — 10%

For provider-documentation, custom, and canonical AWS responses, the overall accuracy score is the unweighted average of the four scenario composite scores. The canonical AWS comparison scores measured app CPU, in-VPC internal-LB latency, goodput/errors and price-list cost. The two extra holdouts have no separate predictions and are not scored.

Scope & Limitations

This benchmark covers a single canonical architecture. Real-world accuracy varies by workload type, instance size, database query complexity, and network topology.

Live CloudWatch data is not fetched — no real AWS credentials are used. The owned campaign is a pinned measurement artifact, not a live feed, and its Burst rung remains a holdout rather than a capacity claim.

Cost accuracy is validated continuously via automated drift checks in CI. See the Simulation Fidelity page for per-provider cost benchmark details across all five cloud providers. Performance reference values are reviewed when major AWS instance generations change.

Simulation outputs are estimates for the stated workloads and evidence scopes, not universal accuracy guarantees, bills, or SLOs.

Accuracy dataset

Cloud World Model accuracy benchmark scores, campaign scopes, frozen engine versions, measurement dates, scoring methodology, and cwm-bench source references.

Current engine version: 1.2.43. Last updated 2026-10-13.

Source: canvascloudai/cwm-bench. Machine-readable output: /accuracy.json.

Owned AWS benchmark

The canonical topology is ALB → 2 × m5.large → db.r5.large MySQL Single-AZ in us-east-2. Measurements come from the company-owned cwm-bench campaign 473f1339-f712-4096-96d6-3d4fc07cb427.

Fit split and holdout

Idle (10 RPS), normal (100 RPS), and peak (500 RPS) are fit rungs. Burst (1,000 RPS), later-day runs, and the us-west-2 second-region run are holdouts.

Burst holdout

The owned canonical CRUD burst measured approximately 875 RPS goodput, approximately 0% errors, approximately 13% app CPU, and 55 maximum DB connections out of roughly 500; RDS BurstBalance was not exhausted.

Engine comparison status

For the eligible owned AWS workload, calibrated engine output is compared with company-owned measurements of app CPU, in-VPC load-balancer latency, goodput, and CRUD errors. Hourly cost uses the public AWS price list, not a measured bill. This is evidence for this architecture and workload, not a universal accuracy guarantee. Other providers use documentation-based reference comparisons, not owned load tests or a cross-provider ranking.

Reproducibility

The campaign repository is cwm-bench at measurement SHA 7416cb63ace3a7ab2e3486bb6f132a2dcb574c34. The public API exposes the campaign provenance and the fixed simulation seed.