{"name":"Cloud World Model Accuracy Benchmark","description":"Cloud World Model accuracy benchmark scores, campaign scopes, frozen engine versions, measurement dates, scoring methodology, and cwm-bench source references.","url":"https://www.cloudworldmodel.ai/accuracy","jsonUrl":"https://www.cloudworldmodel.ai/accuracy.json","dateModified":"2026-10-13","engineVersion":"1.2.42","source":"https://github.com/canvascloudai/cwm-bench","scoring":{"errorRateScoring":{"tolerancePercentagePoints":0.01,"outsideTolerance":"max_inverse_tolerance_or_relative_accuracy"},"canonicalLeanOverallScore":93.8,"canonicalLeanScoringBasis":{"type":"measured_cwm_bench","costReference":"public_price_list"},"canonicalLeanReferenceProvenance":{"source":"owned_campaign","campaignId":"473f1339-f712-4096-96d6-3d4fc07cb427","campaignGeneratedAt":"2026-09-06T03:44:51.130Z","sourceRevision":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","supplementalSourceRevision":"1fdfe97572ef91a270c9d39142fb28f7a10061eb","measurementSha":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","resultsRevision":"117051b05b253b16b4b819d326bf1b15181af124","primaryRegion":"us-east-2","holdoutRegions":["us-west-2"],"fitScenarios":["idle","normal","peak"],"holdoutScenarios":["burst","later-day","second-region"],"repositoryUrl":"https://github.com/canvascloudai/cwm-bench","resultsUrl":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md","runCodeUrl":"https://github.com/canvascloudai/cwm-bench/tree/7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","supplementalRunCodeUrl":"https://github.com/canvascloudai/cwm-bench/tree/1fdfe97572ef91a270c9d39142fb28f7a10061eb","pricingUrl":"https://aws.amazon.com/ec2/pricing/on-demand/","predictionBasis":"SimulationEngine.simulateStep, seed 20240601; modeled in-VPC internal-LB P50/P95, mean of two compute-resource CPU predictions, whole-run goodput and fixed-fee price-list hourly cost (excluding modeled egress and LB variable processing).","cpuBasis":"Scored: mean application-server CPU across two app hosts; database CPU is measured separately. (2 × app + DB) / 3 is a derived diagnostic, not app CPU.","latencyBasis":"Scored: k6 in-VPC through an internal load balancer, excluding internet travel; engine latency is a modeled in-VPC LB path, not an end-to-end internet target.","costBasis":"Fixed-fee public price-list comparison, not a measured bill: scored prediction uses itemized resource costs before display rounding and excludes modeled egress and load-balancer variable data processing/ALB LCUs. The load-balancer base hourly fee remains included; other resource charges are not removed.","holdoutPolicy":"Fit idle, normal, and peak; hold out burst, later-day, and second-region runs.","note":"Canonical AWS scores owned app CPU, internal-LB in-VPC latency, goodput and error; $0.4545/hour is from the us-east-2 public price list, not a measured bill. Later-day and second-region have no separate engine prediction and do not enter the score."},"canonicalLeanScenarios":[{"scenario":{"id":"idle","label":"Idle","description":"Minimal background traffic — keep-alive checks, health probes, and occasional real requests.","trafficRps":10,"referenceMetrics":{"latencyP50Ms":2.62841,"latencyP95Ms":4.5295678,"cpuPct":0.4785260097091355,"throughputRps":8.874155959479843,"errorRatePct":0,"costPerHour":0.4545},"ownedMeasurement":{"source":"cwm-bench","campaignId":"473f1339-f712-4096-96d6-3d4fc07cb427","campaignGeneratedAt":"2026-09-06T03:44:51.130Z","runId":"473f1339-f712-4096-96d6-3d4fc07cb427:idle","sourceRevision":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","measurementSha":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","capturedAt":"2026-09-05T16:32:30.342Z","region":"us-east-2","split":"fit","targetRps":10,"metrics":{"goodputRps":8.874155959479843,"p50Ms":2.62841,"p95Ms":4.5295678,"p99Ms":7.23815152,"errorRatePct":0,"appCpuPct":0.4785260097091355,"dbCpuPct":3.337818548370722,"dbConnectionsMax":2},"blendedCpuPct":1.431623522596331,"notes":"Fit rung; goodput is the campaign's five-minute warmup plus fifteen-minute steady-window average."},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), idle fit run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"cpuPct","citation":"Owned app-server CPU mean of two application hosts; database CPU measured separately, idle fit run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"throughputRps / errorRatePct","citation":"Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 8.874155959479843 RPS goodput and 0% errors at a 10 RPS target.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"costPerHour","citation":"AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr.","url":"https://aws.amazon.com/ec2/pricing/on-demand/"}]},"simulated":{"latencyP50Ms":2.5,"latencyP95Ms":4.4,"cpuPct":0.391,"throughputRps":8.874571,"errorRatePct":0,"costPerHour":0.4545},"simulatedP99Ms":7.024620516766292,"metrics":[{"metric":"latencyP50Ms","label":"P50 in-VPC LB latency","unit":"ms","reference":2.63,"simulated":2.5,"deltaPct":-4.9,"accuracyPct":95.1},{"metric":"latencyP95Ms","label":"P95 in-VPC LB latency","unit":"ms","reference":4.53,"simulated":4.4,"deltaPct":-2.9,"accuracyPct":97.1},{"metric":"cpuPct","label":"App-server CPU","unit":"%","reference":0.48,"simulated":0.39,"deltaPct":-18.3,"accuracyPct":81.7},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":8.87,"simulated":8.87,"deltaPct":0,"accuracyPct":100},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0,"simulated":0,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.4545,"simulated":0.4545,"deltaPct":0,"accuracyPct":100}],"compositeScore":94.6,"comparisonStatus":"complete"},{"scenario":{"id":"normal","label":"Normal","description":"Typical business-hours traffic with steady sustained load.","trafficRps":100,"referenceMetrics":{"latencyP50Ms":2.1881595,"latencyP95Ms":4.0283214,"cpuPct":1.7935897435897439,"throughputRps":87.62477401512365,"errorRatePct":0,"costPerHour":0.4545},"ownedMeasurement":{"source":"cwm-bench","campaignId":"473f1339-f712-4096-96d6-3d4fc07cb427","campaignGeneratedAt":"2026-09-06T03:44:51.130Z","runId":"473f1339-f712-4096-96d6-3d4fc07cb427:normal","sourceRevision":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","measurementSha":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","capturedAt":"2026-09-05T16:53:31Z","region":"us-east-2","split":"fit","targetRps":100,"metrics":{"goodputRps":87.62477401512365,"p50Ms":2.1881595,"p95Ms":4.0283214,"p99Ms":6.78735754,"errorRatePct":0,"appCpuPct":1.7935897435897439,"dbCpuPct":4.433338557944269,"dbConnectionsMax":8},"blendedCpuPct":2.673506015041252,"notes":"Fit rung; goodput is the campaign's five-minute warmup plus fifteen-minute steady-window average."},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), normal fit run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"cpuPct","citation":"Owned app-server CPU mean of two application hosts; database CPU measured separately, normal fit run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"throughputRps / errorRatePct","citation":"Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 87.62477401512365 RPS goodput and 0% errors at a 100 RPS target.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"costPerHour","citation":"AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr.","url":"https://aws.amazon.com/ec2/pricing/on-demand/"}]},"simulated":{"latencyP50Ms":2.4,"latencyP95Ms":4.25,"cpuPct":1.901,"throughputRps":87.624266,"errorRatePct":0,"costPerHour":0.4544999999999999},"simulatedP99Ms":7.048933018961295,"metrics":[{"metric":"latencyP50Ms","label":"P50 in-VPC LB latency","unit":"ms","reference":2.19,"simulated":2.4,"deltaPct":9.7,"accuracyPct":90.3},{"metric":"latencyP95Ms","label":"P95 in-VPC LB latency","unit":"ms","reference":4.03,"simulated":4.25,"deltaPct":5.5,"accuracyPct":94.5},{"metric":"cpuPct","label":"App-server CPU","unit":"%","reference":1.79,"simulated":1.9,"deltaPct":6,"accuracyPct":94},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":87.62,"simulated":87.62,"deltaPct":0,"accuracyPct":100},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0,"simulated":0,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.4545,"simulated":0.4545,"deltaPct":0,"accuracyPct":100}],"compositeScore":95.5,"comparisonStatus":"complete"},{"scenario":{"id":"peak","label":"Peak","description":"Peak business load — product launch, end-of-day batch, or marketing campaign spike.","trafficRps":500,"referenceMetrics":{"latencyP50Ms":1.99503,"latencyP95Ms":3.75995775,"cpuPct":8.63,"throughputRps":437.62281962650434,"errorRatePct":0,"costPerHour":0.4545},"ownedMeasurement":{"source":"cwm-bench","campaignId":"473f1339-f712-4096-96d6-3d4fc07cb427","campaignGeneratedAt":"2026-09-06T03:44:51.130Z","runId":"473f1339-f712-4096-96d6-3d4fc07cb427:peak","sourceRevision":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","measurementSha":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","capturedAt":"2026-09-05T17:14:51Z","region":"us-east-2","split":"fit","targetRps":500,"metrics":{"goodputRps":437.62281962650434,"p50Ms":1.99503,"p95Ms":3.75995775,"p99Ms":7.20503306,"errorRatePct":0,"appCpuPct":8.63,"dbCpuPct":8.22,"dbConnectionsMax":43},"blendedCpuPct":8.493333333333334,"notes":"Fit rung; app CPU and DB CPU were reported as approximate campaign values."},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), peak fit run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"cpuPct","citation":"Owned approximate app-server CPU; database CPU measured separately, peak fit run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"throughputRps / errorRatePct","citation":"Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 437.62281962650434 RPS goodput and 0% errors at a 500 RPS target.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"costPerHour","citation":"AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr.","url":"https://aws.amazon.com/ec2/pricing/on-demand/"}]},"simulated":{"latencyP50Ms":2,"latencyP95Ms":3.75,"cpuPct":8.61,"throughputRps":437.622913,"errorRatePct":0,"costPerHour":0.4545},"simulatedP99Ms":7.156988584272416,"metrics":[{"metric":"latencyP50Ms","label":"P50 in-VPC LB latency","unit":"ms","reference":2,"simulated":2,"deltaPct":0.2,"accuracyPct":99.8},{"metric":"latencyP95Ms","label":"P95 in-VPC LB latency","unit":"ms","reference":3.76,"simulated":3.75,"deltaPct":-0.3,"accuracyPct":99.7},{"metric":"cpuPct","label":"App-server CPU","unit":"%","reference":8.63,"simulated":8.61,"deltaPct":-0.2,"accuracyPct":99.8},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":437.62,"simulated":437.62,"deltaPct":0,"accuracyPct":100},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0,"simulated":0,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.4545,"simulated":0.4545,"deltaPct":0,"accuracyPct":100}],"compositeScore":99.8,"comparisonStatus":"complete"},{"scenario":{"id":"burst","label":"Burst","description":"Traffic burst exceeding normal peak — flash sale, viral event, or coordinated load test.","trafficRps":1000,"referenceMetrics":{"latencyP50Ms":1.9312195,"latencyP95Ms":3.71435975,"cpuPct":13.36,"throughputRps":875.1162528375527,"errorRatePct":0.00009522521716110772,"costPerHour":0.4545},"ownedMeasurement":{"source":"cwm-bench","campaignId":"473f1339-f712-4096-96d6-3d4fc07cb427","campaignGeneratedAt":"2026-09-06T03:44:51.130Z","runId":"473f1339-f712-4096-96d6-3d4fc07cb427:burst","sourceRevision":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","measurementSha":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","capturedAt":"2026-09-05T17:35:55Z","region":"us-east-2","split":"holdout","targetRps":1000,"metrics":{"goodputRps":875.1162528375527,"p50Ms":1.9312195,"p95Ms":3.71435975,"p99Ms":7.01741024,"errorRatePct":0.00009522521716110772,"appCpuPct":13.36,"dbCpuPct":10.91,"dbConnectionsMax":55},"blendedCpuPct":12.543333333333331,"notes":"Holdout rung; one failure in 1,050,142 requests (raw fraction 9.522521716110773e-07, or 0.0000952%). Goodput includes a five-minute ramp and fifteen-minute steady window; no dropped requests are implied."},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), burst holdout run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"cpuPct","citation":"Owned approximate app-server CPU; database CPU measured separately, burst holdout run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"throughputRps / errorRatePct","citation":"Owned Burst holdout measured 875.1162528375527 RPS whole-run goodput and one failure in 1,050,142 (0.0000952%). Five-minute ramp plus fifteen-minute steady window; no dropped requests implied.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"costPerHour","citation":"AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr.","url":"https://aws.amazon.com/ec2/pricing/on-demand/"}]},"simulated":{"latencyP50Ms":1.45,"latencyP95Ms":3.1,"cpuPct":16.997,"throughputRps":875.121221,"errorRatePct":0,"costPerHour":0.45449999999999985},"simulatedP99Ms":7.292058040911319,"metrics":[{"metric":"latencyP50Ms","label":"P50 in-VPC LB latency","unit":"ms","reference":1.93,"simulated":1.45,"deltaPct":-24.9,"accuracyPct":75.1},{"metric":"latencyP95Ms","label":"P95 in-VPC LB latency","unit":"ms","reference":3.71,"simulated":3.1,"deltaPct":-16.5,"accuracyPct":83.5},{"metric":"cpuPct","label":"App-server CPU","unit":"%","reference":13.36,"simulated":17,"deltaPct":27.2,"accuracyPct":72.8},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":875.12,"simulated":875.12,"deltaPct":0,"accuracyPct":100},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0.000095225,"simulated":0,"deltaPct":-100,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.4545,"simulated":0.4545,"deltaPct":0,"accuracyPct":100}],"compositeScore":85.5,"comparisonStatus":"complete"}],"canonicalProviderBases":{"aws":{"type":"measured_cwm_bench","costReference":"public_price_list","label":"Measured (CWM Bench)"},"gcp":{"type":"documentation_reference","costReference":"documentation_reference","label":"Documentation reference (not yet measured)"},"azure":{"type":"documentation_reference","costReference":"documentation_reference","label":"Documentation reference (not yet measured)"},"oci":{"type":"documentation_reference","costReference":"documentation_reference","label":"Documentation reference (not yet measured)"},"digitalocean":{"type":"documentation_reference","costReference":"documentation_reference","label":"Documentation reference (not yet measured)"}}},"campaigns":{"canonicalLean":{"campaignId":"473f1339-f712-4096-96d6-3d4fc07cb427","measurementDate":"2026-09-06T03:44:51.130Z","sourceRevision":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","resultsUrl":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md","repositoryUrl":"https://github.com/canvascloudai/cwm-bench","runCodeUrl":"https://github.com/canvascloudai/cwm-bench/tree/7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","engineVersion":"1.2.42","scope":{"name":"AWS 3-Tier Web App","description":"Standard three-tier web application: Application Load Balancer distributing traffic across two m5.large EC2 instances backed by a single db.r5.large RDS MySQL instance (Single-AZ, us-east-2).","provider":"aws","region":"us-east-2","referenceRegion":"us-east-2","components":[{"role":"Load Balancer","resourceType":"network","instanceType":"ALB (Application Load Balancer)","count":1,"officialHourlyCost":0.0225,"notes":"AWS Application Load Balancer base charge (us-east-2). Scales automatically."},{"role":"Web / App Server","resourceType":"compute","instanceType":"m5.large","count":2,"officialHourlyCost":0.096,"notes":"General-purpose instance: 2 vCPU, 8 GiB RAM, up to 10 Gbps network bandwidth."},{"role":"Database","resourceType":"database","instanceType":"db.r5.large (MySQL 8.0, Single-AZ)","count":1,"officialHourlyCost":0.24,"notes":"Memory-optimized: 2 vCPU, 16 GiB RAM. gp2 storage with 3,000 IOPS burst. ~500 max connections."}]},"score":93.8},"typicalV1":{"campaignId":"typical-v1-20260927c","measurementDate":"2026-09-27","predictionsFrozenOnEngine":"1.2.5","predictionsReproducedOnEngine":"1.2.13","scope":{"campaigns":1,"days":1,"region":"us-east-2","attemptsPerRung":1,"engineVersion":"1.2.3","seed":20240601,"predictionStep":6,"predictionsFrozenBeforeMeasurement":true,"measuredCost":false,"topology":"Internal ALB + 2 × m5.large + db.r5.large MySQL 8.0; two workers and pool 250 per server"},"reportUrl":"https://github.com/canvascloudai/cwm-bench/blob/main/typical/REPORT.md","summaryUrl":"https://github.com/canvascloudai/cwm-bench/blob/main/holdout/exports/typical-v1-20260927c.summary.md","scoresUrl":"https://github.com/canvascloudai/cwm-bench/blob/main/typical/scores.json","predictionsUrl":"https://github.com/canvascloudai/cwm-bench/blob/main/typical/PREDICTIONS.md","beforeFit":{"targetRps":300,"split":"holdout","score":24.3,"scoreWithoutCost":27,"metrics":[{"metric":"p50","label":"P50","unit":"ms","predicted":17,"measured":4.621661,"measuredSource":"owned_measurement","score":0,"weight":0.2,"weightedContribution":0},{"metric":"p95","label":"P95","unit":"ms","predicted":42,"measured":26.9176766,"measuredSource":"owned_measurement","score":44,"weight":0.25,"weightedContribution":11},{"metric":"appCpu","label":"App CPU","unit":"%","predicted":59.553,"measured":15.303205128205128,"measuredSource":"owned_measurement","score":0,"weight":0.2,"weightedContribution":0},{"metric":"throughput","label":"Throughput","unit":"RPS","predicted":299,"measured":262.60872607647724,"measuredSource":"owned_measurement","score":86.1,"weight":0.15,"weightedContribution":12.9},{"metric":"errorRate","label":"Error rate","unit":"%","predicted":0.284,"measured":0.0044423431456231816,"measuredSource":"owned_measurement","score":3.6,"weight":0.1,"weightedContribution":0.4},{"metric":"cost","label":"Cost","unit":"USD/hour","predicted":0.94,"measured":0.4545,"measuredSource":"list_price_reference","score":0,"weight":0.1,"weightedContribution":0}],"unweightedP99":{"predicted":236,"measured":97.54310300000054,"unit":"ms","score":0},"throughputBasis":"Measured throughput is whole-run goodput averaged over the 5-minute warmup plus 15-minute steady window; engine throughput is a post-step point rate, not a duration-weighted average.","costBasis":"Cost was not measured. Score uses the list-price reference of USD 0.4545 per hour (ALB 0.0225 + 2 × 0.096 + 0.24, no egress, no generator). Reference, not a bill.","steadyWindowAlternative":{"scored":false,"throughputRps":300,"throughputScore":99.7,"totalScore":26.3}},"afterFit":{"targetRps":300,"split":"independent holdout","engineVersion":"1.2.20","calibrationId":"aws-crud/typical:typical-v1-20260927c:6aa574d7ff9d3080b88b221bcd59f7d218ae37f0","fitRungs":["typical-fit-20","typical-fit-100","typical-fit-200"],"score":84.2,"scoreWithoutCost":82.4,"metrics":[{"metric":"p50","label":"P50","unit":"ms","predicted":4.165220860655737,"measured":4.621661,"measuredSource":"owned_measurement","score":90.1,"weight":0.2,"weightedContribution":18},{"metric":"p95","label":"P95","unit":"ms","predicted":12.78397378032783,"measured":26.9176766,"measuredSource":"owned_measurement","score":47.5,"weight":0.25,"weightedContribution":11.9},{"metric":"appCpu","label":"App CPU","unit":"%","predicted":14.74,"measured":15.303205128205128,"measuredSource":"owned_measurement","score":96.3,"weight":0.2,"weightedContribution":19.3},{"metric":"throughput","label":"Throughput","unit":"RPS","predicted":262.604014,"measured":262.60872607647724,"measuredSource":"owned_measurement","score":100,"weight":0.15,"weightedContribution":15},{"metric":"errorRate","label":"Error rate","unit":"%","predicted":0,"measured":0.0044423431456231816,"measuredSource":"owned_measurement","score":100,"weight":0.1,"weightedContribution":10},{"metric":"cost","label":"Cost","unit":"USD/hour","predicted":0.454581,"measured":0.4545,"measuredSource":"list_price_reference","score":100,"weight":0.1,"weightedContribution":10}],"throughputBasis":"Measured throughput is whole-run goodput averaged over the 5-minute warmup plus 15-minute steady window; engine throughput is a post-step point rate, not a duration-weighted average. The post-fit model was fit to whole-run goodput at 20, 100, and 200 RPS.","costBasis":"Cost was not measured. USD 0.4545/hour is a fixed-fee public list-price reference, not a bill. The frozen post-fit scored prediction predates modeled LB variable processing. It excludes modeled egress; the historical before-fit prediction included it, so their cost scores are not on identical bases."},"dayAndRegionHoldouts":{"campaignId":"typical-holdouts-v1-20261006","reportUrl":"https://github.com/canvascloudai/cwm-bench/blob/main/typical/REPORT.md#day-and-region-holdouts-typical-holdouts-v1-20261006","comparisonBasis":"page cost basis","original":{"campaignId":"typical-v1-20260927c","targetRps":300,"score":84.2,"scoreWithoutCost":82.4,"p95Ms":26.9},"observed":[{"id":"later-day","label":"Later-day","region":"us-east-2","targetRps":300,"score":76,"scoreWithoutCost":73.3,"p95Ms":51.1},{"id":"second-region","label":"Second region","region":"us-west-2","targetRps":300,"score":81.1,"scoreWithoutCost":79,"p95Ms":32.9}],"repeatability100Rps":{"targetRps":100,"scores":[95.6,91.4]}},"extrapolatedDiagnostic":{"targetRps":500,"label":"diagnostic extrapolation beyond the calibrated and validated range (fit at 20, 100, and 200 RPS; validated at 300 RPS), compared with a measured 500 RPS diagnostic run; excluded from headline accuracy and not a validated saturation finding","engineVersion":"1.2.20","calibrationId":"aws-crud/typical:typical-v1-20260927c:6aa574d7ff9d3080b88b221bcd59f7d218ae37f0","fitRungs":["typical-fit-20","typical-fit-100","typical-fit-200"],"score":73.8,"scoreWithoutCost":70.9,"metrics":[{"metric":"p50","label":"P50","unit":"ms","predicted":4.165220860655737,"measured":5.187581,"measuredSource":"owned_measurement","score":80.3,"weight":0.2,"weightedContribution":16.1},{"metric":"p95","label":"P95","unit":"ms","predicted":16.333451173770428,"measured":88.189861,"measuredSource":"owned_measurement","score":18.5,"weight":0.25,"weightedContribution":4.6},{"metric":"appCpu","label":"App CPU","unit":"%","predicted":24.264,"measured":26.34166695034375,"measuredSource":"owned_measurement","score":92.1,"weight":0.2,"weightedContribution":18.4},{"metric":"throughput","label":"Throughput","unit":"RPS","predicted":437.588385,"measured":437.5769855710571,"measuredSource":"owned_measurement","score":100,"weight":0.15,"weightedContribution":15},{"metric":"errorRate","label":"Error rate","unit":"%","predicted":0,"measured":0.010282815511055931,"measuredSource":"owned_measurement","score":97.2,"weight":0.1,"weightedContribution":9.7},{"metric":"cost","label":"Cost","unit":"USD/hour","predicted":0.4545,"measured":0.4545,"measuredSource":"list_price_reference","score":100,"weight":0.1,"weightedContribution":10}],"throughputBasis":"Measured throughput is whole-run goodput averaged over the 5-minute warmup plus 15-minute steady window; engine throughput is a post-step point rate, not a duration-weighted average. The post-fit model was fit to whole-run goodput at 20, 100, and 200 RPS.","costBasis":"USD 0.4545/hour is a fixed-fee public list-price reference, not a measured bill. Fixed-fee public price-list comparison, not a measured bill: scored prediction uses itemized resource costs before display rounding and excludes modeled egress and load-balancer variable data processing/ALB LCUs. The load-balancer base hourly fee remains included; other resource charges are not removed."}},"typicalScaleV1":{"campaignId":"typical-scale-v1","measurementDate":"2026-10-06","sourceUrl":"https://github.com/canvascloudai/cwm-bench/pull/18","reportUrl":"https://github.com/canvascloudai/cwm-bench/blob/3b6708cee5b00f61254b0027c215ba0e1f389170/typical/REPORT.md#app-server-count-change-typical-scale-v1-20261006","predictionsFrozenOnEngine":"1.2.14","scope":"us-east-2, m5.large app servers (1–3), 100–300 total RPS, internal load balancer, and db.r5.large MySQL","results":[{"id":"1x-100","servers":1,"rps":100,"measuredP95":"9.1","predictedP95":"11.0","score":"89.1","low":"81.5","high":"93.6"},{"id":"1x-200","servers":1,"rps":200,"measuredP95":"37.5","predictedP95":"14.6","score":"82.1","low":"79.8","high":"83.8"},{"id":"1x-300","servers":1,"rps":300,"measuredP95":"95.0","predictedP95":"18.1","score":"74.5","low":"74.0","high":"75.0"},{"id":"2x-100","servers":2,"rps":100,"measuredP95":"8.4","predictedP95":"9.2","score":"95.7","low":"94.4","high":"97.1"},{"id":"2x-200","servers":2,"rps":200,"measuredP95":"15.1","predictedP95":"11.0","score":"89.8","low":"81.1","high":"95.8"},{"id":"2x-300","servers":2,"rps":300,"measuredP95":"41.3","predictedP95":"12.8","score":"77.7","low":"77.5","high":"80.3"},{"id":"3x-100","servers":3,"rps":100,"measuredP95":"8.7","predictedP95":"8.6","score":"98.4","low":"96.6","high":"98.4"},{"id":"3x-200","servers":3,"rps":200,"measuredP95":"12.1","predictedP95":"9.8","score":"94.7","low":"90.4","high":"94.9"},{"id":"3x-300","servers":3,"rps":300,"measuredP95":"20.7","predictedP95":"11.0","score":"86.8","low":"84.9","high":"89.2"}]},"typicalP95V1":{"campaignId":"typical-p95-v1-20261010","measurementDates":["2026-10-10","2026-10-12"],"sourceUrl":"https://github.com/canvascloudai/cwm-bench/pull/21","reportUrl":"https://github.com/canvascloudai/cwm-bench/blob/main/typical/REPORT.md#p95-two-input-model-holdout-typical-p95-v1-20261010","preregistrationUrl":"https://github.com/canvascloudai/cwm-bench/blob/main/typical/p95-v1/PREREGISTRATION.md","predictionsFrozenOnEngine":"1.2.18","scope":"us-east-2, m5.large app servers (1–3), 100–300 total RPS, internal load balancer, and db.r5.large MySQL","medianP95Score":87.6,"results":[{"id":"1x-100","servers":1,"rps":100,"measured":"8.6","candidate":"9.6","engine":"11.0","candidateScore":"89.0","engineScore":"72.6"},{"id":"1x-200","servers":1,"rps":200,"measured":"40.3","candidate":"32.0","engine":"14.6","candidateScore":"79.3","engineScore":"36.1"},{"id":"1x-300","servers":1,"rps":300,"measured":"94.7","candidate":"106.5","engine":"18.1","candidateScore":"87.6","engineScore":"19.1"},{"id":"2x-100","servers":2,"rps":100,"measured":"9.2","candidate":"8.0","engine":"9.2","candidateScore":"87.1","engineScore":"99.5"},{"id":"2x-200","servers":2,"rps":200,"measured":"15.7","candidate":"15.0","engine":"11.0","candidateScore":"95.7","engineScore":"70.2"},{"id":"2x-300","servers":2,"rps":300,"measured":"38.2","candidate":"34.3","engine":"12.8","candidateScore":"89.7","engineScore":"33.5"},{"id":"3x-100","servers":3,"rps":100,"measured":"7.9","candidate":"8.0","engine":"8.6","candidateScore":"99.1","engineScore":"91.0"},{"id":"3x-200","servers":3,"rps":200,"measured":"9.0","candidate":"11.7","engine":"9.8","candidateScore":"71.2","engineScore":"91.4"},{"id":"3x-300","servers":3,"rps":300,"measured":"17.9","candidate":"23.5","engine":"11.0","candidateScore":"68.8","engineScore":"61.5"}]}},"providerBenchmarks":{"aws":{"architecture":{"name":"AWS 3-Tier Web App","description":"Standard three-tier web application: Application Load Balancer distributing traffic across two m5.large EC2 instances backed by a single db.r5.large RDS MySQL instance (Single-AZ, us-east-2).","provider":"aws","region":"us-east-2","referenceRegion":"us-east-2","components":[{"role":"Load Balancer","resourceType":"network","instanceType":"ALB (Application Load Balancer)","count":1,"officialHourlyCost":0.0225,"notes":"AWS Application Load Balancer base charge (us-east-2). Scales automatically."},{"role":"Web / App Server","resourceType":"compute","instanceType":"m5.large","count":2,"officialHourlyCost":0.096,"notes":"General-purpose instance: 2 vCPU, 8 GiB RAM, up to 10 Gbps network bandwidth."},{"role":"Database","resourceType":"database","instanceType":"db.r5.large (MySQL 8.0, Single-AZ)","count":1,"officialHourlyCost":0.24,"notes":"Memory-optimized: 2 vCPU, 16 GiB RAM. gp2 storage with 3,000 IOPS burst. ~500 max connections."}]},"typicalOwnedHoldout":{"appWeight":"typical","source":"owned_cwm_bench","campaignId":"typical-v1-20260927c","measurementCommit":"6aa574d7ff9d3080b88b221bcd59f7d218ae37f0","reportUrl":"https://github.com/canvascloudai/cwm-bench/blob/main/typical/REPORT.md","summaryUrl":"https://github.com/canvascloudai/cwm-bench/blob/main/holdout/exports/typical-v1-20260927c.summary.md","scoresUrl":"https://github.com/canvascloudai/cwm-bench/blob/main/typical/scores.json","predictionsUrl":"https://github.com/canvascloudai/cwm-bench/blob/main/typical/PREDICTIONS.md","scope":{"campaigns":1,"days":1,"region":"us-east-2","attemptsPerRung":1,"engineVersion":"1.2.3","seed":20240601,"predictionStep":6,"predictionsFrozenBeforeMeasurement":true,"measuredCost":false,"topology":"Internal ALB + 2 × m5.large + db.r5.large MySQL 8.0; two workers and pool 250 per server"},"beforeFit":{"targetRps":300,"split":"holdout","score":24.3,"scoreWithoutCost":27,"metrics":[{"metric":"p50","label":"P50","unit":"ms","predicted":17,"measured":4.621661,"measuredSource":"owned_measurement","score":0,"weight":0.2,"weightedContribution":0},{"metric":"p95","label":"P95","unit":"ms","predicted":42,"measured":26.9176766,"measuredSource":"owned_measurement","score":44,"weight":0.25,"weightedContribution":11},{"metric":"appCpu","label":"App CPU","unit":"%","predicted":59.553,"measured":15.303205128205128,"measuredSource":"owned_measurement","score":0,"weight":0.2,"weightedContribution":0},{"metric":"throughput","label":"Throughput","unit":"RPS","predicted":299,"measured":262.60872607647724,"measuredSource":"owned_measurement","score":86.1,"weight":0.15,"weightedContribution":12.9},{"metric":"errorRate","label":"Error rate","unit":"%","predicted":0.284,"measured":0.0044423431456231816,"measuredSource":"owned_measurement","score":3.6,"weight":0.1,"weightedContribution":0.4},{"metric":"cost","label":"Cost","unit":"USD/hour","predicted":0.94,"measured":0.4545,"measuredSource":"list_price_reference","score":0,"weight":0.1,"weightedContribution":0}],"unweightedP99":{"predicted":236,"measured":97.54310300000054,"unit":"ms","score":0},"throughputBasis":"Measured throughput is whole-run goodput averaged over the 5-minute warmup plus 15-minute steady window; engine throughput is a post-step point rate, not a duration-weighted average.","costBasis":"Cost was not measured. Score uses the list-price reference of USD 0.4545 per hour (ALB 0.0225 + 2 × 0.096 + 0.24, no egress, no generator). Reference, not a bill.","steadyWindowAlternative":{"scored":false,"throughputRps":300,"throughputScore":99.7,"totalScore":26.3}},"dayAndRegionHoldouts":{"campaignId":"typical-holdouts-v1-20261006","reportUrl":"https://github.com/canvascloudai/cwm-bench/blob/main/typical/REPORT.md#day-and-region-holdouts-typical-holdouts-v1-20261006","comparisonBasis":"page cost basis","original":{"campaignId":"typical-v1-20260927c","targetRps":300,"score":84.2,"scoreWithoutCost":82.4,"p95Ms":26.9},"observed":[{"id":"later-day","label":"Later-day","region":"us-east-2","targetRps":300,"score":76,"scoreWithoutCost":73.3,"p95Ms":51.1},{"id":"second-region","label":"Second region","region":"us-west-2","targetRps":300,"score":81.1,"scoreWithoutCost":79,"p95Ms":32.9}],"repeatability100Rps":{"targetRps":100,"scores":[95.6,91.4]}},"context":[{"rung":"fit-20","role":"fit_data_not_holdout","score":23,"scoreWithoutCost":15.3},{"rung":"fit-100","role":"fit_data_not_holdout","score":19.8,"scoreWithoutCost":14.9},{"rung":"fit-200","role":"fit_data_not_holdout","score":16.1,"scoreWithoutCost":14.8},{"rung":"saturation-500","role":"diagnostic_only","score":25.6,"scoreWithoutCost":28.5}],"extrapolatedDiagnostic":{"targetRps":500,"label":"diagnostic extrapolation beyond the calibrated and validated range (fit at 20, 100, and 200 RPS; validated at 300 RPS), compared with a measured 500 RPS diagnostic run; excluded from headline accuracy and not a validated saturation finding","engineVersion":"1.2.20","calibrationId":"aws-crud/typical:typical-v1-20260927c:6aa574d7ff9d3080b88b221bcd59f7d218ae37f0","fitRungs":["typical-fit-20","typical-fit-100","typical-fit-200"],"score":73.8,"scoreWithoutCost":70.9,"metrics":[{"metric":"p50","label":"P50","unit":"ms","predicted":4.165220860655737,"measured":5.187581,"measuredSource":"owned_measurement","score":80.3,"weight":0.2,"weightedContribution":16.1},{"metric":"p95","label":"P95","unit":"ms","predicted":16.333451173770428,"measured":88.189861,"measuredSource":"owned_measurement","score":18.5,"weight":0.25,"weightedContribution":4.6},{"metric":"appCpu","label":"App CPU","unit":"%","predicted":24.264,"measured":26.34166695034375,"measuredSource":"owned_measurement","score":92.1,"weight":0.2,"weightedContribution":18.4},{"metric":"throughput","label":"Throughput","unit":"RPS","predicted":437.588385,"measured":437.5769855710571,"measuredSource":"owned_measurement","score":100,"weight":0.15,"weightedContribution":15},{"metric":"errorRate","label":"Error rate","unit":"%","predicted":0,"measured":0.010282815511055931,"measuredSource":"owned_measurement","score":97.2,"weight":0.1,"weightedContribution":9.7},{"metric":"cost","label":"Cost","unit":"USD/hour","predicted":0.4545,"measured":0.4545,"measuredSource":"list_price_reference","score":100,"weight":0.1,"weightedContribution":10}],"throughputBasis":"Measured throughput is whole-run goodput averaged over the 5-minute warmup plus 15-minute steady window; engine throughput is a post-step point rate, not a duration-weighted average. The post-fit model was fit to whole-run goodput at 20, 100, and 200 RPS.","costBasis":"USD 0.4545/hour is a fixed-fee public list-price reference, not a measured bill. Fixed-fee public price-list comparison, not a measured bill: scored prediction uses itemized resource costs before display rounding and excludes modeled egress and load-balancer variable data processing/ALB LCUs. The load-balancer base hourly fee remains included; other resource charges are not removed."},"afterFit":{"targetRps":300,"split":"independent holdout","engineVersion":"1.2.20","calibrationId":"aws-crud/typical:typical-v1-20260927c:6aa574d7ff9d3080b88b221bcd59f7d218ae37f0","fitRungs":["typical-fit-20","typical-fit-100","typical-fit-200"],"score":84.2,"scoreWithoutCost":82.4,"metrics":[{"metric":"p50","label":"P50","unit":"ms","predicted":4.165220860655737,"measured":4.621661,"measuredSource":"owned_measurement","score":90.1,"weight":0.2,"weightedContribution":18},{"metric":"p95","label":"P95","unit":"ms","predicted":12.78397378032783,"measured":26.9176766,"measuredSource":"owned_measurement","score":47.5,"weight":0.25,"weightedContribution":11.9},{"metric":"appCpu","label":"App CPU","unit":"%","predicted":14.74,"measured":15.303205128205128,"measuredSource":"owned_measurement","score":96.3,"weight":0.2,"weightedContribution":19.3},{"metric":"throughput","label":"Throughput","unit":"RPS","predicted":262.604014,"measured":262.60872607647724,"measuredSource":"owned_measurement","score":100,"weight":0.15,"weightedContribution":15},{"metric":"errorRate","label":"Error rate","unit":"%","predicted":0,"measured":0.0044423431456231816,"measuredSource":"owned_measurement","score":100,"weight":0.1,"weightedContribution":10},{"metric":"cost","label":"Cost","unit":"USD/hour","predicted":0.454581,"measured":0.4545,"measuredSource":"list_price_reference","score":100,"weight":0.1,"weightedContribution":10}],"throughputBasis":"Measured throughput is whole-run goodput averaged over the 5-minute warmup plus 15-minute steady window; engine throughput is a post-step point rate, not a duration-weighted average. The post-fit model was fit to whole-run goodput at 20, 100, and 200 RPS.","costBasis":"Cost was not measured. USD 0.4545/hour is a fixed-fee public list-price reference, not a bill. The frozen post-fit scored prediction predates modeled LB variable processing. It excludes modeled egress; the historical before-fit prediction included it, so their cost scores are not on identical bases."}},"seed":20240601,"generatedAt":"2026-10-23T15:29:31.670Z","overallScore":93.8,"scoringBasis":{"type":"measured_cwm_bench","costReference":"public_price_list"},"errorRateScoring":{"tolerancePercentagePoints":0.01,"outsideTolerance":"max_inverse_tolerance_or_relative_accuracy"},"scoreDriverNote":"Canonical AWS compares seeded engine app-server CPU and modeled in-VPC internal-load-balancer latency with owned cwm-bench measurements. Goodput/errors are owned; billed cost uses the us-east-2 price list. CPU and latency misses are shown without engine retuning.","fidelity":{"scorecardIncluded":true,"label":"mixed","components":{"cost":"price_list","latency":"owned","cpu":"owned","throughput":"owned","errorRate":"owned"}},"referenceProvenance":{"source":"owned_campaign","campaignId":"473f1339-f712-4096-96d6-3d4fc07cb427","campaignGeneratedAt":"2026-09-06T03:44:51.130Z","sourceRevision":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","supplementalSourceRevision":"1fdfe97572ef91a270c9d39142fb28f7a10061eb","measurementSha":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","resultsRevision":"117051b05b253b16b4b819d326bf1b15181af124","primaryRegion":"us-east-2","holdoutRegions":["us-west-2"],"fitScenarios":["idle","normal","peak"],"holdoutScenarios":["burst","later-day","second-region"],"repositoryUrl":"https://github.com/canvascloudai/cwm-bench","resultsUrl":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md","runCodeUrl":"https://github.com/canvascloudai/cwm-bench/tree/7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","supplementalRunCodeUrl":"https://github.com/canvascloudai/cwm-bench/tree/1fdfe97572ef91a270c9d39142fb28f7a10061eb","pricingUrl":"https://aws.amazon.com/ec2/pricing/on-demand/","predictionBasis":"SimulationEngine.simulateStep, seed 20240601; modeled in-VPC internal-LB P50/P95, mean of two compute-resource CPU predictions, whole-run goodput and fixed-fee price-list hourly cost (excluding modeled egress and LB variable processing).","cpuBasis":"Scored: mean application-server CPU across two app hosts; database CPU is measured separately. (2 × app + DB) / 3 is a derived diagnostic, not app CPU.","latencyBasis":"Scored: k6 in-VPC through an internal load balancer, excluding internet travel; engine latency is a modeled in-VPC LB path, not an end-to-end internet target.","costBasis":"Fixed-fee public price-list comparison, not a measured bill: scored prediction uses itemized resource costs before display rounding and excludes modeled egress and load-balancer variable data processing/ALB LCUs. The load-balancer base hourly fee remains included; other resource charges are not removed.","holdoutPolicy":"Fit idle, normal, and peak; hold out burst, later-day, and second-region runs.","note":"Canonical AWS scores owned app CPU, internal-LB in-VPC latency, goodput and error; $0.4545/hour is from the us-east-2 public price list, not a measured bill. Later-day and second-region have no separate engine prediction and do not enter the score."},"measurementSource":"owned_campaign","enginePredictionStatus":"available","scenarios":[{"scenario":{"id":"idle","label":"Idle","description":"Minimal background traffic — keep-alive checks, health probes, and occasional real requests.","trafficRps":10,"referenceMetrics":{"latencyP50Ms":2.62841,"latencyP95Ms":4.5295678,"cpuPct":0.4785260097091355,"throughputRps":8.874155959479843,"errorRatePct":0,"costPerHour":0.4545},"ownedMeasurement":{"source":"cwm-bench","campaignId":"473f1339-f712-4096-96d6-3d4fc07cb427","campaignGeneratedAt":"2026-09-06T03:44:51.130Z","runId":"473f1339-f712-4096-96d6-3d4fc07cb427:idle","sourceRevision":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","measurementSha":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","capturedAt":"2026-09-05T16:32:30.342Z","region":"us-east-2","split":"fit","targetRps":10,"metrics":{"goodputRps":8.874155959479843,"p50Ms":2.62841,"p95Ms":4.5295678,"p99Ms":7.23815152,"errorRatePct":0,"appCpuPct":0.4785260097091355,"dbCpuPct":3.337818548370722,"dbConnectionsMax":2},"blendedCpuPct":1.431623522596331,"notes":"Fit rung; goodput is the campaign's five-minute warmup plus fifteen-minute steady-window average."},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), idle fit run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"cpuPct","citation":"Owned app-server CPU mean of two application hosts; database CPU measured separately, idle fit run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"throughputRps / errorRatePct","citation":"Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 8.874155959479843 RPS goodput and 0% errors at a 10 RPS target.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"costPerHour","citation":"AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr.","url":"https://aws.amazon.com/ec2/pricing/on-demand/"}]},"simulated":{"latencyP50Ms":2.5,"latencyP95Ms":4.4,"cpuPct":0.391,"throughputRps":8.874571,"errorRatePct":0,"costPerHour":0.4545},"simulatedP99Ms":7.024620516766292,"metrics":[{"metric":"latencyP50Ms","label":"P50 in-VPC LB latency","unit":"ms","reference":2.63,"simulated":2.5,"deltaPct":-4.9,"accuracyPct":95.1},{"metric":"latencyP95Ms","label":"P95 in-VPC LB latency","unit":"ms","reference":4.53,"simulated":4.4,"deltaPct":-2.9,"accuracyPct":97.1},{"metric":"cpuPct","label":"App-server CPU","unit":"%","reference":0.48,"simulated":0.39,"deltaPct":-18.3,"accuracyPct":81.7},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":8.87,"simulated":8.87,"deltaPct":0,"accuracyPct":100},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0,"simulated":0,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.4545,"simulated":0.4545,"deltaPct":0,"accuracyPct":100}],"compositeScore":94.6,"comparisonStatus":"complete"},{"scenario":{"id":"normal","label":"Normal","description":"Typical business-hours traffic with steady sustained load.","trafficRps":100,"referenceMetrics":{"latencyP50Ms":2.1881595,"latencyP95Ms":4.0283214,"cpuPct":1.7935897435897439,"throughputRps":87.62477401512365,"errorRatePct":0,"costPerHour":0.4545},"ownedMeasurement":{"source":"cwm-bench","campaignId":"473f1339-f712-4096-96d6-3d4fc07cb427","campaignGeneratedAt":"2026-09-06T03:44:51.130Z","runId":"473f1339-f712-4096-96d6-3d4fc07cb427:normal","sourceRevision":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","measurementSha":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","capturedAt":"2026-09-05T16:53:31Z","region":"us-east-2","split":"fit","targetRps":100,"metrics":{"goodputRps":87.62477401512365,"p50Ms":2.1881595,"p95Ms":4.0283214,"p99Ms":6.78735754,"errorRatePct":0,"appCpuPct":1.7935897435897439,"dbCpuPct":4.433338557944269,"dbConnectionsMax":8},"blendedCpuPct":2.673506015041252,"notes":"Fit rung; goodput is the campaign's five-minute warmup plus fifteen-minute steady-window average."},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), normal fit run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"cpuPct","citation":"Owned app-server CPU mean of two application hosts; database CPU measured separately, normal fit run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"throughputRps / errorRatePct","citation":"Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 87.62477401512365 RPS goodput and 0% errors at a 100 RPS target.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"costPerHour","citation":"AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr.","url":"https://aws.amazon.com/ec2/pricing/on-demand/"}]},"simulated":{"latencyP50Ms":2.4,"latencyP95Ms":4.25,"cpuPct":1.901,"throughputRps":87.624266,"errorRatePct":0,"costPerHour":0.4544999999999999},"simulatedP99Ms":7.048933018961295,"metrics":[{"metric":"latencyP50Ms","label":"P50 in-VPC LB latency","unit":"ms","reference":2.19,"simulated":2.4,"deltaPct":9.7,"accuracyPct":90.3},{"metric":"latencyP95Ms","label":"P95 in-VPC LB latency","unit":"ms","reference":4.03,"simulated":4.25,"deltaPct":5.5,"accuracyPct":94.5},{"metric":"cpuPct","label":"App-server CPU","unit":"%","reference":1.79,"simulated":1.9,"deltaPct":6,"accuracyPct":94},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":87.62,"simulated":87.62,"deltaPct":0,"accuracyPct":100},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0,"simulated":0,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.4545,"simulated":0.4545,"deltaPct":0,"accuracyPct":100}],"compositeScore":95.5,"comparisonStatus":"complete"},{"scenario":{"id":"peak","label":"Peak","description":"Peak business load — product launch, end-of-day batch, or marketing campaign spike.","trafficRps":500,"referenceMetrics":{"latencyP50Ms":1.99503,"latencyP95Ms":3.75995775,"cpuPct":8.63,"throughputRps":437.62281962650434,"errorRatePct":0,"costPerHour":0.4545},"ownedMeasurement":{"source":"cwm-bench","campaignId":"473f1339-f712-4096-96d6-3d4fc07cb427","campaignGeneratedAt":"2026-09-06T03:44:51.130Z","runId":"473f1339-f712-4096-96d6-3d4fc07cb427:peak","sourceRevision":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","measurementSha":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","capturedAt":"2026-09-05T17:14:51Z","region":"us-east-2","split":"fit","targetRps":500,"metrics":{"goodputRps":437.62281962650434,"p50Ms":1.99503,"p95Ms":3.75995775,"p99Ms":7.20503306,"errorRatePct":0,"appCpuPct":8.63,"dbCpuPct":8.22,"dbConnectionsMax":43},"blendedCpuPct":8.493333333333334,"notes":"Fit rung; app CPU and DB CPU were reported as approximate campaign values."},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), peak fit run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"cpuPct","citation":"Owned approximate app-server CPU; database CPU measured separately, peak fit run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"throughputRps / errorRatePct","citation":"Owned cwm-bench Admin Benchmarks campaign 473f1339: the canonical CRUD fit rung measured 437.62281962650434 RPS goodput and 0% errors at a 500 RPS target.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"costPerHour","citation":"AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr.","url":"https://aws.amazon.com/ec2/pricing/on-demand/"}]},"simulated":{"latencyP50Ms":2,"latencyP95Ms":3.75,"cpuPct":8.61,"throughputRps":437.622913,"errorRatePct":0,"costPerHour":0.4545},"simulatedP99Ms":7.156988584272416,"metrics":[{"metric":"latencyP50Ms","label":"P50 in-VPC LB latency","unit":"ms","reference":2,"simulated":2,"deltaPct":0.2,"accuracyPct":99.8},{"metric":"latencyP95Ms","label":"P95 in-VPC LB latency","unit":"ms","reference":3.76,"simulated":3.75,"deltaPct":-0.3,"accuracyPct":99.7},{"metric":"cpuPct","label":"App-server CPU","unit":"%","reference":8.63,"simulated":8.61,"deltaPct":-0.2,"accuracyPct":99.8},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":437.62,"simulated":437.62,"deltaPct":0,"accuracyPct":100},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0,"simulated":0,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.4545,"simulated":0.4545,"deltaPct":0,"accuracyPct":100}],"compositeScore":99.8,"comparisonStatus":"complete"},{"scenario":{"id":"burst","label":"Burst","description":"Traffic burst exceeding normal peak — flash sale, viral event, or coordinated load test.","trafficRps":1000,"referenceMetrics":{"latencyP50Ms":1.9312195,"latencyP95Ms":3.71435975,"cpuPct":13.36,"throughputRps":875.1162528375527,"errorRatePct":0.00009522521716110772,"costPerHour":0.4545},"ownedMeasurement":{"source":"cwm-bench","campaignId":"473f1339-f712-4096-96d6-3d4fc07cb427","campaignGeneratedAt":"2026-09-06T03:44:51.130Z","runId":"473f1339-f712-4096-96d6-3d4fc07cb427:burst","sourceRevision":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","measurementSha":"7416cb63ace3a7ab2e3486bb6f132a2dcb574c34","capturedAt":"2026-09-05T17:35:55Z","region":"us-east-2","split":"holdout","targetRps":1000,"metrics":{"goodputRps":875.1162528375527,"p50Ms":1.9312195,"p95Ms":3.71435975,"p99Ms":7.01741024,"errorRatePct":0.00009522521716110772,"appCpuPct":13.36,"dbCpuPct":10.91,"dbConnectionsMax":55},"blendedCpuPct":12.543333333333331,"notes":"Holdout rung; one failure in 1,050,142 requests (raw fraction 9.522521716110773e-07, or 0.0000952%). Goodput includes a five-minute ramp and fifteen-minute steady window; no dropped requests are implied."},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"Owned k6 median/P95, measured in-VPC through an internal load balancer (internet travel excluded), burst holdout run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"cpuPct","citation":"Owned approximate app-server CPU; database CPU measured separately, burst holdout run.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"throughputRps / errorRatePct","citation":"Owned Burst holdout measured 875.1162528375527 RPS whole-run goodput and one failure in 1,050,142 (0.0000952%). Five-minute ramp plus fifteen-minute steady window; no dropped requests implied.","url":"https://github.com/canvascloudai/cwm-bench/blob/117051b05b253b16b4b819d326bf1b15181af124/holdout/exports/473f1339.summary.md"},{"metric":"costPerHour","citation":"AWS us-east-2 public price list, not a measured bill: ALB $0.0225/hr + 2× m5.large $0.096/hr + db.r5.large MySQL $0.24/hr = $0.4545/hr.","url":"https://aws.amazon.com/ec2/pricing/on-demand/"}]},"simulated":{"latencyP50Ms":1.45,"latencyP95Ms":3.1,"cpuPct":16.997,"throughputRps":875.121221,"errorRatePct":0,"costPerHour":0.45449999999999985},"simulatedP99Ms":7.292058040911319,"metrics":[{"metric":"latencyP50Ms","label":"P50 in-VPC LB latency","unit":"ms","reference":1.93,"simulated":1.45,"deltaPct":-24.9,"accuracyPct":75.1},{"metric":"latencyP95Ms","label":"P95 in-VPC LB latency","unit":"ms","reference":3.71,"simulated":3.1,"deltaPct":-16.5,"accuracyPct":83.5},{"metric":"cpuPct","label":"App-server CPU","unit":"%","reference":13.36,"simulated":17,"deltaPct":27.2,"accuracyPct":72.8},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":875.12,"simulated":875.12,"deltaPct":0,"accuracyPct":100},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0.000095225,"simulated":0,"deltaPct":-100,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.4545,"simulated":0.4545,"deltaPct":0,"accuracyPct":100}],"compositeScore":85.5,"comparisonStatus":"complete"}],"unpredictedHoldouts":[{"id":"later-day","label":"Later-day","split":"holdout","region":"us-east-2","targetRps":100,"runId":"473f1339-f712-4096-96d6-3d4fc07cb427:later-day","sourceRevision":"1fdfe97572ef91a270c9d39142fb28f7a10061eb","metrics":{"appCpuPct":1.68,"p50Ms":2.31,"p95Ms":4.27,"errorRatePct":0},"prediction":null,"predictionStatus":"no-separate-prediction","precision":"approximate"},{"id":"second-region","label":"Second region","split":"holdout","region":"us-west-2","targetRps":100,"runId":"473f1339-f712-4096-96d6-3d4fc07cb427:second-region","sourceRevision":"1fdfe97572ef91a270c9d39142fb28f7a10061eb","metrics":{"appCpuPct":1.57,"p50Ms":2,"p95Ms":3.84,"errorRatePct":0},"prediction":null,"predictionStatus":"no-separate-prediction","precision":"approximate"}],"historicalCalibration":{"revision":"b2f414f2d8ec0e72c4ee980baff0a0bff3a4903c","seed":20240601,"scenarioId":"burst","predictionBasis":"Canonical AWS 1,000-RPS Burst; seeded engine mean app-host CPU and modeled in-VPC internal-LB latency before owned CPU/latency calibration.","burstPrediction":{"latencyP50Ms":70,"latencyP95Ms":159,"cpuPct":95.1,"throughputRps":875.121221,"errorRatePct":0,"costPerHour":0.45230400000000004},"unpredictedHoldoutIds":["later-day","second-region"]},"isCustomConfig":false},"gcp":{"architecture":{"name":"Custom GCP 3-Tier Web App","description":"Cloud Load Balancing distributing traffic across 2× e2-standard-2 VM instances backed by a single db-n1-standard-2 Cloud SQL (us-central1).","provider":"gcp","region":"us-central1","referenceRegion":"us-central1","components":[{"role":"Load Balancer","resourceType":"network","instanceType":"Cloud Load Balancing (forwarding rule)","count":1,"officialHourlyCost":0.025,"notes":"GCP Cloud Load Balancing per forwarding-rule charge (us-central1)."},{"role":"Web / App Server","resourceType":"compute","instanceType":"e2-standard-2","count":2,"officialHourlyCost":0.067,"notes":"General-purpose: 2 vCPU (dedicated Intel Ice Lake), 8 GiB RAM. Comparable tier to AWS m5.large."},{"role":"Database","resourceType":"database","instanceType":"db-n1-standard-2 (Cloud SQL)","count":1,"officialHourlyCost":0.1917,"notes":"Standard: 2 vCPU, 7.5 GiB RAM. ~300 max connections, SSD storage."}]},"seed":20240601,"generatedAt":"2026-10-23T15:29:31.686Z","overallScore":98,"scoringBasis":{"type":"documentation_reference","costReference":"documentation_reference"},"errorRateScoring":{"tolerancePercentagePoints":0.01,"outsideTolerance":"max_inverse_tolerance_or_relative_accuracy"},"scoreDriverNote":"GCP uses its own latency and error coefficients fit to e2-standard-2 / db-n1-standard-2 reference data, plus a +5 ms idle base-latency offset (Cloud LB forwarding over AWS ALB) so idle P50 reproduces the documented floor. Latency, error rate (including Burst) and low/mid-load CPU track references closely; the main remaining gap is the shared (AWS-calibrated) database CPU curve, which slightly over-predicts CPU at Burst.","fidelity":{"scorecardIncluded":true,"label":"estimated","components":{"cost":"estimated","latency":"estimated","cpu":"estimated","throughput":"estimated","errorRate":"estimated"}},"referenceProvenance":{"source":"provider_documentation","costBasis":"Fixed-fee public price-list comparison, not a measured bill: scored prediction uses itemized resource costs before display rounding and excludes modeled egress and load-balancer variable data processing/ALB LCUs. The load-balancer base hourly fee remains included; other resource charges are not removed.","note":"Reference values are sourced from provider documentation and published load-test studies."},"measurementSource":"provider_documentation","enginePredictionStatus":"available","scenarios":[{"scenario":{"id":"idle","label":"Idle","description":"Minimal background traffic — keep-alive checks, health probes, and occasional real requests.","trafficRps":10,"referenceMetrics":{"latencyP50Ms":18,"latencyP95Ms":42,"cpuPct":6,"throughputRps":10,"errorRatePct":0.1,"costPerHour":0.3507},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"GCP e2-standard-2 (dedicated-core) + Cloud LB: idle P50 ~16–20 ms, P95 ~38–46 ms. Cloud LB adds ~2–4 ms more overhead than AWS ALB; e2-standard-2 dedicated vCPUs behave similarly to AWS m5.large with comparable baseline latency. Source: GCP Load Balancing and e2 dedicated-core documentation.","url":"https://cloud.google.com/load-balancing/docs/https"},{"metric":"cpuPct / errorRatePct","citation":"GCP e2-standard-2 at idle: ~5–7% CPU (OS + runtime baseline, similar to AWS dedicated-core at equivalent load). db-n1-standard-2 (300 max connections) at 10 RPS: ~2 active connections = 0.7% pool utilisation; transient errors ~0.08–0.12%. Source: GCP Compute and Cloud SQL n1-standard-2 documentation.","url":"https://cloud.google.com/compute/docs/general-purpose-machines#e2_machine_types"},{"metric":"costPerHour","citation":"GCP Pricing (us-central1, August 2026): Cloud LB $0.025/hr + 2× e2-standard-2 $0.067/hr + db-n1-standard-2 $0.1917/hr = $0.3507/hr.","url":"https://cloud.google.com/compute/vm-instance-pricing"}]},"simulated":{"latencyP50Ms":18,"latencyP95Ms":43,"cpuPct":6.3,"throughputRps":9.99,"errorRatePct":0.099,"costPerHour":0.34836},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":18,"simulated":18,"deltaPct":0,"accuracyPct":100},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":42,"simulated":43,"deltaPct":2.4,"accuracyPct":97.6},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":6,"simulated":6.3,"deltaPct":5,"accuracyPct":95},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":10,"simulated":9.99,"deltaPct":-0.1,"accuracyPct":99.9},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0.1,"simulated":0.1,"deltaPct":-1,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.3507,"simulated":0.3484,"deltaPct":-0.7,"accuracyPct":99.3}],"compositeScore":98.3,"comparisonStatus":"complete"},{"scenario":{"id":"normal","label":"Normal","description":"Typical business-hours traffic with steady sustained load.","trafficRps":100,"referenceMetrics":{"latencyP50Ms":22,"latencyP95Ms":52,"cpuPct":22,"throughputRps":100,"errorRatePct":0.2,"costPerHour":0.3507},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"GCP e2-standard-2 (dedicated 2 vCPU) at ~50 RPS/instance: real-world P50 ~20–24 ms, P95 ~48–56 ms. The AWS-calibrated simulator under-predicts by ~10–15% because the queuing-theory latency formula does not model GCP LB's additional forwarding overhead. Source: GCP e2-standard performance docs and Cloud Monitoring latency guidance.","url":"https://cloud.google.com/compute/docs/general-purpose-machines#e2_machine_types"},{"metric":"cpuPct","citation":"GCP e2-standard-2 (dedicated Intel core) at ~50 RPS/instance: effective reported CPU ~19–25%. Dedicated vCPUs report similarly to AWS m5.large at comparable utilisation. Source: GCP Cloud Monitoring CPU utilisation guidance for e2 standard machine types.","url":"https://cloud.google.com/monitoring/api/metrics_gcp#gcp-compute"},{"metric":"errorRatePct","citation":"GCP db-n1-standard-2 (300 max connections): at 100 RPS, ~20 concurrent connections = 6.7% pool utilisation. Well below saturation; real-world error rate ~0.15–0.25% from transient connection overhead. Source: Cloud SQL connection management documentation.","url":"https://cloud.google.com/sql/docs/mysql/manage-connections"}]},"simulated":{"latencyP50Ms":23,"latencyP95Ms":52,"cpuPct":22,"throughputRps":99.8,"errorRatePct":0.2,"costPerHour":0.34835999999999995},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":22,"simulated":23,"deltaPct":4.5,"accuracyPct":95.5},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":52,"simulated":52,"deltaPct":0,"accuracyPct":100},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":22,"simulated":22,"deltaPct":0,"accuracyPct":100},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":100,"simulated":99.8,"deltaPct":-0.2,"accuracyPct":99.8},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0.2,"simulated":0.2,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.3507,"simulated":0.3484,"deltaPct":-0.7,"accuracyPct":99.3}],"compositeScore":99,"comparisonStatus":"complete"},{"scenario":{"id":"peak","label":"Peak","description":"Peak business load — product launch, end-of-day batch, or marketing campaign spike.","trafficRps":500,"referenceMetrics":{"latencyP50Ms":46,"latencyP95Ms":105,"cpuPct":58,"throughputRps":493,"errorRatePct":0.6,"costPerHour":0.3507},"sources":[{"metric":"errorRatePct / throughputRps","citation":"GCP db-n1-standard-2 (300 max connections) at 500 RPS: ~100 concurrent connections = 33% pool utilisation. At 33% utilisation with a dedicated 2-vCPU DB the connection queue rarely backs up; real-world error rate ~0.4–0.8%. Effective throughput ~493 RPS. Source: Cloud SQL operational guidance and GCP performance best practices.","url":"https://cloud.google.com/sql/docs/mysql/connect-overview"},{"metric":"latencyP50Ms / latencyP95Ms","citation":"GCP e2-standard-2 (dedicated vCPU) at ~250 RPS/instance: real-world P50 ~42–50 ms, P95 ~96–114 ms. The AWS-calibrated queuing-theory formula slightly under-predicts due to GCP LB forwarding overhead. Source: GCP Compute and Cloud Monitoring latency guidance.","url":"https://cloud.google.com/architecture/framework/performance-optimization"},{"metric":"cpuPct","citation":"GCP e2-standard-2 at 250 RPS/instance: CPU ~54–62%. Dedicated Intel vCPU utilisation profile is similar to AWS m5.large at equivalent per-instance load. Source: GCP e2 standard machine type performance documentation.","url":"https://cloud.google.com/compute/docs/general-purpose-machines#e2_machine_types"}]},"simulated":{"latencyP50Ms":46,"latencyP95Ms":98,"cpuPct":57.3,"throughputRps":497,"errorRatePct":0.6,"costPerHour":0.34836000000000006},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":46,"simulated":46,"deltaPct":0,"accuracyPct":100},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":105,"simulated":98,"deltaPct":-6.7,"accuracyPct":93.3},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":58,"simulated":57.3,"deltaPct":-1.2,"accuracyPct":98.8},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":493,"simulated":497,"deltaPct":0.8,"accuracyPct":99.2},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0.6,"simulated":0.6,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.3507,"simulated":0.3484,"deltaPct":-0.7,"accuracyPct":99.3}],"compositeScore":97.9,"comparisonStatus":"complete"},{"scenario":{"id":"burst","label":"Burst","description":"Traffic burst exceeding normal peak — flash sale, viral event, or coordinated load test.","trafficRps":1000,"referenceMetrics":{"latencyP50Ms":82,"latencyP95Ms":185,"cpuPct":84,"throughputRps":972,"errorRatePct":2.5,"costPerHour":0.3507},"sources":[{"metric":"errorRatePct / throughputRps","citation":"GCP db-n1-standard-2 (300 max connections) at 1000 RPS: ~200 concurrent connections = 67% pool utilisation. At 67% utilisation the connection pool begins to queue intermittently; real-world error rate ~2–3% from connection wait timeouts — comparable to AWS db.r5.large behaviour at similar relative utilisation. Source: Cloud SQL best practices.","url":"https://cloud.google.com/sql/docs/mysql/manage-connections"},{"metric":"latencyP50Ms / latencyP95Ms","citation":"GCP e2-standard-2 at ~500 RPS/instance under light DB connection pressure: real-world P50 ~76–88 ms, P95 ~174–196 ms. GCP hypervisor and LB overhead add ~10–15% vs AWS at equivalent load. Source: GCP load testing guidance and Cloud Monitoring latency documentation.","url":"https://cloud.google.com/architecture/framework/performance-optimization"},{"metric":"cpuPct","citation":"GCP e2-standard-2 at ~500 RPS/instance: CPU ~80–88% under sustained burst. Dedicated vCPU utilisation curve mirrors AWS m5.large at equivalent per-core RPS. Source: GCP Compute performance documentation.","url":"https://cloud.google.com/compute/docs/general-purpose-machines"}]},"simulated":{"latencyP50Ms":82,"latencyP95Ms":169,"cpuPct":88.1,"throughputRps":975,"errorRatePct":2.5,"costPerHour":0.34836},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":82,"simulated":82,"deltaPct":0,"accuracyPct":100},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":185,"simulated":169,"deltaPct":-8.6,"accuracyPct":91.4},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":84,"simulated":88.1,"deltaPct":4.9,"accuracyPct":95.1},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":972,"simulated":975,"deltaPct":0.3,"accuracyPct":99.7},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":2.5,"simulated":2.5,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.3507,"simulated":0.3484,"deltaPct":-0.7,"accuracyPct":99.3}],"compositeScore":96.8,"comparisonStatus":"complete"}],"isCustomConfig":true},"azure":{"architecture":{"name":"Custom AZURE 3-Tier Web App","description":"Azure Load Balancer distributing traffic across 2× Standard_D2s_v3 VMs backed by a single sql-s2 SQL Database (eastus).","provider":"azure","region":"eastus","referenceRegion":"eastus","components":[{"role":"Load Balancer","resourceType":"network","instanceType":"Standard Load Balancer","count":1,"officialHourlyCost":0.025,"notes":"Azure Standard Load Balancer per-rule charge (East US)."},{"role":"Web / App Server","resourceType":"compute","instanceType":"Standard_D2s_v3","count":2,"officialHourlyCost":0.096,"notes":"General-purpose: 2 vCPU (non-burstable, dedicated Intel Broadwell), 8 GiB RAM. Comparable tier to AWS m5.large."},{"role":"Database","resourceType":"database","instanceType":"sql-s2 (SQL Database)","count":1,"officialHourlyCost":0.1008,"notes":"Standard S2: 50 DTUs, 250 GiB max size. Medium-traffic production apps."}]},"seed":20240601,"generatedAt":"2026-10-23T15:29:31.702Z","overallScore":98,"scoringBasis":{"type":"documentation_reference","costReference":"documentation_reference"},"errorRateScoring":{"tolerancePercentagePoints":0.01,"outsideTolerance":"max_inverse_tolerance_or_relative_accuracy"},"scoreDriverNote":"Azure uses its own latency and error coefficients fit to Standard_D2s_v3 / SQL Standard S2 reference data, plus a +3.75 ms idle base-latency offset so idle P50 reproduces the documented floor. Latency and error rate (including Burst) track references closely; the largest remaining divergence is at Burst, where the shared (AWS-calibrated) database CPU saturation curve over-predicts CPU for the 200-connection S2 tier.","fidelity":{"scorecardIncluded":true,"label":"estimated","components":{"cost":"estimated","latency":"estimated","cpu":"estimated","throughput":"estimated","errorRate":"estimated"}},"referenceProvenance":{"source":"provider_documentation","costBasis":"Fixed-fee public price-list comparison, not a measured bill: scored prediction uses itemized resource costs before display rounding and excludes modeled egress and load-balancer variable data processing/ALB LCUs. The load-balancer base hourly fee remains included; other resource charges are not removed.","note":"Reference values are sourced from provider documentation and published load-test studies."},"measurementSource":"provider_documentation","enginePredictionStatus":"available","scenarios":[{"scenario":{"id":"idle","label":"Idle","description":"Minimal background traffic — keep-alive checks, health probes, and occasional real requests.","trafficRps":10,"referenceMetrics":{"latencyP50Ms":16,"latencyP95Ms":38,"cpuPct":5,"throughputRps":10,"errorRatePct":0.1,"costPerHour":0.3178},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"Azure Standard_D2s_v3 (non-burstable, dedicated Intel Broadwell) + Azure Standard LB: idle P50 ~14–18 ms, P95 ~34–42 ms. Azure LB adds ~2–3 ms more overhead than AWS ALB; D2s_v3 dedicated cores behave similarly to AWS m5.large at equivalent idle load. Source: Azure VM performance docs and Standard Load Balancer documentation.","url":"https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-overview"},{"metric":"cpuPct / errorRatePct","citation":"Azure Standard_D2s_v3 at idle: ~4–6% CPU (OS + runtime baseline, similar to AWS m5.large). Azure SQL S2 (200 max connections) at 10 RPS: ~2 active connections = 1% pool utilisation; transient errors ~0.08–0.12%. Source: Azure VM sizes documentation and Azure SQL S2 resource limits.","url":"https://learn.microsoft.com/en-us/azure/azure-sql/database/resource-limits-dtu-single-databases"},{"metric":"costPerHour","citation":"Azure Pricing (East US, August 2026): Standard LB $0.025/hr + 2× Standard_D2s_v3 $0.096/hr + SQL Standard S2 $0.1008/hr = $0.3178/hr.","url":"https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/"}]},"simulated":{"latencyP50Ms":16,"latencyP95Ms":42,"cpuPct":5.4,"throughputRps":9.99,"errorRatePct":0.099,"costPerHour":0.3175},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":16,"simulated":16,"deltaPct":0,"accuracyPct":100},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":38,"simulated":42,"deltaPct":10.5,"accuracyPct":89.5},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":5,"simulated":5.4,"deltaPct":8,"accuracyPct":92},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":10,"simulated":9.99,"deltaPct":-0.1,"accuracyPct":99.9},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0.1,"simulated":0.1,"deltaPct":-1,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.3178,"simulated":0.3175,"deltaPct":-0.1,"accuracyPct":99.9}],"compositeScore":95.8,"comparisonStatus":"complete"},{"scenario":{"id":"normal","label":"Normal","description":"Typical business-hours traffic with steady sustained load.","trafficRps":100,"referenceMetrics":{"latencyP50Ms":21,"latencyP95Ms":50,"cpuPct":19,"throughputRps":100,"errorRatePct":0.2,"costPerHour":0.3178},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"Azure Standard_D2s_v3 at ~50 RPS/instance: real-world P50 ~19–23 ms, P95 ~46–54 ms. The AWS-calibrated simulator slightly under-predicts by ~10% because the queuing-theory latency formula does not model Azure LB overhead. D2s_v3 dedicated vCPUs perform comparably to AWS m5.large per-core. Source: Azure VM performance benchmarks.","url":"https://learn.microsoft.com/en-us/azure/virtual-machines/dv3-dsv3-series"},{"metric":"cpuPct","citation":"Azure Standard_D2s_v3 (non-burstable, dedicated 2-vCPU) at ~50 RPS/instance: effective CPU ~17–21%. Dedicated core metrics behave identically to AWS m5.large monitoring — no B-series credit model complexity. Source: Azure Monitor CPU metrics for Dv3 series documentation.","url":"https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-supported"},{"metric":"errorRatePct","citation":"Azure SQL Standard S2 (200 max connections, 50 DTUs) at 100 RPS: ~20 concurrent connections = 10% pool utilisation. Well below saturation; real-world error rate ~0.15–0.25%. Source: Azure SQL Database Standard S2 resource limits documentation.","url":"https://learn.microsoft.com/en-us/azure/azure-sql/database/resource-limits-dtu-single-databases"}]},"simulated":{"latencyP50Ms":21,"latencyP95Ms":52,"cpuPct":18.5,"throughputRps":99.799,"errorRatePct":0.201,"costPerHour":0.3175},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":21,"simulated":21,"deltaPct":0,"accuracyPct":100},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":50,"simulated":52,"deltaPct":4,"accuracyPct":96},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":19,"simulated":18.5,"deltaPct":-2.6,"accuracyPct":97.4},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":100,"simulated":99.8,"deltaPct":-0.2,"accuracyPct":99.8},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0.2,"simulated":0.2,"deltaPct":0.5,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.3178,"simulated":0.3175,"deltaPct":-0.1,"accuracyPct":99.9}],"compositeScore":98.4,"comparisonStatus":"complete"},{"scenario":{"id":"peak","label":"Peak","description":"Peak business load — product launch, end-of-day batch, or marketing campaign spike.","trafficRps":500,"referenceMetrics":{"latencyP50Ms":42,"latencyP95Ms":98,"cpuPct":54,"throughputRps":492,"errorRatePct":0.8,"costPerHour":0.3178},"sources":[{"metric":"errorRatePct / throughputRps","citation":"Azure SQL S2 (200 max connections, 50 DTUs) at 500 RPS: ~100 concurrent connections = 50% pool utilisation. At 50% utilisation with dedicated DTU capacity, connection queuing is minimal; real-world error rate ~0.5–1.1%. Effective throughput ~492 RPS. Source: Azure SQL DTU service tier documentation.","url":"https://learn.microsoft.com/en-us/azure/azure-sql/database/service-tiers-dtu"},{"metric":"latencyP50Ms / latencyP95Ms","citation":"Azure Standard_D2s_v3 (dedicated vCPU) at ~250 RPS/instance: real-world P50 ~38–46 ms, P95 ~90–106 ms. The AWS-calibrated model slightly under-predicts due to Azure LB overhead; dedicated non-burstable vCPUs perform similarly to AWS m5.large at peak. Source: Azure VM benchmark documentation.","url":"https://learn.microsoft.com/en-us/azure/virtual-machines/dv3-dsv3-series"},{"metric":"cpuPct","citation":"Azure Standard_D2s_v3 at 250 RPS/instance: CPU ~50–58%. Non-burstable dedicated cores remain in a predictable linear utilisation range, unlike B-series. Source: Azure Monitor D-series VM performance documentation.","url":"https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-supported"}]},"simulated":{"latencyP50Ms":42,"latencyP95Ms":98,"cpuPct":52.3,"throughputRps":496,"errorRatePct":0.8,"costPerHour":0.3174999999999999},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":42,"simulated":42,"deltaPct":0,"accuracyPct":100},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":98,"simulated":98,"deltaPct":0,"accuracyPct":100},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":54,"simulated":52.3,"deltaPct":-3.1,"accuracyPct":96.9},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":492,"simulated":496,"deltaPct":0.8,"accuracyPct":99.2},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0.8,"simulated":0.8,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.3178,"simulated":0.3175,"deltaPct":-0.1,"accuracyPct":99.9}],"compositeScore":99.3,"comparisonStatus":"complete"},{"scenario":{"id":"burst","label":"Burst","description":"Traffic burst exceeding normal peak — flash sale, viral event, or coordinated load test.","trafficRps":1000,"referenceMetrics":{"latencyP50Ms":76,"latencyP95Ms":174,"cpuPct":81,"throughputRps":962,"errorRatePct":3.8,"costPerHour":0.3178},"sources":[{"metric":"errorRatePct / throughputRps","citation":"Azure SQL S2 (200 max connections) at 1000 RPS: ~200 concurrent connections = 100% pool utilisation. At full pool saturation, connection wait timeouts begin; real-world error rate ~3–4.5%. Effective throughput ~962 RPS (1000 × 0.962). Source: Azure SQL performance guidance.","url":"https://learn.microsoft.com/en-us/azure/azure-sql/database/performance-guidance"},{"metric":"latencyP50Ms / latencyP95Ms","citation":"Azure Standard_D2s_v3 at ~500 RPS/instance with moderate DB connection pressure: real-world P50 ~70–82 ms, P95 ~162–186 ms. Non-burstable dedicated cores maintain predictable latency curves at high load. Source: Azure load testing guidance.","url":"https://learn.microsoft.com/en-us/azure/load-testing/overview-what-is-azure-load-testing"},{"metric":"cpuPct","citation":"Azure Standard_D2s_v3 at ~500 RPS/instance: CPU ~77–85% under burst. Non-burstable cores do not throttle at high utilisation (unlike B-series), so CPU scales linearly. Source: Azure D-series VM performance documentation.","url":"https://learn.microsoft.com/en-us/azure/virtual-machines/dv3-dsv3-series"}]},"simulated":{"latencyP50Ms":76,"latencyP95Ms":171,"cpuPct":86.1,"throughputRps":962,"errorRatePct":3.8,"costPerHour":0.3175000000000001},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":76,"simulated":76,"deltaPct":0,"accuracyPct":100},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":174,"simulated":171,"deltaPct":-1.7,"accuracyPct":98.3},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":81,"simulated":86.1,"deltaPct":6.3,"accuracyPct":93.7},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":962,"simulated":962,"deltaPct":0,"accuracyPct":100},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":3.8,"simulated":3.8,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.3178,"simulated":0.3175,"deltaPct":-0.1,"accuracyPct":99.9}],"compositeScore":98.3,"comparisonStatus":"complete"}],"isCustomConfig":true},"oci":{"architecture":{"name":"Custom OCI 3-Tier Web App","description":"OCI Flexible Load Balancer distributing traffic across 2× VM.Standard.E4.Flex-2OCPU VM instances backed by a single mysql-heatwave Database (us-ashburn-1).","provider":"oci","region":"us-ashburn-1","referenceRegion":"us-ashburn-1","components":[{"role":"Load Balancer","resourceType":"network","instanceType":"Flexible Load Balancer (10 Mbps–8 Gbps)","count":1,"officialHourlyCost":0.0224,"notes":"OCI Flexible Load Balancer base charge (us-ashburn-1). Bandwidth charged separately."},{"role":"Web / App Server","resourceType":"compute","instanceType":"VM.Standard.E4.Flex-2OCPU","count":2,"officialHourlyCost":0.074,"notes":"Flexible: 2 OCPU (4 AMD EPYC vCPUs), 16 GiB RAM. Comparable tier to AWS m5.large (us-ashburn-1)."},{"role":"Database","resourceType":"database","instanceType":"mysql-heatwave (Database)","count":1,"officialHourlyCost":0.074,"notes":"MySQL HeatWave DB System: 2 OCPU, 16 GiB RAM (E4.Flex). OLTP + in-database analytics (us-ashburn-1)."}]},"seed":20240601,"generatedAt":"2026-10-23T15:29:31.714Z","overallScore":96.2,"scoringBasis":{"type":"documentation_reference","costReference":"documentation_reference"},"errorRateScoring":{"tolerancePercentagePoints":0.01,"outsideTolerance":"max_inverse_tolerance_or_relative_accuracy"},"scoreDriverNote":"OCI uses its own latency, error, CPU, tail and throughput-shed coefficients fit to VM.Standard.E4.Flex 2-OCPU / MySQL HeatWave reference data, plus a +6.25 ms idle base-latency offset (EPYC + Flexible LB overhead) so idle P50 reproduces the documented floor. HeatWave's low in-memory error rates are captured by the fitted polynomial (Burst error matches closely), and a connection-pool goodput-shed term reproduces the documented Peak/Burst effective throughput (which sits below traffic × (1 − errorRate) due to admission back-pressure). The residual is in CPU: OCI's reference curve is unusually steep (low Normal 12 % yet high Burst 78 %), which the shared √-shaped compute curve plus the shared database CPU curve cannot fully track — leaving CPU slightly over-predicted at Normal and under-predicted at Peak/Burst.","fidelity":{"scorecardIncluded":true,"label":"estimated","components":{"cost":"estimated","latency":"estimated","cpu":"estimated","throughput":"estimated","errorRate":"estimated"}},"referenceProvenance":{"source":"provider_documentation","costBasis":"Fixed-fee public price-list comparison, not a measured bill: scored prediction uses itemized resource costs before display rounding and excludes modeled egress and load-balancer variable data processing/ALB LCUs. The load-balancer base hourly fee remains included; other resource charges are not removed.","note":"Reference values are sourced from provider documentation and published load-test studies."},"measurementSource":"provider_documentation","enginePredictionStatus":"available","scenarios":[{"scenario":{"id":"idle","label":"Idle","description":"Minimal background traffic — keep-alive checks, health probes, and occasional real requests.","trafficRps":10,"referenceMetrics":{"latencyP50Ms":20,"latencyP95Ms":46,"cpuPct":4,"throughputRps":10,"errorRatePct":0.15,"costPerHour":0.2444},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"OCI VM.Standard.E4.Flex 2-OCPU (4 AMD EPYC vCPUs) + OCI Flexible LB: idle P50 ~18–22 ms, P95 ~42–50 ms. OCI Flexible LB adds ~4–6 ms more round-trip overhead than AWS ALB; EPYC vCPU scheduling latency is similar to x86 at idle. Source: OCI Load Balancer and Compute performance documentation.","url":"https://docs.oracle.com/en-us/iaas/Content/Compute/References/computeshapes.htm"},{"metric":"cpuPct / errorRatePct","citation":"OCI E4.Flex 2-OCPU at idle: ~3–5% CPU (OS + runtime on 4 EPYC vCPUs — lower relative utilisation than 1-OCPU due to more available cores). MySQL HeatWave (300 max connections) at 10 RPS: ~2 active connections; OCI network fabric transient errors ~0.12–0.18%. Source: OCI MySQL HeatWave documentation.","url":"https://docs.oracle.com/en-us/iaas/mysql-database/doc/heatwave.html"},{"metric":"costPerHour","citation":"OCI Pricing (us-ashburn-1, August 2026): Flexible LB $0.0224/hr + 2× E4.Flex 2-OCPU ($0.074/hr each: 2 OCPU × $0.025 + 16 GB × $0.0015) + MySQL HeatWave $0.074/hr = $0.2444/hr.","url":"https://www.oracle.com/cloud/price-list/"}]},"simulated":{"latencyP50Ms":21,"latencyP95Ms":46,"cpuPct":4.3,"throughputRps":9.989,"errorRatePct":0.103,"costPerHour":0.249968},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":20,"simulated":21,"deltaPct":5,"accuracyPct":95},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":46,"simulated":46,"deltaPct":0,"accuracyPct":100},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":4,"simulated":4.3,"deltaPct":7.5,"accuracyPct":92.5},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":10,"simulated":9.99,"deltaPct":-0.1,"accuracyPct":99.9},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0.15,"simulated":0.1,"deltaPct":-31.3,"accuracyPct":68.7},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.2444,"simulated":0.25,"deltaPct":2.3,"accuracyPct":97.7}],"compositeScore":94.1,"comparisonStatus":"complete"},{"scenario":{"id":"normal","label":"Normal","description":"Typical business-hours traffic with steady sustained load.","trafficRps":100,"referenceMetrics":{"latencyP50Ms":34,"latencyP95Ms":76,"cpuPct":12,"throughputRps":100,"errorRatePct":0.18,"costPerHour":0.2444},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"OCI E4.Flex 2-OCPU at ~50 RPS/instance: real-world P50 ~30–38 ms, P95 ~70–82 ms. The AWS-calibrated simulator under-predicts by ~20–25% because the latency formula does not model OCI's EPYC scheduling overhead or OCI LB network latency. With 4 vCPUs per instance, queuing at 50 RPS is minimal. Source: OCI Compute performance documentation.","url":"https://docs.oracle.com/en-us/iaas/Content/Compute/References/computeshapes.htm#flexible"},{"metric":"cpuPct","citation":"OCI E4.Flex 2-OCPU (4 EPYC vCPUs) at ~50 RPS/instance: effective CPU ~10–14%. With 4 vCPUs handling 50 RPS, per-core utilisation is low; the AWS-calibrated sqrt-of-utilisation formula over-predicts because it was tuned to 2-vCPU m5.large scaling characteristics. Source: OCI Compute performance documentation.","url":"https://docs.oracle.com/en-us/iaas/Content/Compute/References/computeshapes.htm"},{"metric":"errorRatePct","citation":"MySQL HeatWave (300 max connections) at 100 RPS: ~20 concurrent connections = 6.7% pool utilisation. HeatWave in-memory query acceleration keeps connection hold times short; real-world error rate ~0.14–0.22%. Source: OCI MySQL HeatWave documentation.","url":"https://docs.oracle.com/en-us/iaas/mysql-database/doc/heatwave.html"}]},"simulated":{"latencyP50Ms":32,"latencyP95Ms":67,"cpuPct":12.1,"throughputRps":99.795,"errorRatePct":0.204,"costPerHour":0.249968},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":34,"simulated":32,"deltaPct":-5.9,"accuracyPct":94.1},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":76,"simulated":67,"deltaPct":-11.8,"accuracyPct":88.2},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":12,"simulated":12.1,"deltaPct":0.8,"accuracyPct":99.2},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":100,"simulated":99.8,"deltaPct":-0.2,"accuracyPct":99.8},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0.18,"simulated":0.2,"deltaPct":13.3,"accuracyPct":86.7},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.2444,"simulated":0.25,"deltaPct":2.3,"accuracyPct":97.7}],"compositeScore":94.1,"comparisonStatus":"complete"},{"scenario":{"id":"peak","label":"Peak","description":"Peak business load — product launch, end-of-day batch, or marketing campaign spike.","trafficRps":500,"referenceMetrics":{"latencyP50Ms":82,"latencyP95Ms":162,"cpuPct":48,"throughputRps":487,"errorRatePct":0.35,"costPerHour":0.2444},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"OCI E4.Flex 2-OCPU at ~250 RPS/instance: real-world P50 ~75–89 ms, P95 ~150–174 ms. The AWS-calibrated queuing-theory formula under-predicts OCI latency by ~20–25% at peak due to EPYC scheduling and OCI LB overhead. With 4 vCPUs at 250 RPS, the instance has meaningful headroom. Source: OCI performance best practices.","url":"https://docs.oracle.com/en-us/iaas/Content/Compute/References/computeshapes.htm"},{"metric":"cpuPct","citation":"OCI E4.Flex 2-OCPU at 250 RPS/instance: real-world CPU ~44–52%. 4 EPYC vCPUs at 250 RPS load produce roughly half the per-core utilisation compared to a 2-vCPU AWS m5.large. Source: OCI Compute performance documentation.","url":"https://docs.oracle.com/en-us/iaas/Content/Compute/References/computeshapes.htm"},{"metric":"errorRatePct / throughputRps","citation":"MySQL HeatWave at 500 RPS: ~100 concurrent connections = 33% pool utilisation (300 max). HeatWave in-memory acceleration returns query results before connection hold time causes pool pressure; real-world error rate ~0.25–0.45%. Effective throughput ~487 RPS. Source: OCI MySQL HeatWave benchmarks.","url":"https://www.oracle.com/mysql/heatwave/"}]},"simulated":{"latencyP50Ms":83,"latencyP95Ms":162,"cpuPct":44.4,"throughputRps":487,"errorRatePct":0.35,"costPerHour":0.249968},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":82,"simulated":83,"deltaPct":1.2,"accuracyPct":98.8},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":162,"simulated":162,"deltaPct":0,"accuracyPct":100},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":48,"simulated":44.4,"deltaPct":-7.5,"accuracyPct":92.5},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":487,"simulated":487,"deltaPct":0,"accuracyPct":100},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0.35,"simulated":0.35,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.2444,"simulated":0.25,"deltaPct":2.3,"accuracyPct":97.7}],"compositeScore":98,"comparisonStatus":"complete"},{"scenario":{"id":"burst","label":"Burst","description":"Traffic burst exceeding normal peak — flash sale, viral event, or coordinated load test.","trafficRps":1000,"referenceMetrics":{"latencyP50Ms":155,"latencyP95Ms":298,"cpuPct":78,"throughputRps":882,"errorRatePct":2.8,"costPerHour":0.2444},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"OCI E4.Flex 2-OCPU at ~500 RPS/instance: real-world P50 ~144–166 ms, P95 ~280–316 ms. OCI LB + EPYC scheduling overhead add ~15–20% vs the AWS-calibrated model at equivalent per-instance RPS. Source: OCI performance documentation.","url":"https://docs.oracle.com/en-us/iaas/Content/Compute/References/computeshapes.htm"},{"metric":"cpuPct","citation":"OCI E4.Flex 2-OCPU at 500 RPS/instance: CPU ~74–82% as the 4 EPYC cores approach saturation under combined compute + network I/O load. Still below the saturation threshold a 2-vCPU instance would hit at the same RPS. Source: OCI Compute performance docs.","url":"https://docs.oracle.com/en-us/iaas/Content/Compute/References/computeshapes.htm"},{"metric":"errorRatePct / throughputRps","citation":"MySQL HeatWave at 1000 RPS: ~200 connections = 67% pool utilisation (300 max). HeatWave in-memory processing keeps connection hold times low; real-world error rate ~2.2–3.4% at 67% pool utilisation — lower than a disk-based DB at equivalent utilisation. Effective throughput ~882 RPS. Source: OCI MySQL HeatWave documentation.","url":"https://docs.oracle.com/en-us/iaas/mysql-database/doc/db-system-limits.html"}]},"simulated":{"latencyP50Ms":155,"latencyP95Ms":297,"cpuPct":82.6,"throughputRps":883,"errorRatePct":2.8,"costPerHour":0.24996799999999997},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":155,"simulated":155,"deltaPct":0,"accuracyPct":100},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":298,"simulated":297,"deltaPct":-0.3,"accuracyPct":99.7},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":78,"simulated":82.6,"deltaPct":5.9,"accuracyPct":94.1},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":882,"simulated":883,"deltaPct":0.1,"accuracyPct":99.9},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":2.8,"simulated":2.8,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.2444,"simulated":0.25,"deltaPct":2.3,"accuracyPct":97.7}],"compositeScore":98.5,"comparisonStatus":"complete"}],"isCustomConfig":true},"digitalocean":{"architecture":{"name":"Custom DIGITALOCEAN 3-Tier Web App","description":"DigitalOcean Load Balancer distributing traffic across 2× s-4vcpu-8gb Droplets backed by a single db-s-2vcpu-4gb Managed Database (nyc1).","provider":"digitalocean","region":"nyc1","referenceRegion":"nyc1","components":[{"role":"Load Balancer","resourceType":"network","instanceType":"Standard Load Balancer","count":1,"officialHourlyCost":0.0164,"notes":"DigitalOcean Standard Load Balancer flat charge (NYC1). $12/mo."},{"role":"Web / App Server","resourceType":"compute","instanceType":"s-4vcpu-8gb","count":2,"officialHourlyCost":0.06575342465753424,"notes":"Basic Droplet: 4 vCPU, 8 GiB RAM. Mid-tier general-purpose workload, comparable to AWS m5.large (NYC1)."},{"role":"Database","resourceType":"database","instanceType":"db-s-2vcpu-4gb (Managed Database)","count":1,"officialHourlyCost":0.08342465753424658,"notes":"Managed PostgreSQL/MySQL standard node: 2 vCPU, 4 GiB RAM. ~200 max connections (NYC1)."}]},"seed":20240601,"generatedAt":"2026-10-23T15:29:31.725Z","overallScore":97.4,"scoringBasis":{"type":"documentation_reference","costReference":"documentation_reference"},"errorRateScoring":{"tolerancePercentagePoints":0.01,"outsideTolerance":"max_inverse_tolerance_or_relative_accuracy"},"scoreDriverNote":"DigitalOcean uses its own latency and error coefficients fit to s-4vcpu-8gb / db-s-2vcpu-4gb reference data, plus a +9 ms idle base-latency offset (shared-tenant network fabric) so idle P50 reproduces the documented floor. Latency and CPU track references across all tiers; the main remaining divergence is the very low documented error rates at Idle/Normal, where sub-0.5 % absolute references make small noise-floor differences a large relative delta.","fidelity":{"scorecardIncluded":true,"label":"estimated","components":{"cost":"estimated","latency":"estimated","cpu":"estimated","throughput":"estimated","errorRate":"estimated"}},"referenceProvenance":{"source":"provider_documentation","costBasis":"Fixed-fee public price-list comparison, not a measured bill: scored prediction uses itemized resource costs before display rounding and excludes modeled egress and load-balancer variable data processing/ALB LCUs. The load-balancer base hourly fee remains included; other resource charges are not removed.","note":"Reference values are sourced from provider documentation and published load-test studies."},"measurementSource":"provider_documentation","enginePredictionStatus":"available","scenarios":[{"scenario":{"id":"idle","label":"Idle","description":"Minimal background traffic — keep-alive checks, health probes, and occasional real requests.","trafficRps":10,"referenceMetrics":{"latencyP50Ms":24,"latencyP95Ms":54,"cpuPct":6,"throughputRps":10,"errorRatePct":0.3,"costPerHour":0.2302},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"DigitalOcean s-4vcpu-8gb Droplet + DO Standard Load Balancer: idle P50 ~22–26 ms, P95 ~50–58 ms. DO LB adds ~5–7 ms more overhead than AWS ALB; DO nyc1 network path contributes additional latency vs AWS us-east-1. Source: DigitalOcean Load Balancer and Droplet performance documentation.","url":"https://docs.digitalocean.com/products/droplets/"},{"metric":"cpuPct / errorRatePct","citation":"DO s-4vcpu-8gb at idle: ~5–7% CPU (OS + runtime baseline on 4 shared vCPUs). db-s-2vcpu-4gb (200 max connections) at 10 RPS: ~2 connections = 1% pool utilisation; DO managed DB shared infrastructure causes ~0.2–0.4% background error rate. Source: DigitalOcean Droplet and Managed Databases documentation.","url":"https://docs.digitalocean.com/products/databases/mysql/"},{"metric":"costPerHour","citation":"DO Pricing (nyc1, August 2026): Standard LB $0.0164/hr + 2× s-4vcpu-8gb $0.0658/hr + db-s-2vcpu-4gb $0.08219/hr = $0.2302/hr.","url":"https://www.digitalocean.com/pricing/"}]},"simulated":{"latencyP50Ms":25,"latencyP95Ms":54,"cpuPct":6.3,"throughputRps":9.978,"errorRatePct":0.217,"costPerHour":0.23137397328767123},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":24,"simulated":25,"deltaPct":4.2,"accuracyPct":95.8},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":54,"simulated":54,"deltaPct":0,"accuracyPct":100},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":6,"simulated":6.3,"deltaPct":5,"accuracyPct":95},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":10,"simulated":9.98,"deltaPct":-0.2,"accuracyPct":99.8},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0.3,"simulated":0.22,"deltaPct":-27.7,"accuracyPct":72.3},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.2302,"simulated":0.2314,"deltaPct":0.5,"accuracyPct":99.5}],"compositeScore":95.3,"comparisonStatus":"complete"},{"scenario":{"id":"normal","label":"Normal","description":"Typical business-hours traffic with steady sustained load.","trafficRps":100,"referenceMetrics":{"latencyP50Ms":32,"latencyP95Ms":70,"cpuPct":15,"throughputRps":100,"errorRatePct":0.4,"costPerHour":0.2302},"sources":[{"metric":"latencyP50Ms / latencyP95Ms","citation":"DO s-4vcpu-8gb at ~50 RPS/instance: real-world P50 ~29–35 ms, P95 ~64–76 ms. The AWS-calibrated simulator under-predicts because the queuing-theory latency formula was tuned to AWS m5.large network characteristics and does not model DO's network overhead. Source: DigitalOcean Droplet performance benchmarks.","url":"https://docs.digitalocean.com/products/droplets/resources/choose-plan/"},{"metric":"cpuPct","citation":"DO s-4vcpu-8gb (4 shared vCPUs) at ~50 RPS/instance: real-world CPU ~12–18%. The AWS-calibrated sqrt-of-utilisation formula over-predicts CPU because it was tuned to AWS m5.large (maxThroughput=5000 RPS); at 50/3600 = 1.4% throughput utilisation, real 4-vCPU Droplets run ~12–18%. Source: DigitalOcean Droplet performance guidance.","url":"https://docs.digitalocean.com/products/droplets/resources/choose-plan/"},{"metric":"errorRatePct","citation":"DO db-s-2vcpu-4gb (200 max connections) at 100 RPS: ~20 concurrent connections = 10% pool utilisation. At 10% utilisation with 2-vCPU managed DB, error rate is low; real-world ~0.3–0.5%. Source: DigitalOcean Managed Databases documentation.","url":"https://docs.digitalocean.com/products/databases/mysql/"}]},"simulated":{"latencyP50Ms":32,"latencyP95Ms":68,"cpuPct":15.4,"throughputRps":99.558,"errorRatePct":0.442,"costPerHour":0.23137397328767123},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":32,"simulated":32,"deltaPct":0,"accuracyPct":100},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":70,"simulated":68,"deltaPct":-2.9,"accuracyPct":97.1},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":15,"simulated":15.4,"deltaPct":2.7,"accuracyPct":97.3},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":100,"simulated":99.56,"deltaPct":-0.4,"accuracyPct":99.6},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":0.4,"simulated":0.44,"deltaPct":10.5,"accuracyPct":89.5},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.2302,"simulated":0.2314,"deltaPct":0.5,"accuracyPct":99.5}],"compositeScore":97.6,"comparisonStatus":"complete"},{"scenario":{"id":"peak","label":"Peak","description":"Peak business load — product launch, end-of-day batch, or marketing campaign spike.","trafficRps":500,"referenceMetrics":{"latencyP50Ms":68,"latencyP95Ms":142,"cpuPct":44,"throughputRps":488,"errorRatePct":1.5,"costPerHour":0.2302},"sources":[{"metric":"cpuPct","citation":"DO s-4vcpu-8gb at ~250 RPS/instance (6.9% of 3600-RPS capacity): real-world CPU ~40–48% on 4-vCPU Droplets at this absolute RPS level. The AWS-calibrated sqrt-of-utilisation formula still over-predicts somewhat due to capacity-basis mismatch but is more accurate than s-2vcpu-4gb was. Source: DigitalOcean Droplet performance guidance.","url":"https://docs.digitalocean.com/products/droplets/"},{"metric":"latencyP50Ms / latencyP95Ms","citation":"DO s-4vcpu-8gb at 250 RPS/instance with moderate DB connection pressure: real-world P50 ~62–74 ms, P95 ~132–152 ms. More headroom than s-2vcpu-4gb at the same load, with less queuing pressure. Source: DigitalOcean Managed Databases operational guidance.","url":"https://docs.digitalocean.com/products/databases/mysql/how-to/scale/"},{"metric":"errorRatePct / throughputRps","citation":"DO db-s-2vcpu-4gb (200 max connections) at 500 RPS: ~100 concurrent connections = 50% pool utilisation. At 50% utilisation error rate is modest; real-world ~1.2–1.8%. Effective throughput ~488 RPS. Source: DigitalOcean Managed Databases operational guidance.","url":"https://docs.digitalocean.com/products/databases/mysql/how-to/scale/"}]},"simulated":{"latencyP50Ms":68,"latencyP95Ms":136,"cpuPct":41.9,"throughputRps":492.505,"errorRatePct":1.499,"costPerHour":0.23137397328767123},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":68,"simulated":68,"deltaPct":0,"accuracyPct":100},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":142,"simulated":136,"deltaPct":-4.2,"accuracyPct":95.8},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":44,"simulated":41.9,"deltaPct":-4.8,"accuracyPct":95.2},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":488,"simulated":492.5,"deltaPct":0.9,"accuracyPct":99.1},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":1.5,"simulated":1.5,"deltaPct":-0.1,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.2302,"simulated":0.2314,"deltaPct":0.5,"accuracyPct":99.5}],"compositeScore":97.8,"comparisonStatus":"complete"},{"scenario":{"id":"burst","label":"Burst","description":"Traffic burst exceeding normal peak — flash sale, viral event, or coordinated load test.","trafficRps":1000,"referenceMetrics":{"latencyP50Ms":118,"latencyP95Ms":228,"cpuPct":68,"throughputRps":952,"errorRatePct":4.5,"costPerHour":0.2302},"sources":[{"metric":"cpuPct","citation":"DO s-4vcpu-8gb at 500 RPS/instance (13.9% of 3600-RPS capacity): real-world CPU ~64–72% on 4-vCPU Droplets. The AWS-calibrated formula over-predicts toward ~85% at this load due to capacity-basis mismatch, but less severely than with the smaller s-2vcpu-4gb. Source: DigitalOcean Droplet performance guidance.","url":"https://docs.digitalocean.com/products/droplets/resources/choose-plan/"},{"metric":"latencyP50Ms / latencyP95Ms","citation":"DO s-4vcpu-8gb under burst with DB approaching saturation: real-world P50 ~110–126 ms, P95 ~212–244 ms. Lower than s-2vcpu-4gb at equivalent load due to more vCPU headroom and a 200-connection DB vs the old 75-connection one. Source: DigitalOcean performance guidance.","url":"https://docs.digitalocean.com/products/droplets/resources/choose-plan/"},{"metric":"errorRatePct / throughputRps","citation":"DO db-s-2vcpu-4gb at 1000 RPS: ~200 concurrent connections = 100% pool utilisation, causing connection wait timeouts; real-world error rate ~4–5%. Effective throughput ~952 RPS (1000 × 0.952). Source: DigitalOcean Managed Databases documentation.","url":"https://docs.digitalocean.com/products/databases/mysql/"}]},"simulated":{"latencyP50Ms":118,"latencyP95Ms":230,"cpuPct":70.3,"throughputRps":955,"errorRatePct":4.5,"costPerHour":0.23137397328767126},"metrics":[{"metric":"latencyP50Ms","label":"P50 Latency","unit":"ms","reference":118,"simulated":118,"deltaPct":0,"accuracyPct":100},{"metric":"latencyP95Ms","label":"P95 Latency","unit":"ms","reference":228,"simulated":230,"deltaPct":0.9,"accuracyPct":99.1},{"metric":"cpuPct","label":"CPU Utilization","unit":"%","reference":68,"simulated":70.3,"deltaPct":3.4,"accuracyPct":96.6},{"metric":"throughputRps","label":"Throughput","unit":"req/s","reference":952,"simulated":955,"deltaPct":0.3,"accuracyPct":99.7},{"metric":"errorRatePct","label":"Error Rate","unit":"%","reference":4.5,"simulated":4.5,"deltaPct":0,"accuracyPct":100},{"metric":"costPerHour","label":"Cost / Hour","unit":"$/hr","reference":0.2302,"simulated":0.2314,"deltaPct":0.5,"accuracyPct":99.5}],"compositeScore":99,"comparisonStatus":"complete"}],"isCustomConfig":true}}}