RL environments for autoscaling agents

Definition: observe, act, reward, repeat

An RL autoscaling environment lets an agent select a capacity action, receive a new simulated observation and reward, and repeat until an episode ends. In CWM an environment is linked to a simulation; the policy operates on modeled behavior, not live cluster control. Reward shaping can encourage good cost and latency tradeoffs but does not prove a policy will be safe against real traffic. Think of the environment as a controlled practice setting: the infrastructure state is virtual, the episode can be reset, and actions advance only the simulation.

Read observations as inputs, not guarantees

The environment reference separates decision features from evaluation metrics. Observations can include request rate, CPU utilization, instance count, traffic and simulation time; the metrics report can include hourly cost, p95 latency, error rate, uptime and SLA violations, with details for some database or serverless resources. Before training, decide which values the policy should observe and which outcomes you will use to evaluate it. A policy cannot react to information omitted from its observation, and a reported metric is a modeled value rather than live telemetry.

Design a useful training and evaluation split

Use a repeatable scenario or defined traffic pattern to compare policy behavior, then keep distinct workload patterns for evaluation. Hold the initial topology and offered traffic consistent when comparing a baseline with a learned policy; otherwise a changed input can explain the difference. Inspect the action sequence alongside cost, latency, errors and scaling decisions rather than relying on a single accumulated reward. Reward components reveal how the configured priorities shaped the score, but they do not establish that the policy meets an external service objective.

Keep episodes interpretable

Choose a starting simulation and episode configuration that make the scaling decision visible. Set the starting traffic and target pattern intentionally, and note the time step used by each action. A no-op action can establish a baseline for the observation and reward contract before introducing scale-out or scale-in decisions. Reset the environment when you want a fresh episode; do not compare runs with different starting state as if only the policy changed. The API reference documents optional episode settings and validates the nested action structure.

What the available evidence says

The environment reference documents observation and reward fields and a step/reset loop. The owned AWS accuracy report measures an AWS reference configuration and explicitly discusses which outputs remain documentation-backed; it is not an RL generalization benchmark. Provider coverage also differs by service and evidence type. Check these boundaries when selecting a simulated architecture, then evaluate a trained policy on held-out workload patterns and independent telemetry before considering deployment.

Inspect provider service coverage

Worked API example: one safe autoscaling step

First create a simulation using the linked walkthrough and save its returned ID. Then create an environment referencing that ID. The step action is a nested object, not a plain string: type no_op with empty parameters is a valid baseline action under the RL request schema. Replace the key and IDs with actual values from your own requests; reset to start another episode.

curl -sS -X POST https://www.cloudworldmodel.ai/api/rl/environments \
  -H "Authorization: Bearer $CWM_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"simulationId\":\"$SIM_ID\",\"episodeConfig\":{}}"
# Save the returned environment.id as ENV_ID.
curl -sS -X POST "https://www.cloudworldmodel.ai/api/rl/environments/$ENV_ID/step" \
  -H "Authorization: Bearer $CWM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"action":{"type":"no_op","parameters":{}},"tick_seconds":60}'
curl -sS -X POST "https://www.cloudworldmodel.ai/api/rl/environments/$ENV_ID/reset" \
  -H "Authorization: Bearer $CWM_API_KEY" \
  -H "Content-Type: application/json" -d '{}'

Evaluate transfer, not just reward

Separate training scenarios from evaluation scenarios. Record latency, error rates, cost and scaling decisions for baseline and learned policies on identical workloads; compare any proposed live rollout with telemetry and rollback controls. A higher simulator reward only establishes performance against that reward function and model, not production reliability. When moving beyond simulation, begin with a reviewable recommendation or a guarded, reversible rollout. Check whether live signals match the modeled assumptions and retain a rollback path; simulated evaluation does not itself authorize or automate production control.