RL Environments for Cloud & DevOps Agents

An RL environment is where an agent observes state, takes actions, and receives rewards over episodes. Cloud World Model's RL environments wrap a simulated cloud infrastructure, including traffic, autoscaling, failures, and modeled cost, latency and errors, so agents can train through create, reset and step calls over REST or MCP without provisioning real cloud resources.

Cloud state and observations

An environment links to a simulation. Its observation separates decision features from evaluation metrics: obs includes rps, cpu_util, instances, traffic, currentTime and tick_seconds; metrics reports cost_usd_hr, latency_p95, error_rate, uptime and sla_violations, with database or serverless details when present.

Actions and rewards

Each step submits an action object with a type and parameters. Supported action types are adjust_threshold, scale_out, scale_in, add_resource, remove_resource, no_op and set_recovery_policy. The response includes a scalar reward and reward_components; custom rewardWeights can prioritize cost, resilience, latency, performance and stability.

Configure a documented environment

Create a simulation using a supported provider—AWS, GCP, Azure, OCI or DigitalOcean—and link the environment to it. Episode settings include maxSteps, initialTraffic, targetTrafficPattern (ramp, burst, step, wave or custom), targetSLA, costBudgetPerHour, enableFailures and tick_seconds. Configure rewardWeights for cost, resilience, latency, performance and stability. Simulation scenarios can provide a starting configuration.

Create, reset, step and evaluate

Create with POST /api/rl/environments and a simulationId, then use the returned environment.id. POST /reset starts a fresh episode; POST /step advances one action and returns the next obs, metrics, reward and done flag. GET /observation reads state without advancing a step. POST /batch-step accepts a steps array of up to 30 action entries, executes them in order and stops early if the episode ends; each entry can set tick_seconds. POST /validate-action checks an action against budget, SLA, region and compliance policy without changing state. POST /eval-episodes takes an actions array of ordered episode action sequences and runs isolated rollouts without mutating the stored environment; it is asynchronous by default, or ?sync=true returns results inline for up to 3 episodes of 20 steps each.

# Use the public API base URL; set CWM_API_KEY in your shell before running requests.
BASE="https://www.cloudworldmodel.ai"

# Create an environment linked to an existing simulation
curl -X POST "$BASE/api/rl/environments" \
  -H "Authorization: Bearer $CWM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"simulationId":"<simulation-id>","episodeConfig":{"maxSteps":40,"tick_seconds":60}}'

# Step with a documented nested action object
curl -X POST "$BASE/api/rl/environments/<environment-id>/step" \
  -H "Authorization: Bearer $CWM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"action":{"type":"no_op","parameters":{}},"tick_seconds":60}'

# Reset for another episode
curl -X POST "$BASE/api/rl/environments/<environment-id>/reset" \
  -H "Authorization: Bearer $CWM_API_KEY" \
  -H "Content-Type: application/json" -d '{}'

# Execute ordered actions in one request (up to 30)
curl -X POST "$BASE/api/rl/environments/<environment-id>/batch-step" \
  -H "Authorization: Bearer $CWM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"steps":[{"action":{"type":"scale_out","parameters":{"instanceCount":1}}},{"action":{"type":"no_op","parameters":{}}}]}'

# Check a policy decision without advancing the episode
curl -X POST "$BASE/api/rl/environments/<environment-id>/validate-action" \
  -H "Authorization: Bearer $CWM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"action":{"type":"scale_out","parameters":{"instanceCount":1}}}'

# Evaluate an isolated action sequence; sync mode is for small workloads
curl -X POST "$BASE/api/rl/environments/<environment-id>/eval-episodes?sync=true" \
  -H "Authorization: Bearer $CWM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"actions":[[{"type":"no_op","parameters":{}},{"type":"scale_out","parameters":{"instanceCount":1}}]],"collapseThreshold":0.20}'

Where RL environments can help

Use episodes to explore autoscaling thresholds, compare scale-out and scale-in decisions under changing traffic, test resource changes, and study modeled cost, latency, errors and resilience trade-offs. Scenarios can help make training and evaluation workloads repeatable.

Sim-to-real limits

A learned policy acts on the simulator's modeled state, action behavior and reward—not on a live cloud account. Results depend on the modeled workload, provider coverage and assumptions, and may differ from real telemetry. Evaluate policies on independent workloads and real telemetry, then use staged rollout guardrails; simulation does not establish production safety or guarantee a specific outcome.

Frequently asked questions

Eval episodes

The panel below submits an isolated synchronous evaluation rollout with the selected modeled cost overrides. It does not change the saved environment or establish how a policy will perform in production.