RL Training Environments

Train reinforcement learning agents against a live cloud simulation. Each environment wraps a simulation as a Gym-compatible loop — episodes, observations, scalar rewards, and discrete actions — with no real cloud spend.

Training loop

1.Reset

POST /reset

Start a fresh episode from a clean initial state.

2.Observe

GET /observation

Read the current 6-field observation vector.

3.Step

POST /step

Submit an action and receive next obs + reward.

4.Reward

reward signal

Decomposed reward: latency, cost, and throughput.

Quick start

Create an environment, run a full episode, then reset and repeat.

Python

# 1. Create an environment linked to a simulation
env = requests.post(f"{BASE}/api/rl/environments",
    headers=HEADERS, json={"simulationId": sim_id}).json()
env_id = env["id"]

# 2. Training loop
for episode in range(NUM_EPISODES):
    obs = env["observation"]          # initial observation vector

    while True:
        action = policy(obs)          # your agent's decision
        step = requests.post(
            f"{BASE}/api/rl/environments/{env_id}/step",
            headers=HEADERS,
            json={"action": action}   # scale_up | scale_down | no_op
        ).json()

        obs    = step["observation"]  # next state vector
        reward = step["reward"]       # scalar reward signal
        done   = step["done"]         # episode complete?

        if done:
            break

    # Reset to start a fresh episode
    requests.post(
        f"{BASE}/api/rl/environments/{env_id}/reset",
        headers=HEADERS
    )

Action space

The action space is discrete with three choices. Pass the string label as the action field in the POST /step body.

scale_up

Add one instance to the pool

Reduces CPU / latency, increases cost

scale_down

Remove one instance from the pool

Lowers cost, risks latency spike under load

no_op

Hold current instance count unchanged

Baseline — observe without intervening

Observation vector

Each step returns a 6-field observation vector: rps, cpu_util, instances, traffic, currentTime, and tick_seconds. Only cpu_util is a normalised 0–1 fraction; the others are raw values.

Python

# Observation vector fields
{
  "rps": 1240,            # requests per second
  "cpu_util": 0.72,       # 0–1 fraction, current CPU load
  "instances": 4,         # active compute instances
  "traffic": 1.8,         # current traffic relative to baseline
  "currentTime": 720,     # elapsed simulation time (seconds)
  "tick_seconds": 60      # simulation seconds advanced per step
}

Reward signal

The scalar reward is decomposed into three components so you can inspect what the agent is optimising for.

JSON

# Reward shaping — returned on every /step response
{
  "reward": -0.34,
  "rewardComponents": {
    "latencyPenalty":  -0.20,   # latency above SLO target
    "costPenalty":     -0.18,   # cost above budget
    "throughputBonus":  0.04    # requests served successfully
  },
  "done": false,
  "episodeStep": 12,
  "maxSteps": 100
}

Episode lifecycle

Environments are idle-expired after 2 hours without a/step or/reset call.

Each episode runs for up to 100 steps by default. Whendone: true, callPOST /reset to continue.

Use GET /observation at any time to read the current state without consuming a step.

Eval episodes

After training, run eval episodes with real-world cost dimensions priced in to check whether your policy is robust. A reward collapse means the agent's score dropped significantly once egress and cross-AZ charges were applied — a sign the policy exploited those costs being free during training.

Your environments

Enter your API key to list environments and run eval episodes on each one. Requires a key with write scope.