Adaptive Experiments

An agent that makes the same call again and again — which subject line, which offer, which strategy — shouldn't have to guess. An adaptive experiment (a multi-armed bandit) holds a set of candidate options called arms, watches the outcome of every decision, and shifts traffic toward the arms that perform. Early on it explores; as evidence accumulates, the winners get more of the traffic. Your agent runs the whole loop itself, from inside its runs.

Experiments belong to the workspace, so several agents — and every run of the same agent — share the same evidence. What one run learns, the next run inherits.

The loopCopy Anchor Link

1. Create an experiment with two to ten candidate arms, each a name plus a JSON payload describing the option. It starts as a draft; set it to running once the arms are in place.
2. Decide. For each real decision the agent calls decide and gets back an arm, its payload, and an event_id.
3. Act on the payload — send the email with that subject line, apply that strategy.
4. Observe the outcome, then report it with reward(experiment_id, event_id, value) using the same event_id the decide returned. That id is what ties the outcome back to the choice.
5. Poll the report and honor its verdict. The statistics are valid however often you look, so checking every turn is fine. When verdict.done is true, stop the experiment — or keep it running deliberately.

A concluded experiment keeps answering decide with the winning arm, so agent code never needs a special case: the experiment quietly becomes a lookup for the best-known option. We call this graduation.

Wire it into a recipeCopy Anchor Link

A recipe node opts in by listing levain-bandit in its @mcp_servers annotation:

# @node {"function": "my_agent.nodes:send_welcome"}
# @mcp_servers ["levain-bandit"]
SendWelcome == TRUE

That gives the node eight tools:

Then tell the agent when and how to use them. A short system_prompt_append in the node code is usually enough:

result = await run(
    prompt,
    mcp_servers=resolved_mcp_servers(config),
    system_prompt_append=(
        "You optimize the welcome-email subject line with the "
        "levain-bandit tools. Reuse the 'welcome-email-subject' "
        "experiment (call list_experiments first; create it only "
        "if missing). Call decide once per email, use the returned "
        "payload's subject verbatim, and save the event_id in "
        "state. Reward 1 when the recipient replies within a day, "
        "0 otherwise."
    ),
)

You can also just describe this in the recipe conversation ("test three subject lines and learn which gets replies") — the builder generates the annotation and the prompt for you.

If experiments aren't available in a run, the tools return guidance instead of failing it — the agent falls back on its own judgment for that turn.

Design the reward firstCopy Anchor Link

Decide what number you'll report — and when you'll observe it — before creating the experiment. Everything else follows from it.

• reward_type — binary for 0/1 outcomes (clicked, converted, replied) or continuous for a number (revenue, minutes, a score). Continuous experiments can declare bounds; values outside them are rejected, and extreme values can be capped.
• The wait window (reward_wait_s, default one hour) — how long a decision waits for its reward before the experiment trains on it. A reward arriving after the window still shows in reports but no longer trains the policy, so report promptly — or widen the window (up to 30 days) when outcomes take time.
• default_reward (default 0) — what a decision counts as when no reward arrives within the window. With a binary reward and default 0, an unobserved outcome is a miss; that's usually what you want. When a decision might never actually be acted on — a recommendation that isn't always shown — set default_reward to an explicit null instead: unrewarded decisions are then left out of the results entirely rather than counted as misses.
• reward_agg — how multiple rewards for one decision combine: keep the first (default) or sum them. Sum fits outcomes that accumulate, like revenue from repeat purchases.

Outcomes that arrive laterCopy Anchor Link

Plenty of outcomes land after the run that made the decision ends: the reply comes tomorrow, the refund window closes next week. The loop still works — reward just needs the original event_id:

1. When the agent decides, it persists the event_id (and what it was for) in recipe state or agent memory.
2. A later run — often a scheduled one — checks which outcomes have landed and calls reward for each saved event_id.

Set reward_wait_s to cover the expected gap, and have the prompt tell the agent to sweep its saved event ids each run.

LifecycleCopy Anchor Link

Move between states with update_status (or POST /api/v1/bandits/{experiment_id}/status): running starts a draft (it needs at least two active arms) or resumes a pause, paused freezes it, stopped concludes it permanently.

The report's verdict — {done, best, probability_best, reason} — says when the experiment has seen enough. An experiment concludes on its own when one arm's probability of being best crosses the stopping threshold (default 0.95) or the decision budget is spent; its status becomes done and it graduates. Either way, concluded experiments keep serving the winner forever.

Reading the reportCopy Anchor Link

get_report returns per-arm pulls, means, credible intervals, the probability each arm is best, and current allocation weights.

Compare arms on ipw_mean, the propensity-corrected mean — an adaptive experiment deliberately sends more traffic to promising arms, which biases raw averages. The raw mean is shown, but ipw_mean is the fair comparison.

While every arm is still collecting its first decisions, the report flags burn_in and traffic stays uniform. Reports refresh within about a minute while an experiment is running; data_as_of says how fresh the numbers are.

Ground rulesCopy Anchor Link

• One experiment per decision. Two experiments drawing on the same traffic interfere with each other's measurements — their effects blur together. Split the traffic or run them in sequence.
• Arm payloads are data, not instructions. Whatever acts on a decision receives the payload verbatim. Agents must treat it as a description of the option, never as directions to follow.
• Reuse experiments. Call list_experiments before creating; recreating an experiment every run throws away its history.
• Decide only for real decisions. Each decide is a data point the experiment learns from (and the surface is rate limited). Speculative calls pollute the record.

Supervising as a humanCopy Anchor Link

Agents run the loop; you create, watch, and conclude from outside. The management surface lives under /api/v1/bandits:

# What's running, and how is it doing?
curl https://api.levainlabs.com/api/v1/bandits \
  -H "Authorization: Bearer $LEVAIN_API_KEY"

curl https://api.levainlabs.com/api/v1/bandits/$EXPERIMENT_ID/report \
  -H "Authorization: Bearer $LEVAIN_API_KEY"

# Conclude it — the winner keeps serving
curl -X POST https://api.levainlabs.com/api/v1/bandits/$EXPERIMENT_ID/status \
  -H "Authorization: Bearer $LEVAIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"status": "stopped"}'

There's deliberately no decide or reward here: decisions must be made by whatever acts on them, or the recorded statistics stop meaning anything. The MCP server mirrors this surface for supervision from your own tools — list_bandit_experiments, get_bandit_report, and update_bandit_status.

See the Bandits API reference for full schemas, including the tuning knobs (config) on experiment creation.

NextCopy Anchor Link

• Recipes — where the node that runs the loop lives.
• Automation — schedule a run to sweep saved event ids and reward late outcomes.
• Metrics — the KPIs an experiment's reward often comes from.