Rewards

A reward is one number per run that says how well the run did. Levain never stores it. It stores signals, the facts you and the platform observe, and computes the reward on demand from the agent's objective. A fact that arrives three days late changes the next read; nothing has to be corrected.

SignalsCopy Anchor Link

A signal is a kind, a numeric value and a subject.

The subject decides which run the signal attaches to:

• A run id pins the signal to that run.
• A session id attaches it to the run that had finished most recently when the signal was received. Use this for conversational feedback that arrives between turns.
• An object reference such as zendesk.ticket/42 attaches it to every run that claimed that object. The recipe claims objects as it acts on them, so a business outcome that lands days later still finds the run that caused it.

Record a human rating:

curl -X POST https://api.levainlabs.com/api/v1/agents/$AGENT/signals \
  -H "Authorization: Bearer $LEVAIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"kind": "human.rating", "value": 0.8, "run_id": "'$RUN'", "dedup_key": "msg-19"}'

Repeating a dedup_key records the fact once. occurred_at records when the fact happened; leave it out and it defaults to the time of the call. Signals are appended to a log and become visible to reward reads within about a minute.

From a recipe node, use the SDK:

from levain_sdk.rewards import claim_object, emit_signal

await claim_object("zendesk.ticket/42")
await emit_signal("custom.reply_sent", 1.0)

ObjectivesCopy Anchor Link

The objective maps a run's signals to one number. Every agent starts on the platform default: a failed run scores the lower bound, human ratings and approvals count, and a run with neither is excluded rather than scored. Set your own with PUT /api/v1/agents/{agent_id}/objective:

{
  "definition": {
    "horizon_s": 172800,
    "default": "exclude",
    "guards": [{"signal": "run.succeeded", "op": "lt", "value": 1}],
    "terms": [
      {"signal": "human.rating", "weight": 0.5, "scale": [1, 5]},
      {"signal": "external.ticket_closed", "weight": 0.5, "within_s": 86400, "missing": 0}
    ],
    "bounds": [0, 1]
  },
  "note": "Ratings come from the 1-5 widget; a ticket counts if closed within a day."
}

• horizon_s is how long after a run finishes its reward stays open. Signals received later are reported as late and never used.
• guards are hard constraints. One matching signal floors the run at the lower bound, whatever the terms say.
• terms are weighted components. agg collapses several signals of one kind (last by default, or first, sum, max, min, mean, count). scale maps a raw range onto [0, 1]. within_s only counts facts that occurred within that many seconds of the run's end. missing says what to do when the kind never shows up: skip the term, or impute a value.
• default applies when no evidence term produced a signal. "exclude" leaves the run out of averages; a number scores it. A term marked "evidence": false (a cost penalty, say) never counts as evidence on its own.

Objectives are versioned. A new version applies to every run, past and future, because rewards are computed on read.

Reading rewardsCopy Anchor Link

# One run
curl https://api.levainlabs.com/api/v1/agents/$AGENT/runs/$RUN/reward \
  -H "Authorization: Bearer $LEVAIN_API_KEY"

# Every run of a version
curl https://api.levainlabs.com/api/v1/agents/$AGENT/versions/3/rewards \
  -H "Authorization: Bearer $LEVAIN_API_KEY"

A run's reward carries status (scored, guarded or excluded), value, whether it has matured past the horizon, the per-term breakdown and the count of late signals. The version summary averages matured runs and lists the lowest-scoring ones first, which is the place to look when deciding what to fix next.