> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reilabs.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Combine planning with reward learning

> Report an execution once to score its forecast and update delayed-reward learning.

These mechanisms can operate in the same Domain:

* Sequential learning learns which actions lead to better delayed rewards.
* Transition futures learn what measured outcomes can follow an action.
* Probabilistic planning compares those outcomes against the declared objective.

Enabling futures does not convert `sequential_q_mlp` into a distributional model,
nor does it remove the sequential learner's validation requirements.

Here is a complete combined Domain declaration. It uses the same transition
task as [the single-step planner example](https://docs.reilabs.org/docs/neuroadapt/futures/examples.md) and adds episode reward learning:

```json theme={null}
{
  "domain_id": "combined-futures-example",
  "session_id": "my-stable-session",
  "schema": {
    "signals": ["x", "action", "next_x", "step_reward", "terminal"],
    "event_types": ["outcome", "execution"]
  },
  "hypotheses": [
    {"name": "steady-action", "relation": "controls", "policy": "steady", "when": ["choose"]},
    {"name": "wide-action", "relation": "controls", "policy": "wide", "when": ["choose"]}
  ],
  "learning": {
    "enabled": true,
    "context": {"feature_paths": ["values.x"]},
    "reward": {
      "components": [{"field": "values.step_reward", "goal": "maximize", "min": 0, "max": 1, "weight": 1, "required": true}]
    },
    "sequential": {
      "enabled": true,
      "episode_path": "metadata.episode_id",
      "step_path": "metadata.step",
      "next_context_path": "values.next_state",
      "reward_path": "values.step_reward",
      "terminal_path": "values.terminal",
      "discount": 0.95,
      "n_step": 3,
      "minimum_episodes": 8,
      "minimum_validation_skill": 0.25
    },
    "transition": {
      "enabled": true,
      "event_types": ["outcome", "execution"],
      "input_paths": ["values.x", "values.action"],
      "action_path": "values.action",
      "targets": [{"path": "values.next_x", "type": "number"}],
      "required_support": 4,
      "max_samples": 2048,
      "futures": {"enabled": true},
      "planning": {
        "enabled": true,
        "probabilistic": true,
        "actions": ["wide", "steady"],
        "state_bindings": [{"target_path": "values.next_x", "state_path": "values.x"}],
        "objectives": [{"path": "values.x", "direction": "target", "target": 1}],
        "horizon": 1,
        "policy_weight": 0
      }
    }
  }
}
```

Query with `relation: "controls"`, the pre-action `context`, and request
`decision_id`, `transition_plan` and `selection`. Choose deliberately whether
your application follows the planner or the contextual selector. Keep
`policy_weight: 0` initially to test transition planning independently; a positive
value adds the policy learner's first-action utility preference.

For a planner-selected `steady` action, send this to `/feedback` after execution,
replacing the decision ID, episode information and observations with real values:

```json theme={null}
{
  "session_id": "my-stable-session",
  "decision_id": "REPLACE_WITH_RETURNED_DECISION_ID",
  "feedback_kind": "execution",
  "outcome": "confirmed",
  "relation": "controls",
  "policy": "steady",
  "context": {"values": {"x": 0}},
  "values": {
    "x": 0,
    "action": "steady",
    "next_x": 0.9,
    "next_state": {"values": {"x": 0.9}},
    "step_reward": 1,
    "terminal": true
  },
  "metadata": {"episode_id": "episode-001", "step": 0}
}
```

For this example, the reward is terminal: intermediate steps use zero, and the
terminal success uses one. Set `terminal: true` only when the episode actually
ends. `next_state` is the real observed next context; `x` remains the pre-action
input. All action names, relation names and metadata paths must match the Domain.
If the query used a `future_id`, include it in the feedback metadata too.

Inspect `transition_eligibility.futures_update` for forecast resolution and
`learning_state.subsystems.feedback_policy` for policy learning. The two counters
measure different observations and need not match. Do not post the same execution
again to `/events` as an extra sample.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.