Learn from feedback and outcomes

Bind each observed outcome to the decision that produced it, then query again so later results can reflect the update.

Declare policy choices

For a Domain query to select among policies, declare each selectable option in the top-level hypotheses array when creating the Domain. Give competing options the same relation and each option a unique, non-null policy identifier. Include this fragment in the Domain-create request alongside its domain_id, session_id, and schema. Declare controls in schema.relations and enable learning.enabled for feedback learning. Replace the example options with the application’s available choices.

Domain-create fragment

{
  "hypotheses": [
    {
      "name": "Safe controller",
      "relation": "controls",
      "policy": "safe_controller",
      "predicts": ["execute safe_controller"],
      "weight": 1.0
    },
    {
      "name": "Alternative controller",
      "relation": "controls",
      "policy": "alternative_controller",
      "predicts": ["execute alternative_controller"],
      "weight": 1.0
    }
  ]
}

Use the same relation and policy identifiers when binding execution feedback. These entries declare the available choices; measured outcomes teach their contextual value. Ordinary decision-bound feedback works with sequential learning disabled.

Start with one decision-bound outcome

This example assumes a Domain with the policy choices declared above and a completed query that returned a decision_id. Use the actual relation and policy attached to that decision. Replace the example values with the outcome of your execution.

Preserve the decision and execute it

Save the returned decision_id, decision-time context, relation, and policy. Validate and execute the selected operation in your application, then measure its outcome. Record any application override separately; do not credit an unexecuted policy.

Return the measured outcome

Save the following as feedback.json. Replace the decision, relation, policy, outcome, and reward with the values from your actual interaction. The numeric reward shown is illustrative and must match your Domain’s outcome scale.

feedback.json

{
  "session_id": "ignored",
  "decision_id": "DECISION_ID_FROM_QUERY",
  "feedback_kind": "execution",
  "relation": "controls",
  "policy": "safe_controller",
  "outcome": "success",
  "values": { "reward": 0.8 }
}

Set ADAPT1_DOMAIN_ID to that decision’s Domain and send the body:

Send feedback

curl -fsS --max-time 120 \
  "https://rei-neuroadapt-api.reilabs.org/api/v1/domains/${ADAPT1_DOMAIN_ID}/feedback" \
  -H "Authorization: Bearer $ADAPT1_API_KEY" \
  -H "Content-Type: application/json" \
  --data-binary @feedback.json

Check admission, then query again

Inspect credit_assignment.contextual_learning_applied in the feedback response. On the next Domain query, request the relevant policy_scores and learning_state; confirm the intended feedback_policy sample count and scoped policy entry. Keep the subsequent application decision separately from the learned score.If feedback reports credit_assignment.contextual_learning_applied: true but queries abstain with reason: "no_evidence", first check that the Domain declares selectable hypotheses for the queried relation, with policy identifiers matching the feedback. The flag confirms learner admission; it does not confirm that the query has selectable candidates. Continue inspecting the query’s support diagnostics if the declarations are present.Inspect the response when admission diagnostics are missing or report that learning was skipped. Reconcile a timed-out write before resubmitting, because it may have been applied.

For a complete program that starts by creating its own Domain, use First learned result. That tutorial learns an observable target from events. This guide covers the different case where an executed decision receives a measured utility signal.

Feedback contract

Default path: query a Domain, execute the selected action or policy, and submit the observed outcome as feedback. Ordinary one-decision feedback requires no sequential configuration or learner tuning.After the feedback write succeeds, inspect credit_assignment.contextual_learning_applied and the feedback_policy sample count to confirm learner admission.

For specialized evidence layouts, see Domain learning patterns.

Default feedback loop

observe state → commit decision → execute exact action → record transition → bind consequence/feedback → query again Adapt-1 uses eligible feedback to update the applicable learned state between decisions. Applications can consume its structured results directly.

Record the observed transition as evidence under the transition contract. Submit the application’s utility judgment through the feedback route.

Transition evidence and feedback

Information	Where it belongs
Observations available before the consequence and the proposed intervention	Transition input under the discovered or explicitly declared projection
Observable consequence at the declared event boundary	Transition target or ordinary Domain event, as specified by the Domain contract
Observed or corrected structured target available after prediction	Eligible target-learning event under the declared contract; preserve its source and timing
Utility assigned to the executed choice	Policy feedback; keep its meaning separate from the target being predicted
Hidden evaluator state, withheld evaluation answers, or solver internals outside the learning contract	Application-only context; exclude from learner inputs and targets
Feedback sent to change a scoped policy or relation	`POST /api/v1/domains/{domain_id}/feedback`

A transition target describes the observable consequence or structured result the learner is configured to predict. Keep it out of the inputs used to request that prediction. During a declared learning phase, completed measurements and reference corrections can supply targets afterward. Evaluation answers remain withheld during frozen evaluation. Submit feedback separately to assign application value to the committed decision. For example, a corrected point can teach a coordinate predictor while the measured success of the chosen procedure updates policy selection. Follow the structured scene setup when the application needs both relationships. See Design Domains for transition learning for the separation between the operational record, learner view, and application-only context.

Decision binding

Keep an application interaction record beside the API requests:

{
  "run_id": "run1",
  "trial_id": "run1-trial-004",
  "relation": "in_front_of",
  "policy": "footprint_edge_projection",
  "selected_action": "move_up",
  "executed_action": "move_up",
  "before_state_id": "state-003",
  "after_state_id": "state-004",
  "native_outcome": "level_advanced",
  "outcome_visible_to_adapt_1": "positive"
}

This ledger belongs to the surrounding application and keeps the full reasoning-to-action loop inspectable. Submit only the fields accepted by the selected API route. A policy is the context-dependent rule or preference being updated. An application can use it to select an external action, forward a returned action identifier directly, or apply its own commitment rule; preserve the returned policy or selection beside the exact executed action. When the Domain declares a native outcome such as advancement to mean progress, submit it through the applicable outcome or feedback contract for the executed decision. Return application-defined utility explicitly as feedback.

Feedback horizon

Feedback updates support with respect to the outcome signal it encodes. Contextual feedback updates the bound policy in its decision-time context. Sequential learning uses later rewards to revise earlier actions that changed subsequent states. For goal-driven tasks, record which outcome horizon Adapt-1 receives:

local state improvement;
binary step success;
reward magnitude;
terminal goal completion;
delayed episode return; or
only the next observable consequence.

Use ordinary decision-bound feedback when each choice receives its own attributable outcome.

Choose only the relationship the task needs. These patterns can coexist inside one Domain or remain isolated in separate Domains. Configure more than one only when the application needs more than one relationship.

What the task needs to learn	Use	Configuration
Which policy works in the current context, using an outcome attributed to that decision	Ordinary feedback	Set `learning.enabled: true`. Leave sequential and bounded transition learning disabled when this is the only relationship.
A structured observable target from eligible inputs, with an action included only when the relationship requires one	Structured transition learning	Set `learning.transition.enabled: true` and declare eligible event types and targets. Discover input paths with `autonomous_projection.enabled: true` and empty `input_paths`, or supply explicit paths. Send eligible completed events.
How a candidate action changes an explicit bounded public state relative to a numeric objective	Bounded transition scoring	Set `learning.sequential.bound_transition.enabled: true` and provide the state, action, and objective mappings.
Which action improves a delayed or terminal outcome by changing later state	Sequential learning	Set `learning.sequential.enabled: true` and send the ordered transition fields described below.

Use ordinary feedback for decisions with sufficient attributable outcomes, including decisions collected within long episodes.

When the application computes the return

An application can wait for several measured outcomes, compute a return for an earlier executed decision, and submit it as contextual feedback under the Domain’s reward contract. Preserve the contributing outcomes, return horizon, discount if used, reward scale, and terminal handling alongside the original decision binding. The decision-time context must still contain only information available when the action was chosen. Later outcomes belong in feedback and the application log. Here the application performs attribution across time. Configure sequential learning when Adapt-1 should learn delayed value from ordered transitions. An aggregated contextual reward alone does not enable the sequential learner. If both paths are active, keep the contextual return and sequential per-step reward meanings explicit.

Sequential feedback for delayed outcomes

The sequential learning guide documents the complete contract, configuration surface, diagnostics, tuning profiles, and frozen evaluation protocol.Enable this path only when the application has a real episode boundary, ordered state transitions, and delayed credit across actions. Independent samples, immediate-only labels, and chronological records whose actions do not affect later state should remain on ordinary feedback.Enable both learning switches in the Domain:

{
  "learning": {
    "enabled": true,
    "sequential": {
      "enabled": true
    }
  }
}

Then repeat this loop:

query current state -> execute one policy -> return the observed next state and outcome -> query the new state

With the default mappings, each sequential feedback item supplies:

Field	Request path
Episode identifier	`metadata.episode_id`
Ordered step	`metadata.step`
Observed next state	`values.next_state`
Sequential reward	`values.step_reward`
Episode boundary	`values.terminal`

Each field listed above is required for the transition to enter sequential learning. Use a JSON number for step and step_reward, an object for next_state, and a Boolean for terminal.Policy admission also requires the executed relation and policy. The current runtime can store feedback with a missing relation while skipping contextual policy learning.Decision-time context can be bound through any of these supported routes:

decision_id from a prior Domain query (context_source: "decision");
target_memory_id for a retained memory that supplies the relevant context (context_source: "memory");
explicit structured context in the feedback request (context_source: "request").

Use decision_id for attribution to the query’s sealed context. Memory-bound and explicit request context are also supported for policy admission.Send both reward fields when the same feedback participates in ordinary and sequential learning:

{
  "session_id": "ignored",
  "outcome": "success",
  "feedback_kind": "execution",
  "decision_id": "DECISION_FROM_QUERY",
  "relation": "controls",
  "policy": "safe_controller",
  "values": {
    "reward": 0.8,
    "step_reward": 0.8,
    "next_state": {
      "values": {
        "load": 0.71,
        "temperature": 0.43
      }
    },
    "terminal": false
  },
  "metadata": {
    "episode_id": "episode-0007",
    "step": 38
  }
}

Use the same context shape for the current observation and next_state; this example places state fields under values.values.reward supplies the attributed outcome for contextual learning. values.step_reward supplies the ordered reward used for delayed return. They may contain the same value when that matches the task.Keep the same Domain across a trajectory and query each step with the newly observed state. Use a new episode ID at the real episode boundary, restart step from zero, and set terminal: true on the final transition. Reset the Domain only for an intentional cold-start condition or deliberate replay.

Sequential settings that may need adjustment

Use the defaults unless the task contract requires one of these changes:

Setting	Change it when
`learning.context.max_samples`	Long episodes would otherwise be discarded before enough complete trajectories are retained. Size it above `episode length × episodes retained`.
`learning.credit_assignment.neutral_reward`	The environment’s neutral value differs from the default `0.5`. Use `0.0` when intermediate zero means that reward has not arrived yet.
`learning.sequential.discount`	The real outcome horizon is substantially longer or shorter than the default horizon.
Sequential field paths	The application does not use the default payload paths shown above.

Leave training parameters, validation thresholds, model weights, and n-step settings at their defaults for ordinary use. Adapt-1 validates learned estimates before they affect selection.Request learning_state and policy_diagnostics when verifying the integration. A sequential estimate may not appear immediately because the learner first needs enough completed episodes and held-out validation. Inspect the returned model status and reason, then confirm that sequential_expected_reward is present before treating delayed estimates as part of selection.

Additional feedback patterns

Bounded transition scoring

Bounded transition scoring learns action-conditioned changes in explicit public state and scores candidate policies against a declared numeric objective during selection. Configure it at learning.sequential.bound_transition. The general learning.transition learner predicts declared observable targets from event inputs; delayed-return learning uses learning.sequential.enabled and its ordered transition contract.Use it only when:

the relevant state is public and numerically bounded;
each candidate identifies the state and action it affects;
feedback returns the observed successor state; and
one declared state feature is the objective.

Do not use it for hidden state, an unbounded or undefined objective, or an ordinary context-to-outcome decision.Configure these mappings:

Mapping	Function
`state_selector_feature`	Selects the current state object from candidate policy features
`state_features`	Maps logical state names to public fields under that object
`objective_feature`	Identifies the numeric state feature to optimize
`objective_min` and `objective_max`	Define the public normalization bounds
`transition_values`	Identify state-changing candidate actions
`terminal_values`	Identify terminal candidate actions

Add state_bounds only for real public availability or validity limits. Add action selectors, action features, or signature features only when the action representation requires them. Candidate policies must carry the selector and category values used by the mapping, and feedback must identify the action that actually executed and its observed successor state.Leave minimum_support, model_weight, novelty_bonus, and transition_cost at their defaults unless the task contract gives a concrete reason to change them. A wrong state, action, or objective binding can materially distort selection.

Policy-learning admission

POST /api/v1/domains/{domain_id}/feedback checks learning eligibility after accepting the feedback record:

feedback accepted/stored
        ↓
policy-learning eligibility checked
        ↓
feedback_policy updated or skipped

Inspect the learner-admission diagnostic to determine whether feedback_policy updated. For ordinary contextual policy learning, send:

the actual executed policy;
a relation field;
a measured reward or recognized outcome;
one supported source of decision-time context: decision_id, target_memory_id, or explicit structured context.

The current admission check tests whether relation is present. Always send the real relation used by the Domain. The request validator can accept feedback with a missing relation and store the record while leaving learning_state.subsystems.feedback_policy.sample_count unchanged. The context source is reported under credit_assignment.context_source when available:

Input	Reported context source
Valid `decision_id`	`decision`
Valid `target_memory_id`	`memory`
Explicit structured `context`	`request`

These bindings support policy feedback from memory or application-supplied context. Call /query when Adapt-1 should select a policy, explore alternatives, or seal a decision for later attribution. When a request includes decision_id, its sealed context takes precedence. To verify that the learner actually consumed the feedback, inspect:

credit_assignment.contextual_learning_applied
learning_state.subsystems.feedback_policy.sample_count

contextual_learning_applied is nested under credit_assignment in current responses. Integrations that read only a top-level field can miss the learner-admission status of a stored feedback record. context_source: decision identifies where context was resolved. Verify that the submitted relation and policy match the executed decision even when contextual_learning_applied is true. Confirm the effect in the intended scoped policy entry and feedback_policy state. At the retained-sample limit, its count can stay flat while the learner version or scoped statistics change. For the current feedback validator, use feedback_kind: "execution" or feedback_kind: "semantic". Do not send top-level sequential, decision_context, or run_id; those fields are rejected. When supplying decision-time state directly, use the accepted context field.

Feedback-driven policy scoring

For the currently documented relation/policy workflow:

Step	Route
Create a domain	`POST /api/v1/domains`
Submit the outcome	`POST /api/v1/domains/{domain_id}/feedback`
Read the updated learned signal	`POST /api/v1/domains/{domain_id}/query`

In feedback requests, relation and policy are top-level learner fields. Align them with the executed policy and the decision-time context from the query. Record run_id, trial, action, transition, and outcome provenance in the application ledger, using accepted metadata where the API contract permits it.

{
  "relation": "in_front_of",
  "policy": "footprint_edge_projection",
  "metadata": {
    "run_id": "run1"
  }
}

Sealed-decision feedback

Persist the decision_id returned by a Domain query to attribute feedback to that exact result. For feedback bound to retained memory, use target_memory_id; for application-supplied decision-time state, use structured context. Send decision-bound feedback to: POST https://rei-neuroadapt-api.reilabs.org/api/v1/domains/{domain_id}/feedback

{
  "session_id": "ignored",
  "decision_id": "<decision-id>",
  "feedback_kind": "execution",
  "outcome": "success",
  "relation": "causes",
  "policy": "inspect_feed_path",
  "values": { "reward": 0.9 }
}

Use feedback_kind: "execution" for policy-outcome quality and feedback_kind: "semantic" when the feedback evaluates whether a hypothesis is applicable or supported. Without a declared reward contract, numeric feedback is read in this order: values.reward or metadata.reward, score, utility, Boolean correct/success/accepted, then the outcome label. To use error_distance as reward, declare its normalization and minimization in the Domain. Inspect policy_update, eligibility, attribution, and credit_assignment when returned to determine which learner consumed the feedback. In current responses, credit_assignment.contextual_learning_applied is the direct diagnostic for whether the contextual policy path was applied. Confirm the effect on a later query through the scoped policy entry and feedback_policy sample count or learner version, allowing for the retained-sample limit. For competing policies, set allow_exploration: true before a non-default selection_mode can apply. Supported modes are exploit, ucb, thompson, and auto. They configure Domain policy selection.

Defaults are sufficient for normal use. Use Domain exploration_mode or exploration_strength only when the application intentionally needs different explore/exploit behavior. Use transfer_strength or action_transfer_strength only when evidence should share more broadly across related contexts or actions, or remain more strictly isolated.

Adaptive result

Compare the response-level policy_scores entry for the relation/policy that received positive feedback against the one that received negative feedback. Compare an unrelated relation as a control for the update’s scope.supporting_memories, core_support, and ranked-hypothesis fields add context and make the result easier to inspect.

Other adaptive results

The interval-policy workflow uses /adapt/* and returns its learned result at prediction.<output_key>.

Get started

Concepts

Guides

Planning

Discovery

Operations

API reference

Declare policy choices

Start with one decision-bound outcome

Feedback contract

Default feedback loop

Transition evidence and feedback

Decision binding

Feedback horizon

When the application computes the return

Sequential settings that may need adjustment

Additional feedback patterns

Policy-learning admission

Sealed-decision feedback

​Declare policy choices

​Start with one decision-bound outcome

​Feedback contract

​Default feedback loop

​Transition evidence and feedback

​Decision binding

​Feedback horizon

​When the application computes the return

​Sequential settings that may need adjustment

​Additional feedback patterns

​Policy-learning admission

​Sealed-decision feedback

Declare policy choices

Start with one decision-bound outcome

Feedback contract

Default feedback loop

Transition evidence and feedback

Decision binding

Feedback horizon

When the application computes the return

Sequential settings that may need adjustment

Additional feedback patterns

Policy-learning admission

Sealed-decision feedback