Learn from feedback and outcomes

Bind each observed outcome to the decision that produced it, then query again so later results can reflect the update.

Default path: query a Domain, execute the selected action or policy, and submit the observed outcome as feedback. Ordinary one-decision feedback requires no sequential configuration or learner tuning.A successful feedback write means the record was accepted or stored. It does not by itself prove that feedback_policy admitted a learning observation. When validating an integration, inspect credit_assignment.contextual_learning_applied and the feedback_policy sample count.

For specialized evidence layouts, see Domain learning patterns.

Default feedback loop

observe state -> commit decision -> execute exact action -> record transition -> bind consequence/feedback -> query again

Feedback is part of a continuing reasoning-and-learning loop. It is not a separate training job. Generated language is not required.

Keep explicit feedback, observable consequences, and application-only context separate. An observed transition can supply transition evidence without a success label. An observable consequence is not an application judgment or a feedback write.

Transition evidence and feedback

Information	Where it belongs
Observations available before the consequence and the proposed intervention	Configured transition learner input
Observable consequence at the declared event boundary	Transition target or ordinary Domain event, as specified by the Domain contract
Hidden rule, correct action, solver output, or application metric	Application-only context; never a transition input or observation target
Feedback sent to change a scoped policy or relation	`POST /api/v1/domains/{domain_id}/feedback`

A transition target describes the observable consequence. It does not state if the consequence was good. Feedback can later assign application value to a committed decision. Record feedback as a separate operation. See Design Domains for transition learning for the separation between the operational record, learner view, and application-only context.

Decision binding

Keep an application interaction record beside the API requests:

{
  "run_id": "run1",
  "trial_id": "run1-trial-004",
  "relation": "in_front_of",
  "policy": "footprint_edge_projection",
  "selected_action": "move_up",
  "executed_action": "move_up",
  "before_state_id": "state-003",
  "after_state_id": "state-004",
  "native_outcome": "level_advanced",
  "outcome_visible_to_adapt_1": "positive"
}

This ledger belongs to the surrounding application and keeps the full reasoning-to-action loop inspectable. Submit only the fields accepted by the selected API route. A policy is the context-dependent rule or preference being updated. An application can use it to select an external action, forward a returned action identifier directly, or apply its own commitment rule; preserve the returned policy or selection beside the exact executed action. If a native outcome such as advancement is declared to mean progress, treat it as positive evidence for the bound transition. If only application-private logic assigns that value, keep the judgment outside Adapt-1 unless the integration explicitly returns it as feedback.

Feedback horizon

Feedback updates support with respect to the outcome signal it encodes. Immediate outcomes can strengthen locally useful behavior, while terminal or delayed outcomes let Adapt-1 organize that learning around a longer objective. For goal-driven tasks, record which outcome horizon Adapt-1 receives:

local state improvement;
binary step success;
reward magnitude;
terminal goal completion;
delayed episode return; or
only the next observable consequence.

Ordinary decision-bound feedback is the default. Do not enable sequential learning merely because records arrive in order.

Choose only the relationship the task needs. These patterns can coexist inside one Domain or remain isolated in separate Domains. Configure more than one only when the application needs more than one relationship.

What the task needs to learn	Use	Configuration
Which policy works in the current context, followed by an immediate measured outcome	Ordinary feedback	Set `learning.enabled: true`. Leave sequential and bounded transition learning disabled when this is the only relationship.
A structured observable target from configured inputs, with an action included only when the relationship requires one	Structured transition learning	Set `learning.transition.enabled: true`, declare input and target paths, and send eligible events. Do not enable sequential learning merely because events are ordered.
How a candidate action changes an explicit bounded public state relative to a numeric objective	Bounded transition scoring	Set `learning.sequential.bound_transition.enabled: true` and provide the state, action, and objective mappings.
Which action improves a delayed or terminal outcome by changing later state	Sequential learning	Set `learning.sequential.enabled: true` and send the ordered transition fields described below.

An environment can contain long episodes and still use ordinary feedback. If every decision receives its own immediate outcome and the action does not determine the later state, episode length alone is not a reason to enable sequential learning.

Sequential feedback for delayed outcomes

The sequential learning guide documents the complete contract, configuration surface, diagnostics, tuning profiles, and frozen evaluation protocol.Enable this path only when the application has a real episode boundary, ordered state transitions, and delayed credit across actions. Independent samples, immediate-only labels, and chronological records whose actions do not affect later state should remain on ordinary feedback.Enable both learning switches in the Domain:

{
  "learning": {
    "enabled": true,
    "sequential": {
      "enabled": true
    }
  }
}

Then repeat this loop:

query current state -> execute one policy -> return the observed next state and outcome -> query the new state

With the default mappings, each sequential feedback item supplies:

Field	Request path
Episode identifier	`metadata.episode_id`
Ordered step	`metadata.step`
Observed next state	`values.next_state`
Sequential reward	`values.step_reward`
Episode boundary	`values.terminal`

All five fields are required for the transition to enter sequential learning. Use a JSON number for step and step_reward, an object for next_state, and a Boolean for terminal.Policy admission also requires the executed relation and policy. Current runtime behavior requires the relation field to be present before contextual policy learning is applied. A feedback write can still be accepted and stored when relation is absent, so do not use HTTP success as proof that policy learning occurred.Decision-time context can be bound through any of these supported routes:

decision_id from a prior Domain query (context_source: "decision");
target_memory_id for a retained memory that supplies the relevant context (context_source: "memory");
explicit structured context in the feedback request (context_source: "request").

decision_id is the strongest sealed attribution route, but it is not required for policy admission when another supported context source is supplied.Send both reward fields when the same feedback participates in ordinary and sequential learning:

{
  "session_id": "agent-17",
  "outcome": "success",
  "feedback_kind": "execution",
  "decision_id": "DECISION_FROM_QUERY",
  "relation": "controls",
  "policy": "safe_controller",
  "values": {
    "reward": 0.8,
    "step_reward": 0.8,
    "next_state": {
      "load": 0.71,
      "temperature": 0.43
    },
    "terminal": false
  },
  "metadata": {
    "episode_id": "episode-0007",
    "step": 38
  }
}

values.reward supplies the measured outcome for immediate contextual learning. values.step_reward supplies the ordered reward used for delayed return. They may contain the same value when that matches the task.Keep the same Domain and learner continuity across a trajectory, but query each step with the newly observed state. Use a new episode ID at the real episode boundary, restart step from zero, and set terminal: true on the final transition. Reset the Domain only for an intentional cold-start condition or deliberate replay.

Sequential settings that may need adjustment

Use the defaults unless the task contract requires one of these changes:

Setting	Change it when
`learning.context.max_samples`	Long episodes would otherwise be discarded before enough complete trajectories are retained. Size it above `episode length × episodes retained`.
`learning.credit_assignment.neutral_reward`	The environment’s neutral value differs from the default `0.5`. Use `0.0` when intermediate zero means that reward has not arrived yet.
`learning.sequential.discount`	The real outcome horizon is substantially longer or shorter than the default horizon.
Sequential field paths	The application does not use the default payload paths shown above.

Leave training parameters, validation thresholds, model weights, and n-step settings at their defaults for ordinary use. Adapt-1 validates learned estimates before they affect selection.Request learning_state and policy_diagnostics when verifying the integration. A sequential estimate may not appear immediately because the learner first needs enough completed episodes and held-out validation. Inspect the returned model status and reason, then confirm that sequential_expected_reward is present before treating delayed estimates as part of selection.

Bounded transition scoring

Bounded transition scoring is separate from both delayed-return learning and the general structured-transition learner. learning.transition predicts declared observable targets from event inputs. Bounded transition scoring instead learns action-conditioned changes in an explicit public state and projects candidate policies onto a declared numeric objective during selection. Its configuration is nested at learning.sequential.bound_transition because it consumes state transitions, but it does not turn every feedback task into a sequential-return task. Use it only when:

the relevant state is public and numerically bounded;
each candidate identifies the state and action it affects;
feedback returns the observed successor state; and
one declared state feature is the objective.

Do not use it for hidden state, an unbounded or undefined objective, or an ordinary context-to-outcome decision. The required mapping is intentionally small:

Mapping	Function
`state_selector_feature`	Selects the current state object from candidate policy features
`state_features`	Maps logical state names to public fields under that object
`objective_feature`	Identifies the numeric state feature to optimize
`objective_min` and `objective_max`	Define the public normalization bounds
`transition_values`	Identify state-changing candidate actions
`terminal_values`	Identify terminal candidate actions

Add state_bounds only for real public availability or validity limits. Add action selectors, action features, or signature features only when the action representation requires them. Candidate policies must carry the selector and category values used by the mapping, and feedback must identify the action that actually executed and its observed successor state. Leave minimum_support, model_weight, novelty_bonus, and transition_cost at their defaults unless the task contract gives a concrete reason to change them. A wrong state, action, or objective binding can materially distort selection.

Policy-learning admission

POST /api/v1/domains/{domain_id}/feedback has two distinct outcomes that clients must not conflate:

feedback accepted/stored
        ↓
policy-learning eligibility checked
        ↓
feedback_policy updated or skipped

A successful HTTP response confirms the first step. It does not guarantee the third. For ordinary contextual policy learning, send:

the actual executed policy;
a relation field;
a measured reward or recognized outcome;
one supported source of decision-time context: decision_id, target_memory_id, or explicit structured context.

Current runtime admission checks require relation to be present. The request validator can accept feedback without it, but that record can be stored without increasing learning_state.subsystems.feedback_policy.sample_count. The admission check currently tests presence rather than validating the relation value against the bound decision. Always send the real relation used by the Domain; do not use placeholder values. The context source is reported under credit_assignment.context_source when available:

Input	Reported context source
Valid `decision_id`	`decision`
Valid `target_memory_id`	`memory`
Explicit structured `context`	`request`

These alternate bindings allow policy feedback to be admitted without a prior Domain query. They do not replace /query for Adapt-1 policy selection, exploration, or sealed decision attribution. When decision_id is supplied, treat its sealed context as authoritative; do not expect an additional request context to override that snapshot. To verify that the learner actually consumed the feedback, inspect:

credit_assignment.contextual_learning_applied
learning_state.subsystems.feedback_policy.sample_count

contextual_learning_applied is nested under credit_assignment in current responses. Older integrations that read only a top-level field can miss a stored-but-not-applied feedback record. For the current feedback validator, use feedback_kind: "execution" or feedback_kind: "semantic". Do not send top-level sequential, decision_context, or run_id; those fields are rejected. When supplying decision-time state directly, use the accepted context field.

Feedback-driven policy scoring

For the currently documented relation/policy workflow:

Step	Route
Create a domain	`POST /api/v1/domains`
Submit the outcome	`POST /api/v1/domains/{domain_id}/feedback`
Read the updated learned signal	`POST /api/v1/domains/{domain_id}/query`

Store the relevant relation, policy, and application run_id as structured provenance when the API contract calls for them. In feedback requests, relation and policy are top-level learner fields. run_id is not a top-level feedback field in the current validator; keep it in the surrounding application ledger or accepted metadata instead. Keep the relation, executed policy, and relevant decision-time context aligned between the query and its observed feedback. Keep trial, action, transition, and outcome provenance in the surrounding ledger. Do not hide the scoping values only inside free text.

{
  "relation": "in_front_of",
  "policy": "footprint_edge_projection",
  "metadata": {
    "run_id": "run1"
  }
}

Sealed-decision feedback

When a Domain query returns a decision_id, persist it when you need sealed attribution to that exact result. It is the preferred route for query → action → outcome workflows, but it is not the only policy-learning context route. Memory-bound feedback can use target_memory_id, and callers with the original structured state can supply explicit context. Send decision-bound feedback to: POST https://rei-neuroadapt-api.reilabs.org/api/v1/domains/{domain_id}/feedback

{
  "session_id": "<session-id>",
  "decision_id": "<decision-id>",
  "feedback_kind": "execution",
  "outcome": "success",
  "relation": "causes",
  "policy": "inspect_feed_path",
  "values": { "reward": 0.9 }
}

Use feedback_kind: "execution" for policy-outcome quality and feedback_kind: "semantic" when the feedback evaluates whether a hypothesis is applicable or supported. Without a declared reward contract, numeric feedback is read in this order: values.reward or metadata.reward, score, utility, Boolean correct/success/accepted, then the outcome label. error_distance is not a reward unless the Domain declares its normalization and minimization. Accepted or stored feedback supplies evidence; it does not guarantee that a learner was applied or that the same policy will control a later selection. Inspect policy_update, eligibility, attribution, and credit_assignment when returned. In current responses, credit_assignment.contextual_learning_applied is the direct diagnostic for whether the contextual policy path was applied. Confirm the effect by checking that learning_state.subsystems.feedback_policy.sample_count advances on a later query. For competing policies, set allow_exploration: true before a non-default selection_mode can apply. Supported modes are exploit, ucb, thompson, and auto; these affect Domain policy selection, not memory retrieval.

Defaults are sufficient for normal use. Use Domain exploration_mode or exploration_strength only when the application intentionally needs different explore/exploit behavior. Use transfer_strength or action_transfer_strength only when evidence should share more broadly across related contexts or actions, or remain more strictly isolated.

Adaptive result

For this workflow, compare the response-level policy_scores entry for the relation/policy that received positive feedback against the one that received negative feedback. Use an unrelated relation as a control so that a scoped update is not mistaken for global behavior. For this route, policy_scores is the documented learned signal. supporting_memories, core_support, and ranked-hypothesis fields add context and make the result easier to inspect.

Other adaptation surfaces

policy_scores is one public adaptation surface. Adapt-1 also exposes other adaptive results; the documented interval-policy workflow uses /adapt/* and returns prediction.<output_key>.

Get started

Build

Advanced Domain design

Operate

Modalities

Domains

Interval policies

State

System

Spatial Vision API

Default feedback loop

Transition evidence and feedback

Decision binding

Feedback horizon

Sequential settings that may need adjustment

Bounded transition scoring

Policy-learning admission

Feedback-driven policy scoring

Sealed-decision feedback

Adaptive result

Other adaptation surfaces

​Default feedback loop

​Transition evidence and feedback

​Decision binding

​Feedback horizon

​Sequential settings that may need adjustment

​Bounded transition scoring

​Policy-learning admission

​Feedback-driven policy scoring

​Sealed-decision feedback

​Adaptive result

​Other adaptation surfaces

Default feedback loop

Transition evidence and feedback

Decision binding

Feedback horizon

Sequential settings that may need adjustment

Bounded transition scoring

Policy-learning admission

Feedback-driven policy scoring

Sealed-decision feedback

Adaptive result

Other adaptation surfaces