Skip to main content
Trajectory learning learns command sequences from executed trials and measured outcomes. It is a branch of reinforcement learning with adaptive structure (RLAS). Use it for numeric commands that your application can execute, measure, and revise across attempts. Commands can represent process setpoints, staged operations, actuator inputs, or sequence edits through a fixed decoder. Machina uses this same learner in its robot and machine-control pipeline. This guide takes you from a fresh Domain to acquisition and retained execution. It also shows how to add structural comparisons and contextual corrections.

Choose the learning stage

Begin with acquisition. Add later stages when your application needs selection, structural changes, or corrections across starting conditions.

Define the execution contract

Before making API calls, define what a state coordinate, command, and outcome mean. Preserve those definitions throughout acquisition and reuse. The example uses four state coordinates, a two-coordinate goal, and up to eight commands with two values each. Command values use [-1, 1]. The adapter holds each command for half a second and pads shorter sequences with its declared neutral command, [0.0, 0.0]. These are example interface choices. Define the physical meaning of every coordinate and neutral input before execution. For a process task, map numeric controls to permitted setpoints in your adapter. Implement these application hooks: For this example, measure_outcome returns a value in [0, 1]. Use your task’s measured completion, yield, purity, or other declared objective. Keep the actual completion criterion separately when the learning score measures partial progress. The API supplies proposals and learns from observations. Your adapter supplies the environment execution and measurements.

Connect and create a fresh learner

Use a Unit API Key in the server-side ADAPT1_API_KEY environment variable. The production base is https://rei-neuroadapt-api.reilabs.org/api/v1. The following Python helper uses the standard library. Its journal preserves request intent, responses, and measured execution records. It stops on request errors so the caller can reconcile interrupted writes.
Generate and save the prefix once for a fresh run. To resume, reload the same prefix, Domain IDs, and executor settings. Continue the existing learner after reconciling pending work. Configure acquisition with the interface dimensions:
The example allows one outstanding proposal. Finish its observation or cancellation before requesting another exploratory proposal. Set the total attempt budget in your runner.

Choose default or profile settings

Default settings leave the native acquisition defaults in place. Profile settings enable the optional learning switches for proposal edits and retention. Merge this fragment into config before the initial create_stage call when your adapter provides measured per-command outcomes:
Sending step_outcomes alone does not enable progress-guided edits. Configure the relevant switches and return measurements for the commands that actually executed. If you only measure the final outcome, omit step_outcomes and keep progress-related switches disabled. Save the resolved configuration returned by the service. Use the configuration reference for field ranges and deployed-limit guidance.

Run acquisition

Each attempt uses the same Domain. Reset or measure the environment, request a sequence, execute it, and report what actually happened.

Validate proposals and measured traces

For T executed commands, report T action rows and T + 1 state rows. The first state is the measured starting state. Each later state follows its corresponding action. The following helpers enforce the example’s dimensions and command bounds:

Execute and observe an attempt

Implement the adapter hooks above, then use this function for each saved training reset key:
Call acquire_once for one training reset before starting a longer run. Confirm the proposal ID, action shape, observation acknowledgement, and readable learner state. Then pass your application-owned training reset sequence to run_acquisition. The helper sends propose and observe to /domains/{domain_id}/trajectory/{operation}. Always send mode: "explore" during acquisition. Frozen proposals do not provide the same training workflow. Report an early stop using only the commands and states that actually occurred. Include padding only when it executed. Preserve the proposed controls separately when the adapter clips or changes them. If validation fails after proposal, keep its decision ID and trace for recovery. Cancel the decision only when you intentionally abandon it. Cancellation contributes no outcome.

Inspect and reuse acquired behavior

Kept records

Acquisition revises retained executions. The active kept set can remain smaller than configured capacity. Inspect the actual records when choosing parents or preparing a structural stage. Stop acquisition writes and finish pending decisions before exporting. This helper reads all retained pages and checks that the incumbent stays unchanged:
Preserve each record’s native id, actions, initial_state, and initial_valid. An acquisition_reference must identify a kept record. Keep a local mapping from that ID to its reset conditions when revisiting an acquired context.

Execute with learning stopped

Use a reserved evaluation reset and request frozen controls:
This path sends no observation. Keep evaluation outcomes in the application log. Reuse the same decoder, timing, padding, and score definitions.

Add structural comparisons

Use confirmed_execution after acquisition when you want to select or restructure acquired plans. Create a separate Domain for each intervention. This example selects among the exported executions using the same measured quality score:
Each context ID must map to a saved scenario in your application. Reset that scenario before every candidate execution. The comparison envelope uses an exported initial_state and [0.0] as its goal placeholder. The reset scenario supplies the actual task target. Run a nomination comparison, then a confirmation comparison on separate registered contexts. Use a fresh request ID for each. Read trajectory/state afterward and carry its selected_execution into the next stage.
For deletion or ordering, create another stage with the corresponding intervention. Supply the selected execution and its native ID. Carry the selected result forward even when it remains unchanged. Count returned candidates multiplied by comparison contexts before scheduling execution. Include nomination, confirmation, and later refinement in the total budget. See the structural-stage reference for the complete multi-stage handoff.

Add corrections for changing contexts

Use contextual_execution when a retained base needs corrections for different measured starts. Your application preserves the base and defines the context vector. Acquisition can follow a shared lineage across starts. Use separate acquisition learners when you need independent searches. Refinement learns a correction around its supplied base. Collect context vectors from actual training resets. Keep their coordinate order and scale fixed. Use separate scenario sets for representation support, training comparisons, and final evaluation. The example context contains the two target errors and the remaining two state coordinates. Set support_context_ids to your saved support-scenario IDs. Build context vectors from actual resets:
Configure a new refinement Domain around the selected structural execution:
Run each refinement update as follows:
  1. Request propose with mode: "explore", a fresh request_id, state, goal, context_ids, and discovery_context_ids.
  2. Use a representative measured context as state and [0.0] as the proposal-envelope goal.
  3. Reserve some comparison contexts for confirmation, outside discovery_context_ids.
  4. Reset every candidate/context pair and measure its current context vector.
  5. Call evaluate with context, candidate_index, and the proposal’s decision_id.
  6. Add the returned offsets to the preserved base, apply declared bounds, and execute.
  7. Submit one complete observe body containing every candidate/context outcome, including confirmation contexts.
The observe body uses the same decision_id and outcomes structure as structural comparison. Each row contains candidate_index, context_id, and objectives in the configured order. The reset_context helper resets each scenario and measures its context with the same encoding used in support_contexts. For this example, compose corrections as follows:
Run one complete comparison with this function. Choose comparison IDs outside support_context_ids and keep final evaluation IDs separate:
During retained use, call evaluate with {"context": current_context} only. Omit candidate_index and decision_id. Execute corrected_actions(response["offsets"]) and log the outcome without observe. Keep the base, offset scale, context encoding, and command decoder with the refinement Domain ID. The configured radius is an initial setting. It can expand after successful comparisons and shrink after unsuccessful comparisons. Use contextual refinement for the complete comparison loop and optional normalization, context-basis, and response-guidance settings.

Inspect progress and manage the run

An equal-scoring revision can change the retained sequence. Read the outcome and execution record together when interpreting improvement flags. The read-only retest mode does not update a retained score through an observation decision. Use comparison stages with fresh measurements when selecting among noisy executions. Planning can select among live proposals, but each simulation consumes work. Record whether exact terminal scores or learned estimates determined selection. A deeper search using acquisition_reference depends on available kept records. See planning integration and evaluator readiness.

Resume and recover

Resume with the same Domain IDs and unchanged executor contract. Inspect learner state and pending work before continuing. Preserve the acquired base when resuming its contextual correction. trajectory/reset resets learner state. A new environment attempt uses the same learner and does not require this operation. Use retained use and recovery for reconciliation details. Follow retry guidance for backoff. Budget concurrent traffic together and check deployed limits before expanding the run.