Skip to main content
The canonical Spatial Vision endpoint and its compatibility alias are:
Both routes provide spatial yes/no answers and normalized object-relative pointing. Spatial Vision runs directly and requires no language model. Use /vision/spatial for new integrations; /vision/query is the compatibility alias.
Send the question and image directly, with depth when the task uses it. Spatial Vision processes each request independently. With detector and segmenter processing enabled, object hints and scene masks are optional caller inputs.

​
Request contract

Send a required question and one image field:
  • image, image_base64, or image_url: base64 bytes or a data URL;
  • depth or depth_base64: optional depth input in the same representation;
  • object_hints: optional caller-supplied normalized boxes;
  • scene_masks: optional caller-supplied masks;
  • use_detector and use_segmenter: grounding controls.
The compatibility field image_url accepts base64 bytes or a data URL. Remote HTTP URLs are unsupported.

​
Validate grounded output

Returned points and bounding boxes are normalized to [0, 1]. Inspect answer, confidence, points, rationale, and the parsed query fields. For object-relative pointing, verify these detector diagnostics before treating a successful response as grounded:
When segmentation is needed, also inspect vision.segmenter.status and vision.segmenter.mask_count.

​
Read a recorded response

A recorded image-and-depth request asked for points in the empty space to the left of a bowl. The response below preserves the returned coordinates and grounding diagnostics; unrelated fields are omitted.
Recorded pointing response
query identifies the reference object and requested relationship. The points array supplies normalized coordinates for application use. In this response, the detector reports one region and the segmenter reports ten masks; both completed with status: "ok", and depth was used.

Interpret the result

Read support fields and keep degraded results visible to your application.