vla

Learned visuomotor policies (VLAs) run as bounded executions, and start: the first motion, choosing whether to hand the task to one.

• Layer: learned
• Needs: policy
• Requires: –
• Conflicts with: –
• Never dropped: no
• Mixin: skills.vla:LearnedPolicies
• Manifest: skills/vla/skill.toml

Functions

start

def start(strategy, kind, policy=None)

Your first motion, once (Z2’s first call): the strategy for this task, from the learned policy’s description (list_policies()), and the line of it the task matches (kind, in your words). “A”: hand the whole task to the policy and supervise; “C”: the policy first, then finish what it leaves; “B”: your own tools only. A and C run the policy at once, as run_policy(policy) from the untouched start pose with the task’s own instruction and both arms, and return its trace; B returns None. Before start nothing that moves the arms runs; looking does (get_observation, locate, segment_text, point_to, watch_scene). Called again with the same strategy (a program run again from its top) it does nothing. After B, A or C hands the task over after all (the policy’s fresh run needs the start pose: home() first); other changes are refused.

run_policy

def run_policy(name, instruction=None, arms=ARMS, max_seconds=None, region=None, until=None, reset=False, stall_seconds=STALL_SECONDS, from_anywhere=False)

Run a learned policy (VLA) as a bounded execution, the way RoboDojo’s own evaluation client runs it: every tick it gets RoboDojo’s own observation (the task’s instruction unless instruction is given), each action chunk is executed whole, and the policy keeps its state from one call to the next with the same instruction and arms (reset=True starts it afresh). It commands the listed arms; the other arm holds.

A fresh run (the first, another instruction or arms, or reset=True) starts only from the start pose the policy’s demonstrations start from: elsewhere it ends at once with SKILL_ERROR (detail reason “not at the start pose”; home() first). from_anywhere=True lifts that: the direct-control rung R1 hands the policy sub-tasks wherever the arms are, as fixed-vocabulary orchestrators do.

Without max_seconds it runs until the episode ends, until(obs) holds, the arms stop moving (after they moved in this run, no joint of its arms moves more than 0.02 rad for stall_seconds; COMPLETED with detail stopped = “stalled”; None never stops for that; not while the progress watch measures tracked objects), the task stops progressing, or another event stops it. With the agent’s runtime option policy_progress_seconds, the objects the last locate tracks are measured again every half of that time, and a run in which none of them moved 0.03 m for the whole of it ends: COMPLETED with detail stopped = “no_progress”; the program goes on from there, the model is not asked. An object not measured (hidden by the arms, put away) says nothing either way; with nothing measured nothing is judged. Guards, until, region, the time limit and the shield apply at every tick. With a measured gripper, detail “drops” lists jaws that closed fully after holding something steadily.

list_policies

def list_policies()

Learned skills available to run_policy: name, arms, and a description of the kinds of manipulation it does reliably, partly or not at all.

What the model reads

From the RoboDojo prompt pack (prompts/robodojo/), with every skill and feature on:

  start(strategy, kind, policy=None) -> Trace or None
      Your first motion, once: the strategy for this task and the line of the policy's
      description the task matches (kind, in your words). "A": hand it the whole task and
      supervise; "C": the policy first, then finish what it leaves; "B": your own tools
      only. A and C run the policy at once, as run_policy(policy) from the untouched start
      pose (default: the first policy), and return its trace; B returns None. Until start,
      every motion ends at once with SKILL_ERROR ("call start(strategy, kind) first");
      looking runs (get_observation, locate, segment_text, point_to take no episode
      time; watch_scene takes the time it watches).
      Called again with the same strategy it does nothing. After B, start("A" or "C",
      kind) hands the task over after all (from the start pose: home() first); other
      changes are refused.
      After its first run you have a few model calls (the turn says how many); then it
      runs on to the end of the episode by itself.
  run_policy(name, instruction=None, arms=("left_arm", "right_arm"), max_seconds=None,
             region=None, until=None, reset=False, stall_seconds=5.0) -> Trace
      Hands control to the policy. It was trained on whole tasks: demonstrations that
      start at the start pose and finish the task, labelled with the task's own
      instruction. It does best in that setting: from the untouched start pose, with no
      instruction (the task's own is used) and both arms, left to run. A fresh run (the
      first, another instruction or arms, or reset=True) starts only from the start
      pose: elsewhere it ends at once with SKILL_ERROR, detail reason "not at the start
      pose" (home() first; it keeps the grippers). Without max_seconds it runs until the
      episode ends, until(obs) holds, the arms stop moving (once they have moved, no
      joint of its arms moves 0.02 rad for stall_seconds; arms still from the start are
      waiting), or an event stops it. Stopping or using up max_seconds is a normal end:
      the trace ends with COMPLETED and trace.event.detail has "stopped":
      "stalled" or "max_seconds"; your program goes on from there, so
      check the scene in it (PREDICATE_SATISFIED: until held).
      detail "drops" lists jaws that closed fully after holding something steadily.
      Called again with the same instruction and arms it continues where it was, with
      the rest of the action chunk it was executing (unless the arms moved in between),
      so slices of max_seconds act like one long run. Other instructions, one arm, or a
      start in the middle of the task are outside what it learned: expect much less
      there. Guards, until, region and the shield apply at every tick.
  list_policies() -> [{"name", "description", "arms"}]
      The description says which kinds of manipulation the policy does reliably, partly
      or not at all, and the signs that it is failing. Read it before your first motion.