scene

Watching the scene (watch_scene), clearing the head camera’s view (clear_view) and pressing buttons (press), whichever pick and place tools are on.

• Layer: base
• Needs: one of pointer, vlm
• Requires: pointing
• Conflicts with: –
• Never dropped: no
• Mixin: skills.scene:SceneTools
• Manifest: skills/scene/skill.toml

Functions

Also in the namespace: watch_scene, clear_view, press (constants and names the skill’s exports(session) provides).

What the model reads

From the RoboDojo prompt pack (prompts/robodojo/), with every skill and feature on:

  watch_scene(seconds, every=1.0, cameras=("cam_head",), track=None, until=None) ->
      {"frames": [{"t", "step", "observation"}], "departures", "trace"}   holds both arms
      still and records frames every `every` seconds; shows you a contact sheet of them
      (up to 12, each labelled with its time). track=["red cup", ...] points at those
      objects in the first frame and reports when each left its spot for good
      ("departures", earliest first, with "pixel", "position" and "left_at" seconds, None
      when it stayed). Use it to watch a demonstration or anything that happens while
      the robot must keep still.
  clear_view(speed=0.6) -> [Trace, Trace]   moves both arms out to the sides and up, both
      at once; at the start pose the grippers hide the table below them from cam_head,
      so call it and observe again when fewer objects are found than the instruction implies
  press(target, times=1, arm=None, depth=0.008) -> {"ok", "arm", "presses", "contacts",
      "reason"}   presses a button at [x, y, z] (its top), a scene id ("o1") or a name,
      `times` times with the closed fingertips: descends slowly to contact, pushes `depth`
      further, lifts; the further presses go straight down to that depth and up; then the
      arm returns to its start pose. Use it for buttons and switches instead of open jaws
      or your own millimetre steps.