eef

Tools by object id: locate (SAM 3: positions, footprints, reach; the located objects measured again after every program), pick, release, transfer, push, and move_tip for free motion.

• Layer: base
• Needs: segmenter
• Requires: segmentation
• Conflicts with: manipulation
• Never dropped: no
• Mixin: skills.eef:EefTools
• Manifest: skills/eef/skill.toml

Functions

eef_locate

def eef_locate(object)

Measure the objects named by object (up to 4 names separated by ‘;’) with SAM 3 in every camera. Returns {“objects”: [{“id”, “name”, “aliases”, “position”, “triangulated”, “score”, “length”, “width”, “angle_deg”, “container”, “held_by”, “reach”, “jaws”}], “text”}; the objects (o1, o2, … left to right) are the scene that pick, release, transfer and push take ids from. Names SAM 3 matched to the same pixels are one object (the others in its aliases). A close-up of every object is attached to your next turn: check each is what you named (SAM 3 matches names loosely). The names stay tracked: after every program the scene is measured again and relabelled.

eef_pick

def eef_pick(arm, object, grasp='auto', lift=None)

Grasp the scene object object (an id from the last locate) with arm and lift it. grasp: “auto” (across the width if the open jaws span it; else across a thin part of it, a handle or a band, unless it is a container; else by the edge with the most room for them), “across”, “part”, “edge” or “tilted” (by the near side, the tool tilted forward). lift: how far to rise (at least 0.10 m, plus half the width for an edge grasp). Returns {“ok”, “arm”, “object”, “grasp”, “grasp_point”, “check”, “reason”}; ok is whether the jaws hold it after the lift.

eef_release

def eef_release(arm, into=None, onto=None, x=None, y=None, withdraw=True)

Set down what arm holds: into a container (a scene id: its open middle, or the nearest spot inside it this arm reaches straight down; resting on its floor), onto an object (a scene id: centred on its top), or at the table point (x, y). The object’s centre goes there (the grip offset from the pick applied) and it opens with the object’s lowest point 5 mm above the surface. To cover an object with an upside-down container, give that object’s x, y (onto sets the container on top of it). withdraw (default True) moves the arm over its own side afterwards. Returns {“ok”, “arm”, “where”, “release_z”, “reason”}.

eef_transfer

def eef_transfer(arm, object, into=None, onto=None, x=None, y=None, grasp='auto', withdraw=True)

pick, then, if the jaws hold it, release: one call for moving an object into a container or onto an open spot. Returns {“ok”, “pick”, “release”}; it stops after a pick that holds nothing. Use pick and release apart where the placement must be precise, to see how the object hangs first.

eef_push

def eef_push(arm, object, x, y)

Push the scene object object along the table so its centre ends near (x, y): the jaws close, go behind it on the far side from (x, y), come down just clear of the table and slide in straight pieces of at most 3 cm, then lift clear. Returns {“ok”, “arm”, “object”, “start”, “end”, “reason”}; locate it again to see where it went.

eef_move_tip

def eef_move_tip(arm, x, y, z, yaw_deg=0.0, pitch_deg=0.0, gripper=None, speed=FREE_SPACE_SPEED)

Free motion in the tools’ own terms (Z2’s move_eef): move arm‘s jaws’ centre to (x, y, z), the tool straight down with the jaws closing along yaw_deg (0: along world x, 90: along world y), or tilted forward by pitch_deg (toward +y; 90 is horizontal); gripper (0 closed, 1 open) is set first when given. For free motion only (repositioning, a better wrist view, carrying high and clear); touch objects with pick, push and release. Returns {“ok”, “arm”, “tip”, “arrival_m”, “event”}: arrival_m is how far the jaws ended from the target (more than a centimetre near objects means something was touched).

What the model reads

From the RoboDojo prompt pack (prompts/robodojo/), with every skill and feature on:

  locate(object) -> {"objects": [{"id", "name", "position", "triangulated", "score", "length",
      "width", "angle_deg", "container", "held_by", "reach", "jaws"}], "text"}
      measures every instance of the named objects (up to 4 names separated by ';', each a
      short noun phrase such as "red mug; white basket") in all three cameras: a position
      triangulated from two views that agree (else assumed 2 cm above the table), the
      footprint (length, width, long side's angle from world x), which arms reach it
      straight down or tilted forward, and how the jaws would close on it. They become the
      scene, labelled left to right, and a close-up of every object reaches your next turn:
      check that each is what you named (SAM 3 matches names loosely). Locate an object
      together with where it goes. The names you located stay tracked: after every
      program they are measured again, and your next turn shows "Scene this turn" (each
      object with its label, position, size, reach and whether an arm holds it), the same
      labels on green outlines in the camera images, and each holding arm's grip. Labels
      can change from turn to turn: use this turn's.
  pick(arm, object, grasp="auto", lift=None) -> {"ok", "arm", "object", "grasp",
      "grasp_point", "check", "reason"}   grasps a scene object and lifts it: across its
      width when the open jaws (0.088 m) span it, else across a thin part of it that they
      span with room on both sides (a handle, a band; not for a container), else by the edge
      with the most room for them (about 1 cm below a wall's top); the tool straight down,
      or tilted forward where only that reaches. It travels above, measures the object
      again from there and follows it, comes down the last 3 cm in 1 cm motions, closes,
      settles, lifts (at least 0.10 m, plus half the width for an edge grasp) and checks
      the grip: "check" says whether the
      jaws read something between them and the object rose with them (measured with SAM 3).
      grasp "across", "part", "edge" or "tilted" overrides the choice: only when the
      close-up shows it is wrong. A pick that just failed, or a third at a spot where two
      failed with that arm, is refused: change something first.
  release(arm, into=None, onto=None, x=None, y=None, withdraw=True) -> {"ok", "where",
      "release_z", "reason"}   sets the held object down into a container (an id: its open
      middle, on its floor), onto an object (an id: centred on its top), or at the table
      point (x, y), its centre there (the grip offset applied), opening with its lowest
      point 5 mm above the surface, the last 3 cm in 1 cm motions; then it rises and
      withdraws the arm over its own side (withdraw=False keeps it there). To cover an
      object with an upside-down container, give that object's x, y: onto sets the
      container on top of it.
  transfer(arm, object, into=None, onto=None, x=None, y=None, grasp="auto", withdraw=True)
      -> {"ok", "pick", "release"}   pick, then release if the jaws hold it. Use pick and
      release apart where the placement must be precise.
  push(arm, object, x, y) -> {"ok", "start", "end", "pieces", "reason"}   slides a scene
      object so its centre ends near (x, y): the jaws closed and side by side across the
      push, from behind it, just clear of the table, in straight pieces of at most 3 cm;
      then it lifts clear. A push moves 0.5 to 40 cm; locate it again to see where it went.
  move_tip(arm, x, y, z, yaw_deg=0.0, pitch_deg=0.0, gripper=None, speed=0.6) -> {"ok",
      "arm", "tip", "arrival_m", "event"}   free motion in these tools' terms: the jaws'
      centre to (x, y, z), the tool straight down with the jaws closing along yaw_deg (0:
      along world x, 90: along world y), or tilted forward by pitch_deg (toward +y; 90 is
      horizontal); gripper (0 closed, 1 open) first when given. arrival_m is how far the
      jaws ended from the target: more than a centimetre near objects means something was
      touched. Only through clear space; objects are touched with pick, push and release.