Changelog

What changed in the agent (AGENT_VERSION in src/harness_policy/__init__.py), and the evidence for each change. A version changes the identity of every configuration, so results name the version that produced them. Minor versions change what the agent does; patch versions fix it.

1.6.5

• 1.6.2 of PR #107 (written on 1.6.0) on top of 1.6.4; the evidence below was measured on 1.6.0’s tree with this change.

• locate() no longer triangulates with a view the image border cuts off. The 2 cm offset of 1.6.0’s press_by_number buttons came from that, not from the wrist cameras’ calibration. A depth diagnostic checked the calibration against the simulator’s render on 92 frames over three tasks. The contract’s wrist pose matches the worker’s and the render camera’s to 0.0 mm and 0.013 deg. Wrist depth unprojected with it meets the head camera’s surface after a correction of at most 0.04 deg and 0.5 mm; the residual is 0.6 mm. In the arms’ start pose, the near row of objects sits on the bottom edge of each wrist image. A cut mask’s centroid is not the object’s, and the least-squares point slides along the head camera’s ray: toward +y, outward and low. With exact masks from depth, a cut wrist view put buttons 1.9-4.4 cm off (z 0.70-0.75); a complete one put them 0.0-0.2 cm off. In the 1.4.1 full evaluation, 60 of the 144 wrist pairings of the first locates (start pose) were cut, in 40 episodes. Such an object is now placed from the head camera alone, 2 cm above the table, like an object one camera sees.

• A triangulated point above every object on the table (MAX_OBJECT_Z) and above what the jaws could hold (the highest fingertip + 10 cm) is rejected. It pairs the masks of two instances of one name whose rays cross in the air: arrange_largest_number digits were reported at z 1.28-1.34, 33 cm off.

Evidence: locate replayed on recorded frames of press_by_number, arrange_largest_number and solve_equation (pack-0 layout 0, scripted arm poses), with the same SAM 3 masks for both versions and ground truth from the simulator. Over 226 objects, errors above 2 cm fell from 16 to 2, errors above 1 cm from 44 to 27, and the mean from 0.64 to 0.48 cm (p90 1.92 to 1.12 cm). Objects located more than 10 cm from any true one fell from 5 to 1. press_by_number went from a 2.02 cm maximum to 0.31 cm. The cost: flat digits at the table’s edge that a slightly cut wrist view had put 0.2 cm off are now 1.2 cm off (the assumed 2 cm height). pick re-measures from above before it closes.

Regression: the agent’s own-tool tasks on pack-0 layouts 0-2 (tree 0a11b7c, DeepSeek V4.1 Flash; an unstable layout is replaced by the next, as RoboDojo does).

Task	1.6.2	1.6.0	1.4.1, same layouts
press_by_number	1, 1, 1 (5 calls each)	1, 1, 1	0, 0, 0
general_pickup (layouts 4, 5, 7)	1, 0, 1	-	1, 0, error
arrange_largest_number	0.05 (C), 0.05 (B), 0.05 (C)	-	0.3, 1.0, 0.4 (all B)
solve_equation (layouts 0, 2)	0 (B, 30 calls), 0 (B, 28 calls)	-	0, 0 (layout 1: 1)

In the first locate of each press_by_number episode (start pose), the three buttons were 0.22-0.27 cm off; 1.6.0 had put two of them 2 cm off there. solve_equation layout 2 failed at two releases that could not get above the spot (a motion error). Two of the three arrange_largest_number episodes went to the policy (strategy C), which uses no locate. In the B episode, locate put every digit within 1 cm. Three of its four picks held after re-measuring from above. The episode ran out of steps: a release at the pad was blocked, and one open jaw landed on a disc. solve_equation layouts 1 and 3 were unstable this time. Layout 4 has no earlier result, so its run was stopped. 1.6.0 was not run on these three tasks, so these numbers are not paired. Picks whose held object could not be measured next to the jaws, so that the wrist camera’s look decided, rose from 7 of 19 (1.4.1) to 6 of 10. The look was right every time.

1.6.4

• 1.6.1 of PR #108 (written on 1.6.0), ported onto 1.6.3’s episode core (#109): the core reports each execution’s start, ticks and end to the embodiment’s session (_motion_started, _motion_ticked, _motion_stopped); the robot session keeps the arms’ targets. The evidence below was measured on 1.6.0’s tree with this change.

• An arm that no skill moves holds a fixed target: its last command, or, when its last motion was interrupted (a guard, a fault, until, a region, a timeout, BLOCKED) or ended more than 0.01 rad from its command, the joints where it stopped. Until now every execution commanded the arms it did not move to their measured joints, each tick. Such an arm has no stiffness: whatever pushes it moves it, and it stays there.

• The cause is the other arm, not gravity as 1.6.0 assumed. A per-tick log of a scripted press_by_number (pack-0 layout 0, no model; configs/scripts/idle_arm_ticks.json): with the right arm waiting, fingertips 6 cm above the blue button’s base, the left arm’s probe toward the middle red button took the right arm down with it, tick for tick (link6 z 0.9732 to 0.9471 in 23 ticks, onto the blue button); the right arm did not move in the 36 ticks before, while the left arm closed its gripper and rose, nor in 63 ticks parked higher while the left arm moved in free space. The planner avoids self collision and the table, not the other arm.

• Scripted check (configs/scripts/idle_arm_check.json: the same start, then the left arm presses the middle red button three times, 366 ticks): the right arm’s link6 went from z 0.9732 to 0.9339 in 1.6.0 (39 mm, onto the blue button), and to 0.9730 in 1.6.1 (0.2 mm).

• Not fixed: the arm that pushes is now held back instead. In the same check the left arm’s probe stopped where it met the right arm, fingertips at z 0.824, 25 mm above the red button’s cap (0.799, which 1.6.0’s later probes reached by pushing the right arm on), and press() took that for the button’s contact (ok, 1 press each time); the right arm gave 13 mm while pushed and went back when the left arm lifted. Arms still have to stay out of each other’s way (press() sends its arm home at the end, as in 1.6.0); planning paths around the other arm is the fix for that.

• Regression on pack-0 evaluation layouts, the tasks where the agent’s own tools move the arms (B in 1.6.0’s routing replay; tree 5bd55bf, DeepSeek V4.1 Flash): press_by_number layouts 0-2 3 of 3 (601, 679 and 636 steps, 5 calls and $0.007 an episode), as 1.6.0 (601, 687 and 651 steps, 5 calls); general_pickup 3 of 3 (layouts 0, 3 and 4; 1 and 2 unstable), 3 calls and $0.006 an episode (1.4.1: 1.0 on layout 4, its only B episode; layouts 0-3 unstable in that run); solve_equation 0, 1 and 0 on layouts 0-2, as 1.4.1 (15, 15 and 30 calls, $0.106 an episode; 1.4.1 8, 25 and 19, $0.152); arrange_largest_number 0.3, 0.3 and 0.4 (1.4.1: 0.3, 1.0 and 0.4; 25 calls and $0.088 an episode, 1.4.1 21 and $0.071). On layout 1 the model, all four digits placed, picked them up again to reorder them, and two picks of digits lying close together came up empty. Over the three episodes 5 of 18 picks came up empty (1.4.1: 2 of 15), all five after every digit had been placed, when the model picked them up again (digits lying close together or on their discs). A scripted pick of each of layout 1’s four digits, each set back in place (configs/scripts/idle_arm_picks.json, no model): the change held 4 of 4, 1.6.0 3 of 4. One motion ended BLOCKED in the 12 episodes (1.4.1: 2 in its 9 B episodes of these tasks). python -m unittest discover -s tests on the server: 388 tests OK.

1.6.3

• press() spends fewer steps per call, so press_by_number’s largest counts fit its 700 steps. In 1.6.0’s traces (pack-0 layouts 0-9) a call with one press took 116 ticks: close the gripper 10, move above the button 24, probe 28, first push 15, lift 7, wait 3, back up 6, home 24; each further press 17. Layout 8 (8 + 9 presses in four calls) needed about 720 steps and ran out right before the last confirm. Now the gripper’s close is commanded with the move above the button (the session sends each gripper’s last command with every tick), the probe starts 4 cm above the target instead of 6 (the probe meets RoboDojo’s caps 1.3 cm above the head camera’s target; at least 6 cm above the table), the first push uses the later presses’ speed and time limit, and after the last press the arm goes straight up and home, without the lift and wait meant for a next press: a one-press call takes 82 ticks (above 24, probe 18, push 8, up 8, home 24). The later presses (17 ticks), the 8 mm depth and the homing that stopped the double counts are unchanged.
• Result: scripted press_by_number in the official order (no model, tree 533eb21): layout 8 passes in 567 steps; layouts 0-2 pass in 448, 537 and 494 steps (1.6.0: 572, 643 and 595). The agent on layouts 0-9 (tree 533eb21, DeepSeek V4.1 Flash): 10 of 10 (1.6.0: 8 of 10), 4.8 calls and $0.006 an episode (1.6.0: 5.3 and $0.008). Layout 8 passes in 583 steps; on the eight layouts both versions passed, 128-165 fewer steps (mean 442, 1.6.0 587). Layout 9 passed because the model confirmed between the counts this time (see below), not because of this change. On pack 0 x 3 (layouts 0-2, already 3 of 3) the Memory dimension stays 28.9.
• Not changed: in 1.6.0’s episode on layout 9 the model left out the confirm between the two counts. Its first reply planned “red x5, confirm, red x2, confirm”; after the first count it went on to the second red button. The same call replayed six times (1.6.0’s prompt and conversation up to that call; four more replies ended in provider errors) skipped the confirm once and pressed it five times: a lapse of about one in six at that point, not a misreading of the rule, so the rule’s wording stays until a change can be shown to lower it.
• imitate_sorting_sequence (diagnosis only, no change): RoboDojo replays the support arm’s demonstration 10 trajectory entries per policy step, so it plays only while the policy acts, from the first action to step 428-528 on pack-0 layouts 1-3 (17-21 s of watch_scene); a 20 s pause without actions advanced nothing. Every stage requires the whole demonstration in its basket, and the stages are credited one at a time. The forced-B episodes (0, 0, 0.05) never got the order: watch_scene(track=names) pointed at the policy’s own copies of the objects, which never move, not at the demonstration’s copies on the far side. Details in pull request #106.

1.6.0

• press() presses reliably and fast enough for RoboDojo’s counted presses. Its buttons travel 10 mm and count a press when they go from 95 % up to 50 %; a count above the required one fails the episode at once. A scripted press_by_number in the official order (pack-0 layouts 0-2, no model) failed 6 of 6, and per-step joint ratios showed why: probing every press took 40-60 ticks (layout 1’s 8 + 7 presses did not fit 700 steps), and the arm left above the blue button sank onto it while the other arm moved (link6 z 0.972 to 0.945) and pressed it a second time. Now only the first press probes for the contact; later presses go straight to 8 mm below it and back (about 15 ticks), and the arm returns to its start pose at the end, where an idle arm did not sink. The same script then passed 3 of 3 (572, 643 and 595 steps). The depth stays 8 mm: every press took the button to 0 %.
• press() aims at located buttons from the head camera. press("o1") pointed at the text “o1” with the head camera, beside the button; the probe met the table, and pushing on into it made the simulator’s arm joints run away (-27 rad). Positions triangulated with a wrist camera were 2 cm off for these buttons (o1 at (-0.159, -0.153) for one at (-0.150, -0.170); the head camera alone, 3 mm), and a fingertip that far off the cap’s centre met the table in 2 of 6 presses. A scene id, or a point within 3 cm of a located object at most 5 cm tall, is now aimed at by that object’s head-camera mask on the plane 2 cm above the table, and a first contact less than 12 mm above the table ends the presses instead of pushing on. The gripper closes without the vision model’s look (a network error in that call ended one scripted run).
• Counted presses are strategy B, and the confirm button confirms each count: press it after each card’s presses, before the next card’s. The instruction does not say so, and all seven episodes of 1.3.0 to 1.4.1 that pressed with the agent’s tools confirmed once at the end (0 each); a sentence in press()’s description was not enough (the agent still confirmed once), the rule the model reads when it picks the strategy is. The policy scores 0 on press_by_number in every run of ours (Simate alone 0 of 8).
• Result, press_by_number on pack-0 layouts 0-9 (tree 080fad0, DeepSeek V4.1 Flash): 8 of 10, 5 calls and $0.008 an episode; 1.4.1 and Simate alone scored 0 of 3 on layouts 0-2, where 1.6.0 scores 3 of 3. On pack 0 x 3 this raises the Memory dimension from 12.2 to 28.9 (Simate alone 22.2) and the RoboDojo average from 26.2 to 29.5, the other tasks as they were. The two failures: layout 8 (8 + 9 presses) ran out of the 700 steps before the last confirm, and in layout 9 the model left out the confirm between the two counts.
• Other tasks: press() was called in no other task of 584 episodes of 1.2 to 1.4.1 but make_toast (one development episode). make_toast on pack-0 layouts 0-2 with the new press() (tree 3480ba4, before the aiming change): 1.0, 1.0 and 0.25, the 75 of 1.4.1 on the same layouts, 2 calls each, press() not called. The routing decision of every task replayed under the old and the new prompt (the call at which 1.4.1’s episode on layout 0 called start, with its logged conversation before it, asked again): the split between the agent’s tools (B) and the policy (A or C) is the same for all 42 tasks, B for arrange_largest_number, general_pickup, press_by_number and solve_equation, as in 1.4.1’s run. Four tasks moved between A and C (both hand the task over), noise of the order of the two by which the old prompt’s replay differs from 1.4.1’s logged choices; 13 replies ended in network errors, and those tasks were compared with 1.4.1’s logged choice.
• Not changed, found on the way: an arm left away from its start pose is held at its measured joints and sinks while the other arm moves (only press() avoids it, by going home), and wrist-camera triangulation is about 2 cm off for these objects (only press() avoids it). imitate_sorting_sequence with the agent’s own tools, measured with a prompt that forced B (pack-0 layouts 1-3): 0, 0 and 0.05 (11-18 calls, up to $0.23); the model did not see the demonstration through, so its routing stays.

1.5.0

• The default agent is the configuration submitted to RoboDojo’s leaderboard: no pointer role (the vlm role, the main model, points instead of Molmo2) and no world model (roles = ["main", "vlm", "verifier"], world_models = []). The 1.4.2 package was exported from the default with these two settings as --set, so the default, the measured results and the submission were three different configurations on paper. Evidence that nothing is lost: every 1.4 evaluation run already ran this way (its deploy ran neither Molmo2 nor OpenWAM, so pointing went to the VLM and predict_future was not available); in 1.4.1’s run programs pointed in 3 of 129 episodes (6 point_to, 1 localize_3d). The agent’s models now fit one 24 GB GPU resident (kinematics, SAM 3, Simate: 23.2 GB, against 51.2 GB with Molmo2 and OpenWAM).
• R1 points with the vlm role too; R3, which includes the agent, has the learned policy without OpenWAM. Molmo2 and OpenWAM stay services: an experiment adds them with a pointer role or --set 'agent.services.world_models=["openwam"]'.

The behaviour is that of the 1.4.2 package; the identity is new (06bf87001afc3a8b with DeepSeek V4.1 Flash).

1.4.2

• Objects moving past on a conveyor go to the policy even when which one is read (a picture shown as the target): the model’s own tools are too slow for them. Evidence, agent 1.4.1 on evaluation pack 0 (the local leaderboard run): pick_from_conveyor_by_image scored 0 in its first 4 episodes with B, at 24.5 model calls, $0.09 and 31 minutes an episode (the target passed out of reach between locating and grasping); Simate alone scores 0.0 on it officially, so handing it over loses nothing and costs a few calls.

1.4.1

• Strategy B (the model’s own tools) only for picking one object out of several by what identifies it: its name in the instruction, a number, symbol or label written on it, or a picture shown as the target. Every other task goes to the policy, also when the instruction names colours, an order or categories. Evidence, agent 1.4.0 on evaluation layouts (pack 0, one layout a task, RoboDojo average 29.9 against Simate’s official 35.0): B won where a name picks the object (general_pickup 100 against 49.3) and lost where the instruction gives an order the policy learned (cover_blocks 15 against 100, arrange_largest_number 5 against 18.5); the model also sent sweep_blocks and push_T to B with nothing to read. Tasks handed over scored with the policy (A: 57.9 against 57.1).
• 30 model calls per episode in place of 100: a handed-over task needs a few, and the model’s own tools where they fail used the rest (pick_from_conveyor_by_image 98 calls, $0.38, for 0).

Validation (evaluation pack 0, three layouts a task, an unstable layout replaced by the next as RoboDojo does; frozen code; DeepSeek V4.1 Flash), paired with Simate alone through run_policy on the same layouts and code: RoboDojo average 26.2 against 22.2 (task mean 28.3 against 23.4, paired +4.8, 90% interval [-0.5, +10.1]; 14 tasks above, 19 equal, 9 below).

	Generalization	Memory	Precision	Long-Horizon	Open	RoboDojo average
agent 1.4.1	41.2	12.2	20.2	42.5	14.6	26.2
Simate alone, same layouts	35.3	22.2	13.3	35.8	4.2	22.2

3.6 model calls and $0.010 an episode (median 2 calls); 113 of 125 episodes handed over (2.3 calls, $0.003), 12 kept by the programs (16 calls, $0.08). The agent gains where the programs read (solve_equation 33 against 0, arrange_largest_number 57 against 12, general_pickup 50 against 0) and on several handed-over tasks (fill_pen_holder 72 against 35, make_toast 75 against 33); it loses Memory, where the policy alone did better (cover_blocks 40 against 67, match_and_pick_from_conveyor 33 against 67). Simate alone scored 22.2 on these layouts against 35.0 on the leaderboard: compare on the same layouts.

1.4.0

• Division of labor between the program and the learned policy: the program keeps what the policy cannot do (telling objects apart by a name, number, label, named colour or a picture shown to it); every other task the policy was trained on is handed over whole from the start pose (strategy A or C). A condition of a task it learned whole (orientation, place, order, by size, by category), a precise fit, moving objects or remembering an object seen earlier no longer makes the task B, and Simate’s description no longer decides B: its kinds choose A (usually finishes) or C. Remembering stays with the policy, unlike the paper’s split: it keeps a memory of past frames and scores officially 56.0 on match_and_pick_from_conveyor, 100 on cover_blocks and 38.7 on swap_blocks, where the agent’s tools scored 0 on the first (1.3.0 twice, and 1.4.0’s first version, which still sent remembering to B). Evidence: handing a whole task over from the start pose scores about what Simate alone does (pre-1.2 R4 stack_bowls and pour 100; 1.3.0 fold_clothes 60, store_laptop_and_headphones 50), while 1.3.0’s own tools on tasks the policy learned scored far below it at 40-65 model calls an episode (organize_table 25 against 71.8, pack_objects_into_box 25 against 49.9, sort_nesting_dolls_by_size 0 against 28.0, match_and_pick_from_conveyor 0 against 56); code still beats the policy where names decide (general_pickup 60-80 against 49, stack_blocks_by_language 50 against 17).
• The handed-over task runs uninterrupted: the default agent sets no progress watch (policy_progress_seconds = 0). Stopping the policy early bought nothing, since the model’s own tools after it added no score (below), and measuring the scene while it ran may harm it: on pour_liquid_into_cup, development layouts 0-1, the policy alone succeeded 4 times of 4 (2 scripted runs now, 2 on 13 October) and with the watch measuring every 5 s once of 4 (2 scripted, 2 of agent 1.4.0’s run) — a sign, not a proof, at 8 episodes.
• The program, not the model, watches the policy (an option): run_policy measures the objects of the last locate every half of policy_progress_seconds (10 s at first; SAM 3, no model call) and ends a run in which none moved 3 cm over a whole window (COMPLETED, detail stopped = "no_progress"), so the program finishes what is left. Only positions measured in two cameras count, and an object not measured says nothing: the first version also counted an object found more or fewer times as progress, and while the arms move they hide objects all the time, so it never stopped a run (classify_objects layout 1: the policy ran the whole 1100 steps for 0). While the watch measures tracked objects it replaces the stall stop (arms still for 5 s): Simate waits for the object on match_and_pick_from_conveyor’s conveyor, and the stall stop ended it after 5 s twice; arms that stopped for good stop the objects too, and the watch ends the run 10 s later. A run is not ended before something has moved in it or it has run 20 s: the policy reaches for its first object first (classify_objects layout 0 was stopped at the first judgement, 10 s in, and finished the task for 100 when the model ran it again). The stall stop counts only once the arms have moved in the run: arms still from the start are waiting (on match_and_pick_from_conveyor the objects were seen by one camera only, so nothing was tracked, and the stall stop ended the policy at 5 s twice). It replaces 1.2.2’s slices (POLICY_SLICE, policy_slice_seconds), after each of which the model looked and took over early: 1.3.0’s C tasks averaged 31.9 against Simate alone’s 42.4 on the same tasks (classify_objects 20 against 61.6, build_tower 30 against 87.4).
• Taking one object the instruction names is reading, even when the model located only that object: on general_pickup (“Pick up the speckled ceramic dove by 10 cm”) the model took the dove for the only object, chose C and scored 0 (Simate alone 0 on both layouts); with B it scored 100 on the other layout.
• Matching objects to each other by how they look is not reading: on make_kong (“declare a kong with the matching tiles”) the model took the tile faces for something to read, chose B, and handed over too late (0 points; Simate alone 50 on the same layouts, 76.0 officially).
• The model has 4 calls (post_policy_calls) after the first run of the handed-over task; then the policy runs on to the end of the episode (__policy_to_end__, the driver’s hook). Finishing with its own tools there took about half of the development run’s spend for no score over the policy alone: 7 handed-over episodes with 5 or more calls after the run scored 12.1 against Simate alone’s 10.7 on the same layouts, at $0.46 of the run’s $0.97 (organize_table: 19 calls to check what the policy had done, 22 steps of motion; classify_objects layout 1: 20 calls with 44 steps left).
• Replies that end a block with DeepSeek’s tool-call markup instead of its closing tag run: the markup is dropped and the block closed (50 of 1.4.0’s 677 development replies had it, each a call that ran nothing; fasten_screws layout 0 lost 5 calls in a row to it).
• The prompt says which looking takes episode time: watch_scene before a handover held the arms still for 12 s, half of make_kong’s 600 steps, before the policy started.
• After B, start(“A” or “C”, kind) hands the task over after all, from the start pose, when the model’s own tools fail early at a task the policy was trained on; other changes of strategy are still refused.

1.3.0

• The learned policy of the default agent is Simate ([services] policies), in place of G0.5. Evidence: inside the agent the two scored the same on the development comparison (agent 1.2.0, 8 tasks x development layouts 0-4: 21.8 with Simate against 21.9 with G0.5, paired -0.1, 90% interval [-9.1, +8.8]; stack_blocks +20 and stack_bowls +14, insert_tubes -16, $0.028 an episode against $0.034); alone, from the start pose over all 42 tasks on development layouts, Simate scored more (Long-Horizon 38.9 against 19.1, Generalization 31.2 against 27.2; the 8-task subset 13.1 against 15.2), and the leaderboard scores all 42. Its description lists sorting and inserting a coin as partial kinds, so the model hands those tasks to it (classify_objects went from strategy B to C); since 1.2.2 a handed-over task runs in slices the model looks between. Agents 1.2.1 and 1.2.2 were validated with Simate: 26.8 for 1.2.1 on the 8 tasks, against 25.9 for Z2_v01 in its own setting. Simate needs 14 GB of GPU memory against G0.5’s 16.4. The ablations that include the default (R1, R3) follow it.

1.2.2

• With start’s A or C the handed-over task runs in slices (runtime option policy_slice_seconds, 10 s in the default agent): start runs the first slice, run_policy(name) the next (a longer max_seconds is cut to one slice), the policy continuing where it was. A slice ends with the new event POLICY_SLICE, which stops the program unless it handles the event, so the model sees the scene measured again and continues the policy or takes over. Before, the policy ran until it stopped or the episode ended. Evidence: Simate’s description lists sorting as a partial kind, the model handed it classify_objects (strategy C) and it sorted for the whole 1100 steps, leaving the agent’s tools nothing (development layouts 0-2: 0.15, 0, 0.15, against 17 with G0.5, whose description made the model keep the task); G0.5’s failed stacking runs used the whole 550 or 800 steps too. A first version only ended the slice nominally: after it the model continued with run_policy(name, max_seconds=60), and the policy again ran to step 1096 of 1100.

Validation (the four development tasks the model hands over: classify_objects, insert_tubes, stack_bowls, stack_blocks; layouts 0-4; Simate): 40.2 against 36.5 for 1.2.1 (paired +3.8, 90% interval [-8.5, +17.5]: within the learned policy’s own spread); classify_objects 29 against 14, stack_blocks 66 against 49, stack_bowls 66 against 83, insert_tubes 0 and 0; 10.8 model calls an episode against 6.8, $0.021 against $0.014. A scripted check on a live worker: start(‘C’) and a run_policy asking for 60 s each stopped the program after 250 steps. In the episodes the model continued the policy while objects moved toward the goal; in classify_objects layout 0 it let Simate finish a wrong sort (0.15), and in layout 2 Simate sorted correctly (1.0). (1.2.1 and 1.2.2 ran there without 1.2.1’s last fix, the thin-part grasp staying put.)

1.2.1

Fixes to the tools by object id from the 1.2.0 development comparison’s failures.

• locate: names SAM 3 matched to the same pixels (intersection over union at least 0.6 in a camera) are one object, the other names in its aliases. Evidence: a locate of “tiara; headband; crown” made three objects of one tiara, and pick kept its open jaws clear of the two copies, chose a worse edge and closed on nothing twice (general_pickup layout 0); “yellow spatula; slotted spatula” the same (layout 4). An object in a holder or a basket covers a small part of it and stays separate.
• pick: an object too wide for the open jaws, unless it is a container, is taken straight down across a thin part of its footprint (a handle, a band: the grasping skill’s part grasps, the jaws landing clear of the rest), else by an edge as before; grasp="part" asks for it. Evidence: a flat spatula taken by an edge closed beside its blade (general_pickup layout 4: 0 in 1.2.0, Z2_v01 and the eef arm); in a 1.1.0 run the model called plan_grasp itself, which took the spatula across its 3.4 cm handle, and scored.
• pick: an object seen by one camera at locate had its height assumed (2 cm above the table); from the hover above it, the wrist camera’s measurement with the head camera now sets its height and the grasp height (before, only a flat narrow object’s).
• The turn report says a held object may have slipped out only when it is measured (triangulated) on the table: held scissors seen by the holding arm’s wrist camera only were assumed on the table (general_pickup layout 2).
• start called again with the same strategy (a program run again from its top) does nothing; it raised, and the turn was lost (general_pickup layout 0).
• pick across a thin part does not move the grasp by how far the whole object’s centre turned out to be when it is measured again from above: the part was planned on the head camera’s full footprint before the arm hovered over it, and the centre seen from another view says little about where a handle is. Evidence: the spatula of general_pickup layout 4 (with the fix above): the part grasp was planned on its 3.4 cm handle (x 0.065, where 1.1.0’s plan_grasp took it and lifted it), the re-measured centre was 18 mm off, the grasp followed it to x 0.047 and the jaws closed beside the handle.

Validation (8 tasks x development layouts 0-4, DeepSeek V4.1 Flash, Simate as the learned policy, against 1.2.0 with Simate): 26.8 against 21.8 (paired +5.0, 90% interval [+0.0, +10.9]); general_pickup 60 against 40 (layout 0’s tiara: one object from two names, lifted across its band), deposit_coin 8 against 4, stack_bowls 83 against 63 (the policy’s spread), insert_tubes 0 against 4; against Z2_v01 in its own setting +0.9 [-6.4, +8.5]; 8.5 model calls and $0.018 an episode, against 11.7 and $0.028.

The last fix came after that run: general_pickup again with it (layouts 0-4, Simate): 4 of 5 against 3 of 5 (and for 1.2.0 with G0.5 and Z2_v01). The spatula of layout 4 was lifted by its handle (grasp at x 0.062; the centre measured 22 mm off, the grasp stayed); the flat cookie bag of layout 3 has no thin part and still fails, in every version.

1.2.0

Z2_v01’s capabilities, which 1.0.0 was to keep in the default and mostly left out or off, are in the default agent: its tools, its every-turn scene, its start step and its way of working, in programs over the paper’s layers.

• The tools by object id (skills/eef: locate, pick, release, transfer, push; Z1’s tools, which Z2 used) are the default; the manipulation skill’s pick and place by name are the analytic-manipulation ablation (R1 and RoboTwin use them). watch_scene, clear_view and press are their own skill (skills/scene), so they come with either. Evidence (development comparison, 8 tasks x development layouts 0-4, DeepSeek V4.1 Flash, XPolicyLab contract): the manipulation skill’s pick held in 2 of 46 calls (two runs of 1.1.0: 5 of 82); the tools by object id scored 21.8 against 16.8 and 19.9 for the two runs of 1.1.0 (paired difference +3.4, 90% interval [-6.1, +13.6]; general_pickup 40 against 0 and 20).
• After every program the located objects are measured again with SAM 3 and the next turn shows them labelled for the next program, outlined in the camera images, with each holding arm’s grip offset and a likely slip, and the env steps used (skills may define turn_report(session); Z1 did this after every tool call).
• Without a measured gripper (the XPolicyLab contract), whether the object rose with the jaws decides a pick, as Z1’s pick check did; the VLM’s look at the wrist image only when the rise cannot be measured. Evidence: three development episodes succeeded right after a pick reported that nothing was held.
• With learned policies, start(strategy, kind) is the first motion (Z2’s first call): A hands the task to the policy and supervises, C runs it first and finishes what it leaves, B uses the agent’s own tools; nothing moves before it (runtime.strategy_first), and summary.json records the choice.
• The prompt asks for short programs, one contact with an object each, looking at the result before the next (it said “a program for as much of the task as you can foresee”: four classify_objects programs of 1.1.0 ran past the 30-minute cell limit and were killed with nothing checked), and carries Z2’s way of working (prompts/robodojo/method.md): a plan kept current in each program’s comment, verify every pick, think about each object, move freely and touch gently, set down never drop, work where it is easy, two arms no fighting, learn from failures, finish cleanly.
• One model sees the images, as in Z2: the VLM role (query_vlm, holding() without a measured gripper, pointing without a pointer) is the main model in every hosted model file, no longer qwen3.8-flash, and the prompt asks the model to identify objects itself from the images and close-ups. Evidence: in the first 1.2.0 development episode (classify_objects, layout 0) the model asked qwen3.8-flash what the objects were over a dozen times and got a different answer each time (eraser, toy car, marker for the same object).
• After finish() every gripper that holds nothing opens before the arms return to their start pose. RoboDojo scores 23 of its 42 tasks (in the development subset: stack_blocks, stack_bowls, classify_objects, insert_tubes) only with every gripper at least 0.8 open (is_all_gripper_open), and the wrap-up kept the grippers as they were (as Z2’s did): jaws left closed, by a push for one, lost the whole score. A gripper that holds something (holding(): the measured opening, or the vision model’s look at the wrist camera) stays closed, since the lifting tasks (general_pickup, match_and_pick_from_conveyor) score the object still in the air. Evidence: with both grippers opened, general_pickup layouts 1 and 2 lifted the object 10 cm, finished, and scored 0 when the wrap-up dropped it.
• move_tip(arm, x, y, z, yaw_deg, pitch_deg, gripper): free motion by the jaws’ centre in the tools’ own angles, with the distance it landed from the target, as Z2’s move_eef took it (move_eef takes link6 poses).

Validation (development comparison: 8 tasks x development layouts 0-4, DeepSeek V4.1 Flash, G0.5 and SAM 3; 40 episodes per arm, mean of the task scores):

task	1.2.0	Z2_v01	1.1.0	1.1.0 again	eef	R4 (archived)
classify_objects	17	3	0	0	3	3
deposit_coin	0	8	0	4	12	8
general_pickup	60	60	0	20	40	0
insert_tubes	20	4	8	0	4	8
match_and_pick_from_conveyor	0	0	0	20	0	20
pick_from_conveyor_by_image	0	0	0	0	0	0
stack_blocks	29	66	43	49	49	6
stack_bowls	49	66	83	66	66	66
score	21.9	25.9	16.8	19.9	21.8	13.9
success rate	17.5	22.5	15.0	17.5	17.5	10.0

Z2_v01 ran its archived checkout in its own setting (measured gripper, task recipes, 170 calls); the others ran the XPolicyLab contract. Paired differences (bootstrap 90% over the 40 (task, layout) pairs): 1.2.0 - Z2_v01 -4.0 [-12.6, +4.8]; 1.2.0 - the mean of the two 1.1.0 runs +3.6 [-7.0, +15.0]; the two 1.1.0 runs differ by +3.1 [-6.8, +13.1]. The gap to Z2_v01 is in the two stacking tasks, where the arms mostly hand the whole task to G0.5 in their first one or two calls: the bowls were stacked in 15 of the five arms’ 25 episodes and the blocks in 10 of 25, and 1.2.0’s 2 and 1 of 5 are within that. On the other six tasks, where the agent’s own tools do the work, 1.2.0 scored 16.2, Z2_v01 12.5, 1.1.0 1.3 and 7.3. Of 58 picks that moved, the object’s rise could be measured in 19; the wrist camera’s look decided the other 39.

1.1.0

The learned policies’ descriptions (list_policies(), client.description in services/<name>.toml) come from development layouts only: each policy run alone through run_policy (configs/experiments/policy-alone-dev.toml), its per-task scores sorted by scripts/policy_card.py into reliable (mean >= 0.7), partial (>= 0.1) and unreliable, and generalised into kinds of manipulation.

• G0.5: the earlier description came from official evaluation pack 0. On two development layouts of each task (four where the kind moved): pouring a liquid (pack 0: 1.00, development: 0.50) and standing tubes in a rack (0.70, 0.10) are partial, no longer reliable; hanging on a rack (0.15, 0.04) and sorting into groups (0.28, 0.04) are unreliable, no longer partial; folding cloth (0.20, 0.60) and towers (0.30, 0.65) are partial as before. Score by dimension on development layouts: Generalization 27.2, Long-Horizon 19.1, Precision 16.9, Memory 0, Open 0.
• Simate-beta gets a description (it had only how it was trained): reliable at pouring a liquid and putting objects into a bin or case, partial at packing, stacking, building from parts, appliances, tidying, filling holders; 0 on every Memory and Open task despite its frame memory. Long-Horizon 38.9 against G0.5’s 19.1 on the same layouts.
• Four tasks are not measured: cover_blocks, swap_blocks and play_tic_tac_toe have no development layout (every arrangement is an evaluation layout), and RoboDojo marks both development layouts of imitate_sorting_sequence unstable mid-episode.

1.0.0

The first version of one default agent: the paper’s harness (programs over analytic and learned skills, the reactive layer, handlers, the success verifier, above a fixed safety shield) as configs/default.toml, with the paper’s ablations as configs/ablations/ and models as configs/models/.

• Configuration in layers, as other agent frameworks have them: code defaults, the agent, a model file (--model), an experiment, the command line (--set).
• The named harnesses built on RoboProbe’s step-by-step tool loop (Z0, Z1, and Z2, which never entered main) are archived: tag archive/pre-default-agent, branch archive/z2.
• The agent sees what RoboDojo’s official client hands a policy and no more (the leaderboard protocol; the paper’s App. A): the XPolicyLab observation contract (the instruction, camera colors, measured arm joints, link6 poses, the last gripper commands; camera calibration from the embodiment; no depth, step limit or task name), and so no task descriptions. Every earlier number (R4, Z1, Z2_v01 on the official pack-0 cases) was measured with task descriptions and measured grippers, which an evaluated policy never gets; configs/ablations/with-task-description.toml is the paper’s separate “with task description” row. Programs run isolated where the machine allows (isolation = "auto").
• Learned policies are handed whole tasks from the untouched start pose, as the paper prescribes (§3.2, App. E) and as Z2 did: a fresh run_policy away from the start pose ends with SKILL_ERROR (“not at the start pose”; home() first), a run stops when the arms stop moving for 5 s, and with a measured gripper it reports drops. G0.5’s description (list_policies(), services/g05.toml) is Z2’s capability card: kinds of manipulation it does reliably, partly or not at all, never task names. The prompt asks for the decision before any motion, a look after the policy stops, and no reserve of steps. Evidence (Z2 on the official pack-0 cases, DeepSeek): handing the whole task over from the start pose gave Z2 its lead in Generalization (30.8 vs R4’s 20.2: store_laptop, build_tower, classify_objects done by G0.5 in one model call); the step-budget advice of Z2_v02/v03 lowered the 10 development cases from 51.0 to 35.0 and 23.5, and Z2_v04 without it came back to 45.0 (noise on those cases: about 15 points). The card was derived from pack-0 layouts and is to be rebuilt from development layouts.
• The prompt no longer tells the model to write one program for the whole task: a program covers as much as the model can foresee and ends where the next step depends on what the robot has not seen yet; the model decides then on fresh images (the paper’s App. E prompt leaves the program’s size to the model). Evidence: Z2, which looked after every tool call, scored 25.0 in Memory against R4’s 8.8 on the same 84 official cases; to be confirmed on development layouts, with calls per episode.
• Z1’s end-effector tools are a skill, skills/eef (locate, pick, release, transfer, push by object id), ported from RoboProbe’s step loop onto bounded executions with Z2’s fixes (a grip kept for wide objects, covering by the covered object’s x, y, picking with the arm that reaches the destination). It is optional and conflicts with the manipulation skill: an ablation swapped them (until 1.2.0). The default keeps the manipulation skill (R4 led Precision, 21.9 against Z2’s 17.5) until development runs compare the two.
• The default model is DeepSeek V4.1 Flash (configs/models/deepseek-flash.toml): medium effort, about $0.02 per episode in R4’s 84-case run, and served from our region; paper runs name their model in the experiment.