在 RoboTwin 2.0 上运行
RoboTwin 2.0(SAPIEN)是第二个 benchmark:50 个双臂任务,每个任务由自己的 worker(benches/robotwin/runtime.py)提供,机器人 API 与 RoboDojo 的相同。
- 按
third_party/sources.toml中的 revision 检出 RoboTwin(检出到third_party/RoboTwin,或设置HARNESS_ROBOTWIN_ROOT),并获取其资源(Hugging Face 数据集TianxingChen/RoboTwin2.0;sha256 值在sources.toml中)。 bash environment/setup.sh --only robotwin:在项目环境中安装 SAPIEN 3.0.1、mplib、toppra,应用 RoboTwin 的两个源码补丁,并在third_party/curobo-0.7.8中就地构建 RoboTwin 自己的 cuRobo 0.7.8(environment/robotwin_curobo.sh;HARNESS_ROBOTWIN_CUROBO可指向其他位置)。worker 会把它放在PYTHONPATH的最前面。
只有 RoboTwin 的专家能解决的 seed 才是有效的 RoboTwin seed。episode 索引 n 是第 n 个有效 seed:评测 seed 从 100000 × (1 + group) 开始(官方的那些),开发 seed 从 10,000,000 + 100000 × group 开始。有效 seed 缓存在 seed_cache_dir/<split>-<group>/<task_config>/<task>.json 中(seed_cache_dir 默认为 <RoboTwin>/harness_seeds)。在运行文件中,layouts = "procedural" 选择开发 split,layout_pack_seed 选择 group。
# RoboTwin 2.0 development check on stack_blocks_two: the agent with the RoboTwin prompt# pack (AgileX ALOHA). No learned skill for this embodiment (no policy or world model# trained on RoboTwin is served), and RoboTwin has no task descriptions.description = "the agent on RoboTwin, development seeds"bench = "robotwin"tasks = ["stack_blocks_two"]seeds = [0]layouts = "procedural"endpoint = "http://127.0.0.1:8772"
[budget]max_wall_seconds = 5400max_cost_usd = 4.0
[overrides.agent]prompt = "../../prompts/robotwin"# The tools by object id (skills/eef) are tuned to RoboDojo's ARX X5 (table height,# parking spots, jaw width); RoboTwin's ALOHA uses the manipulation skill's.skills = { layers = { learned = false }, enable = ["manipulation"], disable = ["eef"] }services = { policies = [], world_models = [] }# RoboTwin has no XPolicyLab embodiment profile here: the worker shows programs its own# observations (the default agent's contract is RoboDojo's leaderboard protocol).runtime = { task_descriptions = false, contract = "native" }harness run configs/experiments/robotwin-dev.toml --deploy mine --output runs/robotwin-devharness run configs/experiments/robotwin-dev.toml --deploy mine --tasks all --seeds 0-9 --output runs/robotwin-all或者手动针对一个 worker 运行:
harness serve --factory benches.robotwin.runtime:create_environment \ --options configs/workers/robotwin_runtime_dev.toml --port 8772harness program configs/experiments/robotwin-dev.toml与 RoboDojo 的区别
Section titled “与 RoboDojo 的区别”| RoboDojo | RoboTwin | |
|---|---|---|
| 控制频率 | 25 Hz | 10 Hz |
| 指尖偏移 | 0.150 m | 0.12 m |
finish 之后机械臂回到起始位姿 |
是(任务要求如此) | 否 |
| 学习型 skill | Simate(agent 使用的)、G0.5、其他 XPolicyLab 策略;OpenWAM 用于实验 | 不提供 |
| 操作工具 | 按物体 id(locate、pick、release 等) |
操作 skill 的按名称的 pick 和 place |
| prompt pack | prompts/robodojo |
prompts/robotwin(自己的 core.md,其他部分共享) |
| 自我改进脚本 | 有 | 无(它启动的是 RoboDojo worker) |
该实验用 RoboTwin prompt pack 运行 agent([overrides.agent]):关闭学习层(因此没有 strategy 步骤),用操作 skill 代替按物体 id 的工具(那些工具是针对 RoboDojo 的 ARX X5 调校的),使用 SAM 3,不提供任务描述,agent 的 30 次模型调用。