配置与 identity

只有一个 agent。一次运行评测的是这个 agent（或它的某个消融）搭配一个模型，在一个 benchmark 上，并针对一台机器的 deploy 文件解析得到的结果。这些文件的分层方式与其他 agent 框架相同（SWE-agent 的默认配置、Codex 和 Claude Code 的设置文件）：后面的层优先。

层	位置	设置什么
代码默认值	代码	文件中未写出的每个键的值
agent	`configs/default.toml`	prompt pack、skill、它们使用的服务、运行时设置、预算、默认模型
模型	`configs/models/<name>.toml`	模型角色：main（编写程序）、vlm、pointer、verifier
实验	`configs/experiments/<name>.toml`	benchmark、任务、seed、layout、预算、worker 选项、覆盖项
命令行	`--model NAME`、`--agent FILE`、`--set key=value`、`--tasks`、`--seeds`	以上任何内容，仅对一次调用生效

configs/
├── default.toml   the agent
├── ablations/     the paper's ablations: each includes default.toml and switches parts off
├── models/        model roles: hosted models or services, plus the runtime settings a model needs
├── experiments/   the agent with a model on a bench: tasks, seeds, layouts, budgets, overrides
├── deploy/        per machine: GPUs, weights, where each service runs
├── workers/       options for starting a worker by hand (harness serve --options ...)
└── scripts/       scripted model replies for pipeline checks

服务在哪里运行由 deploy 文件决定，从不由 agent 决定；密钥放在环境变量中，从不放在这些文件中。

agent

# The agent: what `harness run` runs unless an experiment names another agent file.
# Every layer of the harness is on: programs over analytic and learned skills (Simate),
# the reactive layer (guards and controllers every tick), handlers, and the success
# verifier, above the fixed safety shield. It sees what RoboDojo's official client hands
# a policy, nothing more (the leaderboard protocol), and it is the configuration
# submitted to the leaderboard. The ablations of the paper (configs/ablations/) include
# this file and switch parts off.
description = "Harness Policy: programs over analytic and learned skills, reactive layer, handlers, verifier"
prompt = "../prompts/robodojo"
# The model roles (configs/models/<name>.toml); an experiment's `model` or --model NAME
# picks another.
model = "deepseek-flash"
# No pointer role: the vlm role (the main model) points, as in every 1.4 evaluation run,
# where Molmo2 was not running and programs pointed in 3 of 129 episodes.
roles = ["main", "vlm", "verifier"]

[skills]
layers = { learned = true, reactive = true, handlers = true }

[services]
segmenter = "sam3"
# The learned policy: Simate. Inside the agent it scored as G0.5 did on the development
# comparison; alone it scored more over all 42 tasks (Long-Horizon 38.9 against 19.1).
policies = ["simate"]
# No world model: no 1.4 evaluation run had OpenWAM, and without it the models fit one
# 24 GB GPU (kinematics, SAM 3, Simate: 23.2 GB). `--set 'agent.services.world_models=
# ["openwam"]'` adds predict_future.
world_models = []

[runtime]
# Long enough for pointing calls inside a program to wait out a network outage.
cell_timeout_seconds = 1800
reactive_budget_ms = 20.0
default_max_seconds = 30.0
cameras = ["cam_head", "cam_left_wrist", "cam_right_wrist"]
verifier = true
# The first motion is start(strategy, kind): hand the task to the learned policy (A, C)
# or not (B), decided from its description before anything moves (Z2's first call).
strategy_first = true
# The handed-over task runs uninterrupted: no progress watch (policy_progress_seconds
# 0). Stopping the policy early bought nothing (the model's own tools after it added no
# score), and measuring the scene while it ran may harm it: pour_liquid_into_cup,
# development layouts 0-1, policy alone 4 of 4 (2 scripted now, 2 on 13 October), with
# the watch measuring 1 of 4 (2 scripted, 2 of agent 1.4.0). The stall stop still ends
# a run whose arms moved and then stopped.
policy_progress_seconds = 0.0
# After the handed-over task's first run the model has this many calls to finish (its own tools, the policy
# again, or finish); then the policy runs on to the end of the episode. Finishing with
# the tools took about half of the development run's spend for no score over the policy
# alone.
post_policy_calls = 4
# The XPolicyLab observation contract (core/contract.py): instruction, camera colors,
# measured arm joints, link6 poses, the last gripper commands; camera calibration from
# the embodiment; no depth, no step limit, no task name, so no task descriptions
# (configs/ablations/with-task-description.toml adds them, Tab. 4's second row).
# `--set agent.runtime.contract=native` shows a development run everything the
# simulator worker measures.
contract = "xpolicylab"
# Programs run isolated where the machine allows user namespaces (an evaluation host
# need not).
isolation = "auto"

[budget]
# 30 model calls per episode (Zhang et al. 2026b ran 100). A handed-over task needs a
# few; the model's own tools where they fail used the rest of 100: agent 1.4.0 on
# evaluation layouts, pick_from_conveyor_by_image 98 calls ($0.38) for 0.
max_model_calls = 30

agent 的代码版本是 src/harness_policy/__init__.py 中的 AGENT_VERSION；各版本之间的变化及其证据见更新日志。每个键都见配置参考。

消融

一个消融包含 agent，并且只改变论文中该消融所改变的内容：

# Ablation (Tab. 5): the agent without the success verifier (verify() returns None).
description = "without the verifier"
include = ["../default.toml"]

[runtime]
verifier = false

文件可以 include 其他文件（按顺序合并；文件自身的键优先）。路径相对于引用它们的文件。接口阶梯 R0-R3 和 Tab. 5 中的消融列在接口阶梯中。

模型

模型与 agent 分开，因此 agent 可以搭配任何模型运行：

# grok-4.7 at low reasoning effort, for development. At medium effort xAI streams no
# reasoning, and OpenRouter cuts a request after about 5 minutes with nothing received:
# long replies are lost. Low effort answers in 13-60 s (about $0.11 per RoboDojo episode).
description = "grok-4.7 (low) writes programs, answers questions about images and points; Molmo2 for a pointer role"
include = ["grok-medium.toml"]

[main]
reasoning_effort = "low"

# Self-hosted development model, no API spend: Qwen3-VL-32B-Instruct (4 bit, the
# qwen3-vl-32b service) writes the programs, answers questions and points; Molmo2 for a pointer role.
description = "Qwen3-VL-32B (self-hosted, 4 bit) writes programs, answers questions and points; Molmo2 for a pointer role"

[main]
service = "qwen3-vl-32b"
max_output_tokens = 2048
# Sampling breaks the exact-repeat loops a small model falls into at temperature 0.
temperature = 0.6
timeout_seconds = 600

[vlm]
service = "qwen3-vl-32b"
timeout_seconds = 300

[pointer]
service = "molmo2"

[runtime]
keep_images = 1
# The 32B model in 4 bit on one 24 GB GPU runs out of memory beyond about 7k prompt
# tokens; with images downscaled by the server three turns fit.
max_history_turns = 3
# The task descriptions do not fit its context either.
task_descriptions = false

一个角色是一个托管模型（backend = "openrouter"，默认值，或 openai_chat，用于任何 OpenAI 兼容的服务器）或 deploy 文件中的一个服务（service = "molmo2"），并在其上叠加自己的设置。密钥从角色指定的环境变量中读取（api_key_env，默认 OPENROUTER_API_KEY），从不从文件中读取。--model NAME（或实验中的 model = "NAME"）选择 configs/models/<NAME>.toml。

一个实验

# Development: the agent with its default model on one task and development layouts
# (never evaluation layouts). `harness run configs/experiments/dev.toml
# --deploy <machine>` starts the simulator workers itself; `harness program` uses the
# worker at `endpoint`.
description = "the agent, one task, development layouts"
bench = "robodojo"
tasks = ["general_pickup"]
seeds = [0]
layouts = "procedural"
endpoint = "http://127.0.0.1:8771"

[budget]
max_wall_seconds = 5400
# Cost guard for development checks; the paper runs set no cost cap.
max_cost_usd = 4.0

--set key.path=value 对一次调用修改实验；--set agent.a.b=v 和 --set model.a.b=v 修改 agent 和模型角色：

harness run configs/experiments/subset8.toml --model grok-medium --output runs/medium
harness run configs/experiments/dev.toml --set agent.runtime.keep_images=1 --set budget.max_cost_usd=2 --output runs/k1
harness run --tasks stack_bowls --seeds 0-2 --agent configs/ablations/r2.toml --output runs/r2

没有实验文件时，harness run 在 RoboDojo 的开发 layout 上，针对命令行给出的任务和 seed 运行 agent。

lock 与 identity

针对一个 deploy 文件解析 agent，会得到 driver 的配置和一个 lock：决定这次运行计算什么的一切，而不包含任何关于在哪里运行的信息。

在 lock 中	不在其中
agent 的版本和配置名	URL、端口、GPU
prompt 的 sha256 和来源	服务是常驻、按需还是外部的
已启用的 skill，以及被移除的 skill 及其原因	API 密钥
模型角色（由服务支持的角色以其服务名表示）	机器
skill 使用的服务，以及其清单的 sha256
运行时设置和预算

lock 的哈希值就是配置的 identity（16 个十六进制字符）。每个 episode 都在其 manifest.json 中记录 lock、identity、agent 版本和 git commit；identity 相同的两次运行运行的是同一个配置。论文中的一张表、排行榜上的一个条目和一个导出的策略各自指明一个 identity。

harness lock --out agent.lock                          # the lock, the identity, a code fingerprint
harness config configs/experiments/dev.toml --json     # everything the experiment resolves to

严格解析

在缺少相应服务的机器上，agent 可能会失去一些 skill，而较小的配置有另一个 identity。因此评测运行（layouts = "eval"）会严格解析：agent 指名的某个服务在该机器上没有运行时，运行会停止，而不是以较小的配置运行；skills.names 中指名的 skill 无法运行时同样如此。消融按设计省略的 skill（R0 没有视觉角色）在每台机器上都会被移除，不算错误。--strict / --no-strict 可覆盖默认行为。