Configuration and identity

There is one agent. What a run evaluates is that agent (or one of its ablations) with a model, on a bench, resolved against a machine’s deploy. The files layer the way other agent frameworks layer theirs (SWE-agent’s default config, Codex’s and Claude Code’s settings files): later layers win.

Layer	Where	What it sets
code defaults	the code	the value of every key a file leaves out
the agent	configs/default.toml	prompt pack, skills, the services they use, runtime settings, budget, the default model
a model	configs/models/<name>.toml	model roles: main (writes programs), vlm, pointer, verifier
an experiment	configs/experiments/<name>.toml	the bench, tasks, seeds, layouts, budgets, worker options, overrides
the command line	--model NAME, --agent FILE, --set key=value, --tasks, --seeds	anything above, for one invocation

configs/
├── default.toml   the agent
├── ablations/     the paper's ablations: each includes default.toml and switches parts off
├── models/        model roles: hosted models or services, plus the runtime settings a model needs
├── experiments/   the agent with a model on a bench: tasks, seeds, layouts, budgets, overrides
├── deploy/        per machine: GPUs, weights, where each service runs
├── workers/       options for starting a worker by hand (harness serve --options ...)
└── scripts/       scripted model replies for pipeline checks

Where services run is the deploy’s business, never the agent’s; keys live in environment variables, never in these files.

The agent

# The agent: what `harness run` runs unless an experiment names another agent file.
# Every layer of the harness is on: programs over analytic and learned skills (Simate),
# the reactive layer (guards and controllers every tick), handlers, and the success
# verifier, above the fixed safety shield. It sees what RoboDojo's official client hands
# a policy, nothing more (the leaderboard protocol), and it is the configuration
# submitted to the leaderboard. The ablations of the paper (configs/ablations/) include
# this file and switch parts off.
description = "Harness Policy: programs over analytic and learned skills, reactive layer, handlers, verifier"
prompt = "../prompts/robodojo"
# The model roles (configs/models/<name>.toml); an experiment's `model` or --model NAME
# picks another.
model = "deepseek-flash"
# No pointer role: the vlm role (the main model) points, as in every 1.4 evaluation run,
# where Molmo2 was not running and programs pointed in 3 of 129 episodes.
roles = ["main", "vlm", "verifier"]

[skills]
layers = { learned = true, reactive = true, handlers = true }

[services]
segmenter = "sam3"
# The learned policy: Simate. Inside the agent it scored as G0.5 did on the development
# comparison; alone it scored more over all 42 tasks (Long-Horizon 38.9 against 19.1).
policies = ["simate"]
# No world model: no 1.4 evaluation run had OpenWAM, and without it the models fit one
# 24 GB GPU (kinematics, SAM 3, Simate: 23.2 GB). `--set 'agent.services.world_models=
# ["openwam"]'` adds predict_future.
world_models = []

[runtime]
# Long enough for pointing calls inside a program to wait out a network outage.
cell_timeout_seconds = 1800
reactive_budget_ms = 20.0
default_max_seconds = 30.0
cameras = ["cam_head", "cam_left_wrist", "cam_right_wrist"]
verifier = true
# The first motion is start(strategy, kind): hand the task to the learned policy (A, C)
# or not (B), decided from its description before anything moves (Z2's first call).
strategy_first = true
# The handed-over task runs uninterrupted: no progress watch (policy_progress_seconds
# 0). Stopping the policy early bought nothing (the model's own tools after it added no
# score), and measuring the scene while it ran may harm it: pour_liquid_into_cup,
# development layouts 0-1, policy alone 4 of 4 (2 scripted now, 2 on 13 October), with
# the watch measuring 1 of 4 (2 scripted, 2 of agent 1.4.0). The stall stop still ends
# a run whose arms moved and then stopped.
policy_progress_seconds = 0.0
# After the handed-over task's first run the model has this many calls to finish (its own tools, the policy
# again, or finish); then the policy runs on to the end of the episode. Finishing with
# the tools took about half of the development run's spend for no score over the policy
# alone.
post_policy_calls = 4
# The XPolicyLab observation contract (core/contract.py): instruction, camera colors,
# measured arm joints, link6 poses, the last gripper commands; camera calibration from
# the embodiment; no depth, no step limit, no task name, so no task descriptions
# (configs/ablations/with-task-description.toml adds them, Tab. 4's second row).
# `--set agent.runtime.contract=native` shows a development run everything the
# simulator worker measures.
contract = "xpolicylab"
# Programs run isolated where the machine allows user namespaces (an evaluation host
# need not).
isolation = "auto"

[budget]
# 30 model calls per episode (Zhang et al. 2026b ran 100). A handed-over task needs a
# few; the model's own tools where they fail used the rest of 100: agent 1.4.0 on
# evaluation layouts, pick_from_conveyor_by_image 98 calls ($0.38) for 0.
max_model_calls = 30

The agent’s code version is AGENT_VERSION in src/harness_policy/__init__.py; what changed between versions, and the evidence for it, is in the changelog. Every key is in the configuration reference.

Ablations

An ablation includes the agent and changes only what the paper’s ablation changes:

# Ablation (Tab. 5): the agent without the success verifier (verify() returns None).
description = "without the verifier"
include = ["../default.toml"]

[runtime]
verifier = false

Files include others (merged in order; the file’s own keys win). Paths are relative to the file that names them. The interface ladder R0-R3 and the Tab. 5 ablations are listed in The interface ladder.

Models

The models are separate from the agent, so the agent runs with any model:

# grok-4.7 at low reasoning effort, for development. At medium effort xAI streams no
# reasoning, and OpenRouter cuts a request after about 5 minutes with nothing received:
# long replies are lost. Low effort answers in 13-60 s (about $0.11 per RoboDojo episode).
description = "grok-4.7 (low) writes programs, answers questions about images and points; Molmo2 for a pointer role"
include = ["grok-medium.toml"]

[main]
reasoning_effort = "low"

# Self-hosted development model, no API spend: Qwen3-VL-32B-Instruct (4 bit, the
# qwen3-vl-32b service) writes the programs, answers questions and points; Molmo2 for a pointer role.
description = "Qwen3-VL-32B (self-hosted, 4 bit) writes programs, answers questions and points; Molmo2 for a pointer role"

[main]
service = "qwen3-vl-32b"
max_output_tokens = 2048
# Sampling breaks the exact-repeat loops a small model falls into at temperature 0.
temperature = 0.6
timeout_seconds = 600

[vlm]
service = "qwen3-vl-32b"
timeout_seconds = 300

[pointer]
service = "molmo2"

[runtime]
keep_images = 1
# The 32B model in 4 bit on one 24 GB GPU runs out of memory beyond about 7k prompt
# tokens; with images downscaled by the server three turns fit.
max_history_turns = 3
# The task descriptions do not fit its context either.
task_descriptions = false

A role is a hosted model (backend = "openrouter", the default, or openai_chat for any OpenAI-compatible server) or a service of the deploy (service = "molmo2"), with its own settings on top. Keys are read from the environment variable the role names (api_key_env, default OPENROUTER_API_KEY), never from a file. --model NAME (or an experiment’s model = "NAME") picks configs/models/<NAME>.toml.

An experiment

# Development: the agent with its default model on one task and development layouts
# (never evaluation layouts). `harness run configs/experiments/dev.toml
# --deploy <machine>` starts the simulator workers itself; `harness program` uses the
# worker at `endpoint`.
description = "the agent, one task, development layouts"
bench = "robodojo"
tasks = ["general_pickup"]
seeds = [0]
layouts = "procedural"
endpoint = "http://127.0.0.1:8771"

[budget]
max_wall_seconds = 5400
# Cost guard for development checks; the paper runs set no cost cap.
max_cost_usd = 4.0

--set key.path=value changes an experiment for one invocation; --set agent.a.b=v and --set model.a.b=v change the agent and the model roles:

harness run configs/experiments/subset8.toml --model grok-medium --output runs/medium
harness run configs/experiments/dev.toml --set agent.runtime.keep_images=1 --set budget.max_cost_usd=2 --output runs/k1
harness run --tasks stack_bowls --seeds 0-2 --agent configs/ablations/r2.toml --output runs/r2

Without an experiment file, harness run runs the agent on RoboDojo’s development layouts for the tasks and seeds the command line gives.

The lock and the identity

Resolving the agent against a deploy gives the driver’s configuration and a lock: everything that decides what the run computes and nothing about where it runs.

In the lock	Not in it
the agent’s version and configuration name	URLs, ports, GPUs
the prompt’s sha256 and sources	whether a service is resident, on demand or external
the enabled skills, and the dropped ones with their reasons	API keys
the model roles (a service-backed role by its service name)	the machine
the services the skills use, and their manifests’ sha256
the runtime settings and the budget

The lock’s hash is the configuration’s identity (16 hex characters). Every episode records the lock, the identity, the agent version and the git commit in its manifest.json; two runs with the same identity ran the same configuration. A paper table, a leaderboard entry and an exported policy each name one.

harness lock --out agent.lock                          # the lock, the identity, a code fingerprint
harness config configs/experiments/dev.toml --json     # everything the experiment resolves to

Strict resolution

The agent can lose skills on a machine that lacks their services, and a smaller configuration has another identity. Evaluation runs (layouts = "eval") therefore resolve strictly: a service the agent names that the machine does not run stops the run instead of running a smaller configuration, as does a skill named in skills.names that cannot run. Skills an ablation leaves out by design (R0 has no vision roles) drop on every machine and are not an error. --strict / --no-strict override the default.