CLI

harness (or python scripts/harness from a checkout). Every command prints JSON or a table; errors go to stderr as {"error": ...} with exit status 1.

Command	What it does
run	Run the agent over the tasks and seeds of an experiment (configs/experiments/*.toml, or –tasks and –seeds)
program	Run the seeds of one task against a worker that already runs
config	Show what the agent (or an experiment) resolves to on a deploy: skills, services, identity
lock	Write the agent’s lock: what decides its behavior, and its identity
export	Export the agent as a policy another evaluator runs
report	RoboDojo Score/SR table over program-harness runs
services	List the services (services/*.toml)
up	Start the resident services of a deploy (all, or the ones named)
down	Stop services that harness up started
status	Where each service of a deploy runs and whether it answers
doctor	Check that this machine can run a deploy: files, GPUs, ports, interpreters
fetch	Download the weights of services into the deploy’s weights root
serve	Serve a worker (a robot or simulator) in its own Python process

harness run

Run the agent over the tasks and seeds of an experiment (configs/experiments/*.toml, or –tasks and –seeds)

usage: harness run [-h] [--deploy DEPLOY] [--model MODEL] [--agent AGENT] [--set KEY=VALUE]
                   [--tasks TASKS] [--seeds SEEDS] [--strict | --no-strict] --output OUTPUT
                   [--gpus GPUS] [--workers-per-gpu WORKERS_PER_GPU] [--base-port BASE_PORT]
                   [--rerun-errors] [--max-total-cost-usd MAX_TOTAL_COST_USD] [--script SCRIPT]
                   [config]

Argument	Default	Meaning
config		experiment file (default: none; give –tasks)
--deploy		deploy name (configs/deploy/<name>.toml) or path; default $HARNESS_DEPLOY
--model		model roles: a name of configs/models/ (e.g. grok-low) or a .toml file
--agent		agent file in place of configs/default.toml (e.g. configs/ablations/r2.toml)
--set		override an experiment key, e.g. budget.max_cost_usd=2, agent.runtime.keep_images=1, model.main.reasoning_effort=low (repeatable)
--tasks		tasks overriding the experiment’s: names, dimensions, moving, all
--seeds		seeds overriding the experiment’s, e.g. 0-4 or 0,2,5
--strict, --no-strict		refuse to run without a skill or service the agent names (default: on for evaluation layouts)
--output		suite directory (resumes when it exists)
--gpus		GPUs for simulator workers, e.g. 3,5 (default: the deploy’s [sim] gpus)
--workers-per-gpu
--base-port
--rerun-errors		run again the episodes that ended in an error
--max-total-cost-usd		start no new episode once the suite spent this much
--script		JSON replies to replay instead of calling a model

harness program

Run the seeds of one task against a worker that already runs

usage: harness program [-h] [--deploy DEPLOY] [--model MODEL] [--agent AGENT] [--set KEY=VALUE]
                       [--tasks TASKS] [--seeds SEEDS] [--strict | --no-strict] [--task TASK]
                       [--endpoint ENDPOINT] [--output OUTPUT] [--script SCRIPT]
                       [config]

Argument	Default	Meaning
config		experiment file (configs/experiments/*.toml)
--deploy		deploy name (configs/deploy/<name>.toml) or path; default $HARNESS_DEPLOY
--model		model roles: a name of configs/models/ (e.g. grok-low) or a .toml file
--agent		agent file in place of configs/default.toml (e.g. configs/ablations/r2.toml)
--set		override an experiment key, e.g. budget.max_cost_usd=2, agent.runtime.keep_images=1, model.main.reasoning_effort=low (repeatable)
--tasks		tasks overriding the experiment’s: names, dimensions, moving, all
--seeds		seeds overriding the experiment’s, e.g. 0-4 or 0,2,5
--strict, --no-strict		refuse to run without a skill or service the agent names (default: on for evaluation layouts)
--task		task (default: the experiment’s first)
--endpoint		worker URL (default: the experiment’s endpoint)
--output	runs
--script		JSON list of model replies to replay instead of calling a model

harness config

Show what the agent (or an experiment) resolves to on a deploy: skills, services, identity

usage: harness config [-h] [--deploy DEPLOY] [--model MODEL] [--agent AGENT] [--set KEY=VALUE]
                      [--tasks TASKS] [--seeds SEEDS] [--strict | --no-strict] [--json]
                      [config]

Argument	Default	Meaning
config		experiment file or agent file (default: configs/default.toml)
--deploy		deploy name (configs/deploy/<name>.toml) or path; default $HARNESS_DEPLOY
--model		model roles: a name of configs/models/ (e.g. grok-low) or a .toml file
--agent		agent file in place of configs/default.toml (e.g. configs/ablations/r2.toml)
--set		override an experiment key, e.g. budget.max_cost_usd=2, agent.runtime.keep_images=1, model.main.reasoning_effort=low (repeatable)
--tasks		tasks overriding the experiment’s: names, dimensions, moving, all
--seeds		seeds overriding the experiment’s, e.g. 0-4 or 0,2,5
--strict, --no-strict		refuse to run without a skill or service the agent names (default: on for evaluation layouts)
--json		print the whole resolved configuration

harness lock

Write the agent’s lock: what decides its behavior, and its identity

usage: harness lock [-h] [--deploy DEPLOY] [--model MODEL] [--agent AGENT] [--set KEY=VALUE]
                    [--tasks TASKS] [--seeds SEEDS] [--strict | --no-strict] [--out OUT]
                    [config]

Argument	Default	Meaning
config		agent file or experiment file (default: configs/default.toml)
--deploy		deploy name (configs/deploy/<name>.toml) or path; default $HARNESS_DEPLOY
--model		model roles: a name of configs/models/ (e.g. grok-low) or a .toml file
--agent		agent file in place of configs/default.toml (e.g. configs/ablations/r2.toml)
--set		override an experiment key, e.g. budget.max_cost_usd=2, agent.runtime.keep_images=1, model.main.reasoning_effort=low (repeatable)
--tasks		tasks overriding the experiment’s: names, dimensions, moving, all
--seeds		seeds overriding the experiment’s, e.g. 0-4 or 0,2,5
--strict, --no-strict		refuse to run without a skill or service the agent names (default: on for evaluation layouts)
--out		write the lock here (default: print it)

harness export

Export the agent as a policy another evaluator runs

usage: harness export [-h] {xpolicylab} ...

harness export xpolicylab

An XPolicyLab policy folder (policy//) for RoboDojo evaluation

usage: harness export xpolicylab [-h] --name NAME --out OUT [--deploy DEPLOY] [--model MODEL]
                                 [--set KEY=VALUE] [--vendor] [--xpolicylab XPOLICYLAB]
                                 [--allow-dirty] [--embodiment EMBODIMENT] [--check] [--gpu GPU]
                                 [--env ENV] [--script SCRIPT]
                                 [config]

Argument	Default	Meaning
config		agent file or experiment file (default: configs/default.toml); its runtime.contract must be xpolicylab
--name		the policy’s folder and module name, e.g. HarnessPolicy
--out		the folder to write (XPolicyLab/policy/<Name>)
--deploy		deploy name (configs/deploy/<name>.toml) or path; default $HARNESS_DEPLOY
--model		model roles: a name of configs/models/ or a .toml file
--set
--vendor		copy this repository into the folder (the-harness/) instead of letting install.sh clone it
--xpolicylab		XPolicyLab checkout (default: the deploy’s)
--allow-dirty		export with uncommitted code changes
--embodiment	arx_x5	the robot: a name of embodiments/ (default arx_x5, RoboDojo’s); it must be calibrated
--check		then run XPolicyLab’s PR checks and the debug closed loop
--gpu	0	–check: policy GPU ids, e.g. 2,3
--env	harness-policy	–check: conda environment of policy and client
--script		–check: model replies to replay (default: configs/scripts/xpolicylab_check.json)

harness report

RoboDojo Score/SR table over program-harness runs

usage: harness report [-h] [--json] runs [runs ...]

Argument	Default	Meaning
runs		experiment directories (searched recursively)
--json		print JSON instead of Markdown

harness services

List the services (services/*.toml)

usage: harness services [-h] [--json]

Argument	Default	Meaning
--json

harness up

Start the resident services of a deploy (all, or the ones named)

usage: harness up [-h] [--deploy DEPLOY] [--json] [names ...]

Argument	Default	Meaning
names		services (default: all the deploy runs)
--deploy		deploy name (configs/deploy/<name>.toml) or path; default $HARNESS_DEPLOY
--json

harness down

Stop services that harness up started

usage: harness down [-h] [--deploy DEPLOY] [--json] [names ...]

Argument	Default	Meaning
names		services (default: all the deploy runs)
--deploy		deploy name (configs/deploy/<name>.toml) or path; default $HARNESS_DEPLOY
--json

harness status

Where each service of a deploy runs and whether it answers

usage: harness status [-h] [--deploy DEPLOY] [--json] [names ...]

Argument	Default	Meaning
names		services (default: all the deploy runs)
--deploy		deploy name (configs/deploy/<name>.toml) or path; default $HARNESS_DEPLOY
--json

harness doctor

Check that this machine can run a deploy: files, GPUs, ports, interpreters

usage: harness doctor [-h] [--deploy DEPLOY] [--json] [names ...]

Argument	Default	Meaning
names		services (default: all the deploy runs)
--deploy		deploy name (configs/deploy/<name>.toml) or path; default $HARNESS_DEPLOY
--json

harness fetch

Download the weights of services into the deploy’s weights root

usage: harness fetch [-h] [--deploy DEPLOY] names [names ...]

Argument	Default	Meaning
names
--deploy		deploy name (configs/deploy/<name>.toml) or path; default $HARNESS_DEPLOY

harness serve

Serve a worker (a robot or simulator) in its own Python process

usage: harness serve [-h] --factory FACTORY [--host HOST] [--port PORT] [--options OPTIONS]
                     [--contract {native,xpolicylab}] [--embodiment EMBODIMENT]

Argument	Default	Meaning
--factory		module:callable, e.g. benches.robodojo.runtime:create_environment
--host	127.0.0.1
--port	8765
--options		TOML keyword arguments passed to the factory
--contract	native	what clients may see (core/contract.py); xpolicylab: what RoboDojo’s official client hands a policy
--embodiment		embodiment name (embodiments/<name>/embodiment.toml) for the contract