Services

harness_policy.services.registry

Service manifests: the model processes skills and agents call.

A service is one deployable model server, described by services/<name>.toml:

kind = "segmenter"            segmenter | pointer | vlm | llm | policy | world_model | kinematics
summary = "..."
protocol = "http"             http | openai_chat | xpolicylab
command = ["{python}", "services/servers/serve_sam3.py", "--checkpoint", "{weights}/sam3/sam3.pt",
           "--port", "{port}"]
port = 8810                   default port
vram_gb = 7.2                 GPU memory it holds once loaded (measured)
load_seconds = 60             start until it answers
client = { url = "{url}" }    the kernel's or agent's config for it ({url}, {host}, {port})
[[weights]]                   where `harness fetch` gets its files
dir = "sam3"
modelscope_model = "facebook/sam3"

An XPolicyLab policy needs no command: its [xpolicylab] table names the policy folder, action type and overrides, and the server is XPolicyLab’s own setup_policy_server.py. Where a service runs (GPU, port, resident or on demand, or not at all) is the deploy file’s business (deploy.py), not the manifest’s.

ServiceSpec

class ServiceSpec

A service manifest (MANIFEST_KEYS) as loaded: name is the file’s stem and path the file. Placeholders such as {python} or {port} are left for the deploy to fill.

load_services

def load_services(root: Path=SERVICES_ROOT) -> dict[str, ServiceSpec]

Every service manifest under root (its *.toml files) by name, validated; read once per root.

service

def service(name: str, root: Path=SERVICES_ROOT) -> ServiceSpec

The manifest of service name; an unknown name is an error.

harness_policy.services.deploy

Deploy files: where the services of one machine run.

configs/deploy/<machine>.toml says which GPUs this machine gives the project, where its weights live, and for each service how it runs (git ignores machine files there; only the example-*.toml ones are tracked):

weights = "/data/weights"         root of the services' weight directories
state = "/data/harness-services"  pid files and logs of started services
gpus = [2, 3, 4, 5]               GPUs services and simulators may use
gpu_memory_gb = 24

[sim]                             simulator workers of `harness run`
gpus = [5]
workers_per_gpu = 2

[services.sam3]
gpu = 3                           a GPU index, or "auto" (fit by vram_gb)
[services.molmo2]
mode = "external"                 someone else runs it
url = "http://10.0.0.7:8830"
[services.openwam]
mode = "off"

[services.g05]
gpu = 1
mode = "on_demand"                started when a skill first calls it ...
swap = "big"                      ... and sharing GPU 1 with the other services of
[services.openwam]                its swap group, one loaded at a time
gpu = 1
mode = "on_demand"
swap = "big"

A service the deploy file does not list is off. Modes: resident (started by harness up, kept running), on_demand (started by the broker when a skill needs it, stopped after idle_seconds or when a peer of its swap group is needed; see broker.py), external (only a URL), off. Placement never changes what a run computes, only where and when; systems decide which services a run uses.

Placement

class Placement

How one service runs on a deploy (PLACEMENT_KEYS). A service the deploy file does not list has mode off; loading the deploy replaces gpu = “auto” with a GPU index.

Deploy

class Deploy

A loaded deploy file: the machine’s weights, state and checkout paths, its GPUs, the simulator settings, and the placement of each service it lists. Without a deploy file (name “none”) every service is off.

services

def services() -> dict[str, ServiceSpec]

Every service manifest (under services_root) by name, whether it runs here or not.

placement

def placement(name: str) -> Placement

How service name runs here (mode off when the file does not list it); an unknown service is an error.

variables

def variables() -> dict

The placeholders of service commands, settings and needed files: python (this process’s interpreter), repo, weights, robodojo, xpolicylab, xpolicylab_parent, state.

deploy_path

def deploy_path(name: str | os.PathLike | None) -> Path | None

The deploy file for a name (configs/deploy/<name>.toml) or a path; None names $HARNESS_DEPLOY, and without it there is no deploy file.

load_deploy

def load_deploy(name: str | os.PathLike | None=None, *, services_root: Path=SERVICES_ROOT) -> Deploy

The deploy of name (see deploy_path); without one, every service is off and only hosted models are available.

planned_memory

def planned_memory(deploy: Deploy) -> dict[int, float]

GPU memory the deploy plans per GPU: resident services add up, a swap group counts as its largest member (one is loaded at a time).

render

def render(value, variables: dict)

Fill {name} placeholders in strings (recursively in lists and dicts).

port

def port(deploy: Deploy, name: str) -> int

The port a service listens on here: its placement’s, else its manifest’s.

url

def url(deploy: Deploy, name: str) -> str | None

Where clients reach a service, or None when it is off.

client_config

def client_config(deploy: Deploy, name: str) -> dict | None

The config a skill or agent uses for this service (its manifest’s client table with the URL filled in), or None when the deploy does not run it.

launch

def launch(deploy: Deploy, name: str) -> tuple[list[str], dict, str]

(argv, environment, working directory) that start a service of this deploy.

weight_dirs

def weight_dirs(deploy: Deploy, name: str) -> list[Path]

The service’s weight directories under the deploy’s weights root, one per [[weights]] entry.

missing_files

def missing_files(deploy: Deploy, name: str) -> list[str]

Files the service needs that are not on this machine.

harness_policy.services.supervisor

Start, stop and check the services of a deploy (harness up, down, status).

Each service runs as its own process group with its log in <state>/logs/<name>.log and a record in <state>/<name>.json (pid, port, GPU, command). A service whose port already answers is left alone: on a shared machine it may have been started by hand or by another checkout, and starting a second copy would only fail on the port.

listening

def listening(port: int, host: str='127.0.0.1', timeout: float=0.5) -> bool

Whether something accepts TCP connections on host:port.

record

def record(deploy, name: str) -> dict | None

The record of a service started here (<state>/<name>.json: pid, port, GPU, argv, cwd, env, start time), or None when there is none or it is unreadable.

status

def status(deploy, names=None) -> list[dict]

One row per service the deploy places (or names): mode, GPU, URL, whether its port answers and the process harness up started for it.

start

def start(deploy, name: str, *, wait: bool=True, log=print) -> dict

Start one resident service unless its port already answers. Returns its status.

wait_ready

def wait_ready(deploy, name: str, process=None, timeout: float | None=None) -> None

Wait until service name answers on its port. Raises RuntimeError when process (or the pid of its record) exits first, and TimeoutError after timeout seconds (default three times its load_seconds, at least 60).

stop

def stop(deploy, name: str, *, grace: float=15.0, log=print) -> bool

Stop a service harness up started (SIGINT, then SIGTERM, then SIGKILL to its process group). Services started otherwise are not touched.

up

def up(deploy, names=None, *, log=print) -> list[dict]

Start the resident services (all, or names) at once, then wait until each answers; start the broker when the deploy has on-demand services.

start_broker

def start_broker(deploy, log=print)

Start the broker of the deploy’s on-demand services, unless there are none or it runs.

down

def down(deploy, names=None, *, log=print) -> None

Stop the services harness up (or the broker) started; without names also the broker.

harness_policy.services.broker

On-demand services: started when a skill first calls them, swapped on a shared GPU.

A deploy places a service with mode = “on_demand” (and optionally swap = “<group>”, idle_seconds = N). The broker, one per deploy and machine, starts such a service the first time a client asks for it (POST /ensure), keeps at most one service of a swap group loaded, and stops one that nobody used for idle_seconds. Clients ask before each call (services/broker.py:ensure, throttled), so a service in use is never swapped out: a request for another member of its group waits until it has been idle for SWAP_GRACE seconds.

The simulator does not advance while a skill waits, so loading a model on demand costs wall time only, not episode steps; results are those of a resident deploy. A policy with memory (scoped = true) that is swapped out starts its memory afresh.

python -m harness_policy.services.broker --deploy two-gpu      (harness up starts it)

Broker

class Broker
    __init__(deploy, log=print, swap_grace: float=SWAP_GRACE)

The broker of one deploy’s on-demand services: it starts them when asked, swaps members of a swap group on their GPU, and stops services left idle.

on_demand

def on_demand() -> list[str]

The deploy’s on-demand services, sorted by name.

ensure

def ensure(name: str, timeout: float=1800.0) -> dict

Make name answer, swapping out idle peers of its group; returns when it does.

reap

def reap()

Stop on-demand services unused for their idle_seconds.

status

def status() -> list[dict]

One row per on-demand service: its swap group, whether it answers, and seconds since it was last asked for (None if never).

serve

def serve(host='127.0.0.1', port=None)

The broker’s HTTP server (GET: status; POST /ensure {“service”}: ensure), bound but not yet serving, on the deploy’s broker port by default. Starts the idle reaper.

broker_port

def broker_port(deploy) -> int

The port of the deploy’s broker (default 8899).

broker_url

def broker_url(deploy) -> str

The URL of the deploy’s broker on this machine.

ensure

def ensure(config: dict | None, *, timeout: float=1800.0) -> None

Before calling a service: when its client config is on demand, ask the broker to make it answer (at most every CLIENT_INTERVAL seconds while it is in use).

without_on_demand

def without_on_demand(config: dict | None) -> dict | None

The client config without the broker entry (for clients that pass it on).

main

def main(argv=None)

Run the broker of --deploy until interrupted, with its pid and port in <state>/broker.json while it runs.

harness_policy.services.clients

Clients of the model services that skills call: the segmentation server and vision-language models behind an OpenAI-compatible endpoint (pointing, questions, the success verifier).

Segmenter

class Segmenter
    __init__(url, timeout=120.0, on_demand=None)

Client of services/servers/serve_sam3.py.

segment_point

def segment_point(image, points, labels=None)

Candidate masks of the object under the points (SAM 3 interactive predictor), best predicted IoU first.

segment

def segment(image, prompt, threshold=None)

Instances best first. SAM 3 scores depend strongly on phrasing (on one image “bowl” scored 0.13 and “white bowl” 0.72 for the same three bowls), so by default instances are kept when they score at least half the best score (and at least 0.05); pass threshold for an absolute cut instead.

VisionModel

class VisionModel
    __init__(config)

A vision-language model role (vlm, pointer or verifier) on a ChatModel, at temperature 0 with 1024 output tokens unless its config says otherwise. usage adds up its calls, tokens and cost; point_format (json or molmo) is how it points.

ask

def ask(images, question, system='Answer briefly and precisely.')

The model’s text answer to question about images (RGB arrays, sent as JPEG) under system; the call is added to usage.

point

def point(image, prompt)

The pixel points (x, y) the model gives for prompt in image, read from its 0-1000 coordinates (JSON point_2d, or Molmo2 points); [] when it gives none.

molmo_points

def molmo_points(text, width, height)

Pixels from a Molmo2 reply: <points coords=“1 1 433 424 2 614 465”>…</points>, i.e. the image number, then index x y per point with x and y scaled to 0-1000 (Molmo 1’s <point x=“52.3” y=“40.2”> in percent too).

image_data_url

def image_data_url(image)

An HxWx3 RGB array as a JPEG data URL.