Skip to content

fix(v1): a real body refuses a LeRobot checkpoint that does not say its norm mode - #4804

Open
rmncardoso wants to merge 2 commits into
strands-labs:devfrom
rmncardoso:fix/v1-refuse-undeclared-norm
Open

rmncardoso wants to merge 2 commits into
strands-labs:devfrom
rmncardoso:fix/v1-refuse-undeclared-norm

Conversation

@rmncardoso

Copy link
Copy Markdown
Collaborator

Fix to part 4 of #4749 · roadmap #3818

In one paragraph

A LeRobot checkpoint stores no norm mode, so Policy("<checkpoint>") takes the registry's label, so101-range-gripper-percent-v1 for an SO-101 (registry/robots.json:401-404, policy/lerobot/contract.py:138). LeRobot has recorded SO arms in degrees by default since v0.4.4, the pinned 53feaef3 included. Read as range, a degrees checkpoint commanded to +90° landed up to 72° off on a hardware so101 over the repo's fake Feetech bus ([1] row 1). #4749 says a deployment's units are checked "before a session exists"; a checkpoint's were guessed, and a degrees default would misread range checkpoints instead ([1] rows 3, 19). Now LeRobotPolicy.attach refuses, on a real body, a checkpoint given neither norm= nor semantics= (or an empty one), before LeRobot is imported, the checkpoint fetched or the bus touched, naming both options and modes. A simulated body keeps the registry's default. A Mesh.connect() robot is real even for a simulated device ([2]), and a spec string alone cannot declare a mode ([1] row 14).

Measured

gate dev this PR
pytest tests/test_units_lerobot_modes.py, default env (the required check) 3 failed (ImportError: it loads LeRobot), 4 passed 7 passed
pytest tests/test_units_lerobot_modes.py tests/policy/test_lerobot.py, LeRobot env 4 failed (3 Missing 'type' field in config.json, 1 DID NOT RAISE), 13 passed 17 passed
[1] row 1: a degrees checkpoint at +90°, no norm= runs; +10.2, +20.4, −1.6, +6.0, +72.0° off ValueError, 0 bus frames
[1] rows 5-9, 11, 12, 14, 17, 18: empty semantics/norm, units=, start(), a Hub id, from_spec, sim first, twin=True all but the Hub id run, 1 download asked refused, 0 frames, 0 downloads
[1] rows 2, 4, 13: norm= degrees, semantics= range, a simulated so101 0.0°, 0.0° off; registry label the same
[2] Mesh.connect(), no norm=: a fake-bus so101, a simulated so101 both run both ValueError, 0 bus writes
LeRobot set (Evidence) 36 passed 40 passed
pytest, whole suite — 647 passed, 32 skipped, 1 deselected
ruff check, ruff format --check, mypy, lint-imports — clean
git merge-tree with #4784, #4790, #4794-#4803, each; then tests/test_layout.py and the second row — no conflict; 4 and 17 passed

Not in this PR

The served path: server_config() without norm= still takes the registry's label, which a ZenohPolicy client then uses (policy/lerobot/remote.py:67).

Questions

Should server_config refuse an undeclared checkpoint too, as LeRobot's ServerConfig does? Simulated arms then need norm=: 8 of 10 tests/policy/test_lerobot.py tests fail (Evidence).

Evidence

How it was run. macOS 26.7.1 arm64, Python 3.12.10, pytest 9.1.1, on a shared machine (load average 17 to 22), so no timing is claimed. The default env has no LeRobot, like CI's required check (hatch run test); the LeRobot env has LeRobot 53feaef3, the commit pyproject.toml pins, and torch 2.11.0. No hardware: FakeSerial (tests/test_drivers.py) is six STS3215 servos in memory, camera frames are synthetic, HF_HUB_OFFLINE=1 and STRANDS_ROBOTS_LIVE was unset. dev is a git archive of cc15d9a's strands_robots/, tests/, tests_integ/ and pyproject.toml; "dev with this PR's tests" is the same plus this PR's two test files. This PR's change is one commit, dc83ef3, on cc15d9a; the changelog fragment is a second commit. Every pytest below is nice -n 10 pytest -q -p no:cacheprovider --timeout=300 --basetemp=<a scratch dir> ..., run from the tree's root.

Red first.

  • Default env, pytest tests/test_units_lerobot_modes.py -rA: dev with this PR's tests 3 failed, 4 passed. Each case of test_a_real_arm_refuses_a_checkpoint_that_does_not_say_its_norm_mode ends ImportError: LeRobot policies need the lerobot extra: ... at policy/lerobot/contract.py:30, so dev went on to load LeRobot. This PR: 7 passed. The test needs no LeRobot, so the required check, which skips tests/policy, covers the refusal.
  • LeRobot env, pytest tests/test_units_lerobot_modes.py tests/policy/test_lerobot.py -rA: dev with this PR's tests 4 failed, 13 passed. The three cases fail Regex pattern did not match on Missing 'type' field in config.json of <tmp>/... (dev read the checkpoint), and test_a_real_arm_runs_a_checkpoint_only_once_its_norm_mode_is_declared fails DID NOT RAISE ValueError (tests/policy/test_lerobot.py:64). The guard test_a_local_checkpoint_drives_the_arm_through_run (a simulated arm keeps the registry's label) passes on both. This PR: 17 passed.

Code reading at cc15d9a and LeRobot 53feaef3, unless a GET is quoted.

  • so101's registry entry says "lerobot_norm": {"*": "range_m100_100", "gripper": "range_0_100"} (registry/robots.json:401-404), which semantics_for maps to so101-range-gripper-percent-v1 through units/semantics.json. server_config takes semantics or semantics_for(description, norm) (policy/lerobot/contract.py:138), so semantics="" gets the registry's label too, and norm={} gets radians-v1 (units/semantics.py:36-38).
  • LeRobot: use_degrees: bool = True (src/lerobot/robots/so_follower/config_so_follower.py:42), and norm_mode_body = MotorNormMode.DEGREES if config.use_degrees else MotorNormMode.RANGE_M100_100 (so_follower.py:72). By GET at each of LeRobot's 11 tags (gh api "repos/huggingface/lerobot/tags?per_page=100"), gh api "repos/huggingface/lerobot/contents/<path>?ref=<tag>" -H "Accept: application/vnd.github.raw" | grep -n "use_degrees: bool" gives False from v0.3.2 to v0.4.2 (robots/so101_follower/config_so101_follower.py:41) and at v0.4.3 (robots/so_follower/config_so_follower.py:43), and True from v0.4.4 (:43) to v0.6.1 (:42). LeRobot's own ServerConfig says "Explicit feature conventions are required when checkpoint metadata is incomplete." (src/lerobot/remote_inference/configs.py:75) and refuses an empty semantics (:87-88).
  • The order: run checks its arguments and then binds, before anything else (robot/run/loop.py:48-52), and bind calls attach(robot) before units, requirements and reset (robot/run/bind.py:64-65; units at :71). The check is the first step of attach after its same-robot return (policy/lerobot/local.py:53-58 here), before lerobot() (:59) and before server_config resolves the checkpoint (:62; a Hub id is downloaded there, contract.py:137). Each Robot loads its own Description (robot/robot.py:54, description/description.py:102-116), so a policy attached to a simulated arm is checked again on a real one ([1] row 17). is_real is port= or driver= given (robot/robot.py:56), the flag the operator gate reads (robot/gate.py:31), and Mesh.connect() returns Robot(name, driver=body, twin=False, id=device_id) (mesh/mesh.py:196).
  • A string policy is make(spec) with no keywords (policy/callable.py:38, policy/factory.py:76-80), and a checkpoint spec takes no query (factory.py:10, :58-63), so from_spec, or a string passed to run, cannot declare a mode.
  • The size of the error: range_m100_100 spreads -100..100 over a joint's calibrated sweep of R counts, and degrees put 360 on 4095 counts, so +v degrees read as range lands near v · R · 360 / (200 · 4095) degrees. With the probe's sweeps (R = 2534, 2789, 2234, 2425, 4095, shoulder_pan to wrist_roll) that is +10.2, +20.3, −1.6, +5.9, +72.0 off at +90; [1] row 1 reads back the integer goal counts as +10.2, +20.4, −1.6, +6.0, +72.0. The error is negative below R = 2275.

[1] A hardware so101 on the fake bus, LeRobot env, from a checkout: HF_HUB_OFFLINE=1 PYTHONPATH=.:tests:tests/policy python units_probe.py | grep -v "^Loading weights". Only act_chunk (LeRobot's PolicyRunner.predict on one frame, local.py:82-90) is replaced, by a fixed action; the checkpoint still loads, so each row is what binding, units, the calibrated conversion and the driver write for that action. The goal counts are read back with LeRobot's own FeetechMotorsBus._normalize, on a bus that is never connected. LeRobot's download is replaced by a stand-in that records the ask and fetches nothing.

"""A LeRobot checkpoint on a hardware so101 over the repo's fake Feetech bus, and on a simulated so101.
LeRobot venv, from a checkout: HF_HUB_OFFLINE=1 PYTHONPATH=.:tests:tests/policy python units_probe.py"""

import json
import tempfile
import time
from pathlib import Path

import lerobot.remote_inference.deployment as deployment
import numpy as np
import test_drivers
from lerobot.motors import Motor, MotorCalibration, MotorNormMode
from lerobot.motors.feetech import FeetechMotorsBus
from tiny_act import save_tiny_act

from strands_robots import Policy, Robot, Simulation
from strands_robots.policy import from_spec
from strands_robots.policy.lerobot import LeRobotPolicy
from strands_robots.robot.drivers import feetech as ft

# The repo's fixture, with wrist_roll 0..4095 and drive_mode 0 as LeRobot's SOFollower.calibrate writes them
CAL = json.loads((Path(test_drivers.__file__).parent / "fixtures" / "so101_follower_calibration.json").read_text())
CAL["wrist_roll"].update(drive_mode=0, range_min=0, range_max=4095)
DEG, RANGE = {"*": "degrees", "gripper": "range_0_100"}, "so101-range-gripper-percent-v1"
MODES = {"degrees": MotorNormMode.DEGREES, "range": MotorNormMode.RANGE_M100_100}
ACTION, HUB, SEEN = [np.zeros(6)], [], set()
LeRobotPolicy.act_chunk = lambda self, obs, *, instruction=None: np.tile(ACTION[0], (10, 1))  # the network, fixed


def no_hub(repo_id, **kw):  # LeRobot's download, replaced: it records the ask and fetches nothing
    HUB.append(repo_id)
    raise RuntimeError("no network in this probe")


deployment.snapshot_download = no_hub
CKPT = str(save_tiny_act(Path(tempfile.mkdtemp()) / "tiny-act-so101", 6))


def read_back(s, mode):  # the goal counts on the wire, through LeRobot's own conversion (a bus never connected)
    m = MODES[mode]
    motors = {k: Motor(c["id"], "sts3215", m if k != "gripper" else MotorNormMode.RANGE_0_100) for k, c in CAL.items()}
    bus = FeetechMotorsBus("/dev/null", motors, {k: MotorCalibration(**c) for k, c in CAL.items()})
    at = slice(ft.REG_GOAL_POSITION, ft.REG_GOAL_POSITION + 2)
    goal = {c["id"]: int.from_bytes(s.regs[c["id"]][at], "little") for c in CAL.values()}
    return list(goal.values()), list(bus._normalize(goal).values())


def outcome(p, go):
    try:
        return f"ran {go().steps} step(s) under {p.inner.contract.semantics}"
    except Exception as e:
        msg = str(e).replace(CKPT, "<ckpt>")
        said = msg if msg not in SEEN else "(the same message)"
        SEEN.add(msg)
        return f"{type(e).__name__}: {said}"


def real(target=90.0, mode="degrees", units="auto", spec=None, start=False, policy=None, twin=False, **said):
    ACTION[0] = np.array([target] * 5 + [50.0])
    d, s = test_drivers.make_driver(cal=CAL)
    d.images = lambda: {"front": np.zeros((48, 64, 3), np.uint8)}  # a synthetic frame: no camera is opened
    arm = Robot("so101", driver=d, twin=twin)
    s.frames.clear()
    p = policy or Policy(spec or CKPT, device="cpu", **said)
    if start:
        task = arm.start(p, steps=1)
        while task.running:
            time.sleep(0.05)
        out = f"status {task.status}; result: {outcome(p, lambda: task.result)}"
    else:
        out = outcome(p, lambda: arm.run(p, steps=1, units=units))
    names = {ft.REG_TORQUE_ENABLE: "Torque_Enable", ft.REG_GOAL_POSITION: "Goal_Position"}
    out += f" | {len(s.frames)} frames, SYNC_WRITE {[names.get(f[5]) for f in s.frames if f[4] == ft.SYNC_WRITE]}"
    if any(f[4] == ft.SYNC_WRITE and f[5] == ft.REG_GOAL_POSITION for f in s.frames):
        counts, back = read_back(s, mode)
        out += f" | goal {counts}; LeRobot {mode} minus {target:g}: {[round(v - target, 1) + 0.0 for v in back[:5]]}"
    arm.close()
    return out


def sim(steps, p):
    world = Simulation()
    arm = Robot("so101", sim=world)
    world.camera("front", size=(64, 48), attach=arm)
    try:
        return outcome(p, lambda: arm.run(p, steps=steps))
    finally:
        world.close()


ROWS = {
    "1 a degrees recording at +90, no norm=": lambda: real(),
    "2 the same, norm=DEG": lambda: real(norm=DEG),
    "3 a range recording at +90, no norm=": lambda: real(mode="range"),
    "4 the same, semantics=RANGE": lambda: real(mode="range", semantics=RANGE),
    '5 degrees, semantics=""': lambda: real(semantics=""),
    "6 degrees, norm={}": lambda: real(norm={}),
    '7 degrees, units="rad"': lambda: real(units="rad"),
    "8 degrees, units=None": lambda: real(units=None),
    '9 degrees, units="lerobot"': lambda: real(units="lerobot"),
    "10 degrees, norm={'*': 'degrees'}": lambda: real(norm={"*": "degrees"}),
    "11 start(), no norm=": lambda: real(start=True),
    "12 a Hub repo id, no norm=": lambda: f"{real(spec='someone/so101-degrees-act')} | downloads asked: {len(HUB)}",
    "13 a simulated so101, no norm=, 3 steps": lambda: sim(3, Policy(CKPT, device="cpu")),
    "14 from_spec(<ckpt>), a spec string alone": lambda: real(policy=from_spec(CKPT)),
    '15 norm=DEG, then run(..., units="lerobot")': lambda: real(norm=DEG, units="lerobot"),
    "16 semantics=<the degrees label> with norm=<the range modes>": lambda: real(
        semantics="so101-degrees-gripper-percent-v1", norm={"*": "range_m100_100", "gripper": "range_0_100"}
    ),
}
for name, row in ROWS.items():
    print(f"{name}: {row()}", flush=True)
p = Policy(CKPT, device="cpu")
print(f"17 one Policy, on a simulated so101: {sim(1, p)}; then on the hardware so101: {real(policy=p)}")
print(f"18 row 1 with the default twin=True: {real(twin=True)}")
print(f"19 row 3 with norm=DEG, as a degrees default would read it: {real(mode='range', norm=DEG)}")

dev cc15d9a:

1 a degrees recording at +90, no norm=: ran 1 step(s) under so101-range-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3165, 3262, 2979, 3136, 3890, 2734]; LeRobot degrees minus 90: [10.2, 20.4, -1.6, 6.0, 72.0]
2 the same, norm=DEG: ran 1 step(s) under so101-degrees-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3049, 3030, 2998, 3068, 3071, 2734]; LeRobot degrees minus 90: [0.0, 0.0, 0.0, 0.0, 0.0]
3 a range recording at +90, no norm=: ran 1 step(s) under so101-range-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3165, 3262, 2979, 3136, 3890, 2734]; LeRobot range minus 90: [0.0, 0.0, 0.0, 0.0, 0.0]
4 the same, semantics=RANGE: ran 1 step(s) under so101-range-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3165, 3262, 2979, 3136, 3890, 2734]; LeRobot range minus 90: [0.0, 0.0, 0.0, 0.0, 0.0]
5 degrees, semantics="": ran 1 step(s) under so101-range-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3165, 3262, 2979, 3136, 3890, 2734]; LeRobot degrees minus 90: [10.2, 20.4, -1.6, 6.0, 72.0]
6 degrees, norm={}: ran 1 step(s) under radians-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3292, 3401, 3091, 3257, 4095, 3446]; LeRobot degrees minus 90: [21.4, 32.6, 8.2, 16.6, 90.0]
7 degrees, units="rad": ran 1 step(s) under so101-range-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3292, 3401, 3091, 3257, 4095, 3446]; LeRobot degrees minus 90: [21.4, 32.6, 8.2, 16.6, 90.0]
8 degrees, units=None: ran 1 step(s) under so101-range-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3292, 3401, 3091, 3257, 4095, 3446]; LeRobot degrees minus 90: [21.4, 32.6, 8.2, 16.6, 90.0]
9 degrees, units="lerobot": ran 1 step(s) under so101-range-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3165, 3262, 2979, 3136, 3890, 2734]; LeRobot degrees minus 90: [10.2, 20.4, -1.6, 6.0, 72.0]
10 degrees, norm={'*': 'degrees'}: ValueError: no semantics label for so101 with modes {'*': 'degrees'}; add one to semantics.json | 0 frames, SYNC_WRITE []
11 start(), no norm=: status done; result: ran 1 step(s) under so101-range-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3165, 3262, 2979, 3136, 3890, 2734]; LeRobot degrees minus 90: [10.2, 20.4, -1.6, 6.0, 72.0]
12 a Hub repo id, no norm=: RuntimeError: no network in this probe | 0 frames, SYNC_WRITE [] | downloads asked: 1
13 a simulated so101, no norm=, 3 steps: ran 3 step(s) under so101-range-gripper-percent-v1
14 from_spec(<ckpt>), a spec string alone: ran 1 step(s) under so101-range-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3165, 3262, 2979, 3136, 3890, 2734]; LeRobot degrees minus 90: [10.2, 20.4, -1.6, 6.0, 72.0]
15 norm=DEG, then run(..., units="lerobot"): ran 1 step(s) under so101-degrees-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3165, 3262, 2979, 3136, 3890, 2734]; LeRobot degrees minus 90: [10.2, 20.4, -1.6, 6.0, 72.0]
16 semantics=<the degrees label> with norm=<the range modes>: ran 1 step(s) under so101-degrees-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3049, 3030, 2998, 3068, 3071, 2734]; LeRobot degrees minus 90: [0.0, 0.0, 0.0, 0.0, 0.0]
17 one Policy, on a simulated so101: ran 1 step(s) under so101-range-gripper-percent-v1; then on the hardware so101: ran 1 step(s) under so101-range-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3165, 3262, 2979, 3136, 3890, 2734]; LeRobot degrees minus 90: [10.2, 20.4, -1.6, 6.0, 72.0]
18 row 1 with the default twin=True: ran 1 step(s) under so101-range-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3165, 3262, 2979, 3136, 3890, 2734]; LeRobot degrees minus 90: [10.2, 20.4, -1.6, 6.0, 72.0]
19 row 3 with norm=DEG, as a degrees default would read it: ran 1 step(s) under so101-degrees-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3049, 3030, 2998, 3068, 3071, 2734]; LeRobot range minus 90: [-9.2, -16.6, 1.7, -5.6, -40.0]

this PR:

1 a degrees recording at +90, no norm=: ValueError: Policy('<ckpt>') does not say which LeRobot norm mode its data was recorded in, and so101 is a real body: pass norm={'*': 'degrees' | 'range_m100_100', ...} (LeRobot's SO arms default to degrees since 0.4.4, range_m100_100 before) or semantics= (see units/semantics.json) | 0 frames, SYNC_WRITE []
2 the same, norm=DEG: ran 1 step(s) under so101-degrees-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3049, 3030, 2998, 3068, 3071, 2734]; LeRobot degrees minus 90: [0.0, 0.0, 0.0, 0.0, 0.0]
3 a range recording at +90, no norm=: ValueError: (the same message) | 0 frames, SYNC_WRITE []
4 the same, semantics=RANGE: ran 1 step(s) under so101-range-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3165, 3262, 2979, 3136, 3890, 2734]; LeRobot range minus 90: [0.0, 0.0, 0.0, 0.0, 0.0]
5 degrees, semantics="": ValueError: (the same message) | 0 frames, SYNC_WRITE []
6 degrees, norm={}: ValueError: (the same message) | 0 frames, SYNC_WRITE []
7 degrees, units="rad": ValueError: (the same message) | 0 frames, SYNC_WRITE []
8 degrees, units=None: ValueError: (the same message) | 0 frames, SYNC_WRITE []
9 degrees, units="lerobot": ValueError: (the same message) | 0 frames, SYNC_WRITE []
10 degrees, norm={'*': 'degrees'}: ValueError: no semantics label for so101 with modes {'*': 'degrees'}; add one to semantics.json | 0 frames, SYNC_WRITE []
11 start(), no norm=: status error; result: ValueError: (the same message) | 0 frames, SYNC_WRITE []
12 a Hub repo id, no norm=: ValueError: Policy('someone/so101-degrees-act') does not say which LeRobot norm mode its data was recorded in, and so101 is a real body: pass norm={'*': 'degrees' | 'range_m100_100', ...} (LeRobot's SO arms default to degrees since 0.4.4, range_m100_100 before) or semantics= (see units/semantics.json) | 0 frames, SYNC_WRITE [] | downloads asked: 0
13 a simulated so101, no norm=, 3 steps: ran 3 step(s) under so101-range-gripper-percent-v1
14 from_spec(<ckpt>), a spec string alone: ValueError: (the same message) | 0 frames, SYNC_WRITE []
15 norm=DEG, then run(..., units="lerobot"): ran 1 step(s) under so101-degrees-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3165, 3262, 2979, 3136, 3890, 2734]; LeRobot degrees minus 90: [10.2, 20.4, -1.6, 6.0, 72.0]
16 semantics=<the degrees label> with norm=<the range modes>: ran 1 step(s) under so101-degrees-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3049, 3030, 2998, 3068, 3071, 2734]; LeRobot degrees minus 90: [0.0, 0.0, 0.0, 0.0, 0.0]
17 one Policy, on a simulated so101: ran 1 step(s) under so101-range-gripper-percent-v1; then on the hardware so101: ValueError: (the same message) | 0 frames, SYNC_WRITE []
18 row 1 with the default twin=True: ValueError: (the same message) | 0 frames, SYNC_WRITE []
19 row 3 with norm=DEG, as a degrees default would read it: ran 1 step(s) under so101-degrees-gripper-percent-v1 | 6 frames, SYNC_WRITE ['Torque_Enable', 'Goal_Position'] | goal [3049, 3030, 2998, 3068, 3071, 2734]; LeRobot range minus 90: [-9.2, -16.6, 1.7, -5.6, -40.0]

Row 10 is the same on both: norm= must match a semantics.json entry exactly, gripper included, and a partial one is refused at contract.py:138, before the weights load or the bus is touched, but after :137 has read the checkpoint's config (a Hub id is downloaded there). Rows 15 and 16 are the same on both: units="lerobot" takes the registry's range mode over a declared degrees norm=, and semantics= wins over norm=. Row 19 is the same on both: a range recording run with a degrees norm=, as a degrees default would run every range checkpoint, lands -9.2, -16.6, +1.7, -5.6 and -40.0 off.

[2] Mesh.connect(), LeRobot env, from a checkout: HF_HUB_OFFLINE=1 PYTHONPATH=.:tests:tests/policy python units_mesh.py | grep -v "^Loading weights". Only Goal_Position writes are counted: the device's state stream reads its bus on its own schedule.

"""A LeRobot checkpoint run through Mesh.connect() on a so101 device (fake Feetech bus, its operator gate says yes)
and on a simulated so101 device, over the in-process mem:// transport. Units only.
LeRobot venv, from a checkout: HF_HUB_OFFLINE=1 PYTHONPATH=.:tests:tests/policy python units_mesh.py"""

import tempfile
import uuid
from pathlib import Path

import numpy as np
from test_drivers import make_driver
from tiny_act import save_tiny_act

from strands_robots import Policy, Robot, Simulation
from strands_robots.mesh import Mesh
from strands_robots.policy.lerobot import LeRobotPolicy
from strands_robots.robot.drivers import feetech as ft

LeRobotPolicy.act_chunk = lambda self, obs, *, instruction=None: np.tile([90.0] * 5 + [50.0], (10, 1))
CKPT = str(save_tiny_act(Path(tempfile.mkdtemp()) / "tiny-act-so101", 6))
url = f"mem://{uuid.uuid4().hex}"
dev, ctl = Mesh(url, tenant="units"), Mesh(url, tenant="units")
d, s = make_driver()
d.images = lambda: {"front": np.zeros((48, 64, 3), np.uint8)}  # a synthetic frame: no camera is opened
dev.mount(Robot("so101", driver=d, twin=False, confirm=lambda targets: True), device_id="real-arm", watchdog_ms=None)
world = Simulation()
arm = Robot("so101", sim=world)
world.camera("front", size=(64, 48), attach=arm)
dev.mount(arm, device_id="sim-arm", watchdog_ms=None)
for did in ("real-arm", "sim-arm"):
    ctl.discover(did, min_count=1, timeout=3.0)
    remote = ctl.connect(did, images=True)
    for tag, said in (("no norm=", {}), ("norm=DEG", {"norm": {"*": "degrees", "gripper": "range_0_100"}})):
        n = len(s.frames)
        try:
            out = f"ran {remote.run(Policy(CKPT, device='cpu', **said), steps=1).steps} step"
        except ValueError as e:
            out = f"ValueError: {str(e).replace(CKPT, '<ckpt>')[:72]}..."
        goals = sum(f[4] == ft.SYNC_WRITE and f[5] == ft.REG_GOAL_POSITION for f in s.frames[n:])
        print(f"{did} (is_real={remote.is_real}), {tag}: {out}; Goal_Position writes on the so101 bus: {goals}")
ctl.close()
dev.close()
world.close()

dev cc15d9a:

real-arm (is_real=True), no norm=: ran 1 step; Goal_Position writes on the so101 bus: 1
real-arm (is_real=True), norm=DEG: ran 1 step; Goal_Position writes on the so101 bus: 1
sim-arm (is_real=True), no norm=: ran 1 step; Goal_Position writes on the so101 bus: 0
sim-arm (is_real=True), norm=DEG: ran 1 step; Goal_Position writes on the so101 bus: 0

this PR:

real-arm (is_real=True), no norm=: ValueError: Policy('<ckpt>') does not say which LeRobot norm mode its data was recor...; Goal_Position writes on the so101 bus: 0
real-arm (is_real=True), norm=DEG: ran 1 step; Goal_Position writes on the so101 bus: 1
sim-arm (is_real=True), no norm=: ValueError: Policy('<ckpt>') does not say which LeRobot norm mode its data was recor...; Goal_Position writes on the so101 bus: 0
sim-arm (is_real=True), norm=DEG: ran 1 step; Goal_Position writes on the so101 bus: 0

The question's alternative. This PR plus a refusal in server_config before the checkpoint is read (ahead of contract.py:137):

    if not semantics and not norm:  # the alternative: every deployment declares its units, sim or real
        raise ValueError(f"{checkpoint}: pass norm= or semantics=")

pytest tests/policy/test_lerobot.py -rA, LeRobot env: 8 failed, 2 passed. The two that pass are test_a_real_arm_runs_a_checkpoint_only_once_its_norm_mode_is_declared and test_unknown_semantics_are_refused_naming_the_table; the in-process sim run (test_a_local_checkpoint_drives_the_arm_through_run), test_server_config_is_built_from_the_robot_and_the_checkpoint and the six tests that serve the checkpoint fail.

Merges. git merge-tree --write-tree of dc83ef3 with #4794 (fe59b12), #4795 (0d37932), #4796 (2805bfa), #4797 (06e97cf), #4798 (b72cb68), #4799 (1e0a3a3), #4800 (14b2bb1), #4801 (3c2a427), #4802 (6c6ba4c), #4803 (f439675), #4790 (b4b0da9) and #4784 (73f8102), one at a time, heads by GET at 22:50 UTC: no conflict, and none of them touches this PR's three files. On each merged tree (a git archive of it), pytest tests/test_layout.py: 4 passed, the largest files at 250 lines, none of them this PR's (local.py has 94); and, LeRobot env, pytest tests/test_units_lerobot_modes.py tests/policy/test_lerobot.py: 17 passed.

Checks, from the checkout at dc83ef3, default env unless named.

  • ruff check --no-cache strands_robots tests tests_integ: All checks passed!. ruff format --no-cache --check strands_robots tests tests_integ: 188 files already formatted. mypy strands_robots: Success: no issues found in 123 source files. lint-imports --no-cache: Contracts: 8 kept, 0 broken. local.py grows from 87 to 94 lines, tests/policy/test_lerobot.py from 170 to 193 and tests/test_units_lerobot_modes.py from 48 to 62; pytest tests/test_layout.py: 4 passed.
  • pytest -q -p no:cacheprovider --timeout=300 -rs --deselect tests/mesh/test_mesh_multiprocess.py::test_two_devices_and_a_controller_discover_each_other_zero_config_then_a_kill_leaves: 647 passed, 32 skipped, 1 deselected. The deselected test opens a bare zenoh://, which scouts by multicast on the LAN (mesh/transports/zenoh.py:3). Every skip is an environment one: Isaac 16, Newton 5, LeRobot 10, the opt-in live model 1.
  • The LeRobot set, LeRobot env: pytest tests/policy tests/test_policy_factory.py tests/test_units_lerobot_modes.py tests/test_units_lerobot_calibrated.py tests/recording/test_record_checkpoint_units.py: dev 36 passed, this PR 40 passed. Only this PR's tests run a LeRobot checkpoint on a hardware Robot: the fake-bus tests in tests/test_policy_factory.py and tests/test_units_lerobot_calibrated.py use their own policies, and tests/recording runs checkpoints on simulated arms (code reading).

Docs (code reading). No example on dev or in #4781 (f70a53a) runs Policy("<checkpoint>") in-process on a hardware robot: docs/index.md:43 runs Policy("lerobot/smolvla_base") on Robot("so101"), and notebooks/02_policies.ipynb its checkpoints on Robot("so101"), both simulated. #4781's "The unit conversion is automatic: the policy says what it speaks, the robot converts." (docs/robots.md:31-32) holds on a real arm once norm= or semantics= says what the policy speaks.

Refs #3818

…ts norm mode

A checkpoint stores no norm mode, so Policy("<ckpt>") took the
registry's so101 label, so101-range-gripper-percent-v1, while the
pinned LeRobot defaults SO arms to degrees (use_degrees=True since
0.4.4). On the fake Feetech bus a degrees checkpoint's +90 deg landed
+20.4 deg off on shoulder_lift and +72.0 deg off on the full-turn
wrist_roll. On a real body LeRobotPolicy.attach now refuses a
checkpoint given neither norm= nor semantics=, before LeRobot is
imported and before anything reaches the bus, and the error names both
options and both modes. An empty one says no mode either:
semantics="" fell back to the registry's label and norm={} to radians.
A simulated body keeps the registry default.
@rmncardoso
rmncardoso marked this pull request as ready for review October 18, 2026 23:07

@yinsong1986 yinsong1986 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Adds a fail-closed guard at the top of LeRobotPolicy.attach: when the robot is a hardware body (Robot.is_real, i.e. port=/driver= given, which also covers Mesh.connect() robots) and the policy was built with neither a non-empty norm= nor a non-empty semantics=, it raises ValueError naming both options and both SO norm modes. The check sits before lerobot() is imported, before server_config resolves (and for a Hub id, downloads) the checkpoint, and before bind reaches units/requirements, so nothing touches the bus. A simulated body keeps the registry default, so existing sim tests and docs are unchanged. The _kw dict always carries both keys, every attach caller (run/loop.py, run/lockstep.py via bind) passes a Robot, and Description.load is not cached, so the early-return on contract.description is robot.description cannot let a sim-attached policy slip onto a different real body. No new public API, wire format or persisted schema; the behaviour change on hardware is an explicit refusal with the migration spelled out in the message, not a silent default flip. Changelog fragment is well-formed (one ###, no ##).

What's good

  • Guard placement is the cheapest possible point: zero network, zero torch import, zero bus frames on refusal, and the default-env test proves it without the lerobot extra.
  • tests/test_units_lerobot_modes.py covers the three undeclared shapes ({}, semantics="", norm={}) and asserts s.frames == []; the LeRobot-env test asserts both contract is None and _runner is None on refusal, then that each declared form actually drives a Goal_Position sync write.
  • Scope discipline: the served path (server_config / ZenohPolicy) is explicitly left alone and called out as a question rather than quietly widened.

Verification suggestions

  • The LeRobot-env test needs the remote extra: hatch run lerobot:test tests/policy/test_lerobot.py tests/test_units_lerobot_modes.py (expect 17 passed). I could only run the default-env side here (tests/test_units_lerobot_modes.py: 7 passed; tests/test_layout.py, tests/test_public_surface.py, tests/test_package.py: green; ruff check/ruff format --check/mypy clean on the three changed source/test files).
  • python scripts/assemble_changelog.py --check flags only the pre-existing 0000-... placeholder on dev, not this PR's 4804-... fragment.

@cagataycali

Copy link
Copy Markdown
Member

@yinsong1986 your approval is newer than the last push, call-test-lint is green and the PR is mergeable: ready to squash.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants