[cagataycali]

Part 8 of #4749 · roadmap #3818

In one paragraph

Before this, giving an agent a robot meant hand-writing a tool spec and keeping it in sync with the API by hand. Now robot.tool and sim.tool are each a single strands AgentTool generated from the device's own verbs: one rpc selector plus one parameter object per rpc, whose JSON schema is read straight from each method's Python signature (an ast pass, so nested generics and unions are exact and a parameter that cannot cross a wire is left out or, if required, refused). A new rpc is a new tool action with nothing hand-written. A command from a tool is treated as an agent's: inside from_agent() an act is dispatched through confirm_act, so the safety chain and the operator gate run on every step, including a background run/start whose context travels to its thread; resume and calibrate stay the operator's and a model never sees them. strands is imported only when a tool is built (the [tools] extra).

What it looks like

from strands import Agent
from strands_robots import Robot

arm = Robot("so101")
agent = Agent(model=model, tools=[arm.tool])
agent("Observe the robot, then set shoulder_pan to 0.5 rad with steps=200.")

Rendered headless in MuJoCo as agent-style act tool calls land through the gated dispatch path; shoulder_pan swings across 0.9 rad each cycle.

Two so101 arms driven in lock-step by one sim.tool run call per frame ({"a": ..., "b": ...}), rendered headless in MuJoCo. Measured in the same run: pans 0.603 / -0.603 rad after 1.6 s; the cube dropped from z 0.40 and describe() and pose("cube") both read z 0.025.

Measured

gate	command	result
tool suite	pytest tests/tools	43 passed, 1 skipped
world verbs (live poses, typed vectors, lock-step by id)	pytest tests/tools/test_world_verbs.py	13 passed; 11 of them red on the previous head
whole suite on current dev	MUJOCO_GL=egl pytest tests	439 passed, 22 skipped
real model turn (Bedrock, Sonnet 4.5)	STRANDS_ROBOTS_LIVE=1 pytest tests/tools/test_live_model.py	1 passed; observe then act, shoulder_pan 0.0000 to 0.4692 rad
operator refusal	pytest tests/tools/test_tool_gate.py	a real robot's act with the operator saying no raises RequestRefused, zero bytes on the fake bus
run loop + recording unaffected	pytest tests/test_run.py tests/recording tests/test_operator_gate.py	green
lint / mypy / import-linter	ruff check; ruff format --check; mypy strands_robots; lint-imports	clean (7 contracts kept)
layout	pytest tests/test_layout.py	loop.py 204 lines (<=250), lockstep moves to run/lockstep.py

tool spec: 7 actions on robot.tool, 14 on sim.tool.

Under the hood

Landed from the reference robots-v1-2 (part 8's files per its SNAPSHOT.md): tools/{schema,spec,wire,world,call,agent}.py, robot/run/source.py, tests/tools/. What differs from the reference: the reference's run loop had been reshaped around an earlier dev; here the gate and the from_agent context are added to the loop as it stands on dev (events, record=), and lockstep is split into run/lockstep.py so loop.py stays under the 250-line layout cap. Robot.run/Robot.start become the loop's own functions (robot first) so the tool spec reads their full signatures; a policy spec string is accepted wherever a policy is.

The real turn's tool calls:

robot_so101  {"rpc": "observe", "observe": {"images": true}}
robot_so101  {"rpc": "act", "act": {"action": {"shoulder_pan": 0.5}, "steps": 200}}

Files: 7 new in strands_robots/tools/, robot/run/{source,lockstep}.py; robot.py, simulation.py, run/loop.py, run/__init__.py, policy/callable.py, pyproject.toml touched. Package ~430 lines, tests ~550, in line with the reference module.

Refs #3818

[yinsong1986]

Summary

Adds strands_robots/tools/ and the robot.tool / sim.tool properties: one strands PythonAgentTool per device whose spec is derived from the manifest's rpcs (describe, observe, act, run, stop, hold, estop; resume/calibrate are refused with PermissionError before any lookup) and, for the world, from Simulation's public methods whose parameters all cross the wire. Schemas come from an ast pass over each signature, so stop_when/stop_event fall out and policy becomes a spec string (as_policy now accepts str | PathLike via the factory). Dispatch rewrites an agent's act to confirm_act and validates arguments against the declared rpc first, so confirmed=/confirm= from the model are rejected rather than honoured (decision 7 of #4749). from_agent() is a ContextVar the run loop reads once per _loop, choosing confirm_act for every step; start copies the context into the Task thread so a background run is gated the same way. lockstep moves to run/lockstep.py and Robot.run/Robot.start become the loop functions themselves, so the manifest and the tool see the full keyword set with no **kw. strands is imported only inside tools/agent.py behind the new [tools] extra. The diff matches the title and description; the drawings/changelog fragment are in place.

What's good

• Gate semantics are pinned by tests on a fake bus: refused act writes zero goal frames; confirmed=True from the model is rejected as an unknown argument of act; an agent run with confirm= asks once per step; calibrate/resume put no byte on the bus.
• Argument validation runs against handler(device, rpc) (the declared verb) before anything moves, which is what keeps the confirm_act-only keywords off the model's surface.
• start() lost its **kw sink and spells every run keyword; test_robot_run_has_the_run_loops_own_signature... keeps the two in lockstep.
• Lazy strands import verified in a fresh interpreter; import-linter contracts updated and kept (7/7); mypy, ruff clean; full unit suite green (367 passed, 47 skipped) on a fresh env with [dev,mujoco,tools]; scene.py --all --verify reports no drift.

Verification suggestions

• tests/tools/test_sim_tool.py::test_an_agent_builds_a_scene_steps_and_renders_an_image_block needs an offscreen GL backend on a display-less box (MUJOCO_GL=egl or osmesa); with the default GLFW it fails on X11: Failed to open display, which is the environment, not the PR.
• Spot-check the generated run schema and confirm it is intended that record and push are model-callable: python -c "from strands_robots import Robot; from strands_robots.tools import tool_spec; import json; print(json.dumps(tool_spec(Robot('so101'))['inputSchema']['json']['properties']['run']['properties'], indent=1))". Today a tool call can run(policy=..., record='hf://owner/name', push=True) with the process's ambient HF credentials and no gate on that path (the operator gate covers motion only). If that is not the intent, parameters(..., exclude=) already exists in tools/schema.py and is currently unused.

[cagataycali]

Merged dev (one conflict, run/loop.py: lockstep stays in run/lockstep.py with dev's b.realtime) and added c8f0554: describe()/pose() read live poses, vectors are typed on the wire, run takes robot ids from sim.tool, randomize ranges, reset keyframe check, remove objects/cameras. Pinned by tests/tools/test_world_verbs.py (11 red before). Full suite 439 passed. Re-requesting review.

[yinsong1986]

Summary

Adds strands_robots/tools/ and the robot.tool / sim.tool properties: one strands PythonAgentTool per device whose input schema is {rpc: <enum>, <rpc>: {args}}, generated by an ast pass over each verb's signature (tools/schema.py), so stop_when/stop_event/callables fall out, Action becomes a per-joint number object, and policy becomes a spec string (as_policy now routes str | PathLike through the factory). Dispatch (tools/call.py) validates the model's arguments against the declared rpc first and only then swaps act for confirm_act, so confirmed=/confirm= from a model are refused as unknown arguments rather than honoured; record/push are refused as developer-only; resume/calibrate are never offered; hold on a body with a gate asks the operator before torque goes on. from_agent() is a ContextVar the run loop samples once per _loop/_lockstep to choose confirm_act, and start() copies the context into the Task thread. lockstep moves to run/lockstep.py, Robot.run/Robot.start are the loop functions themselves (no **kw sink), and the world gains pose(), live positions in describe(), remove() of objects/cameras, typed vectors via geometry.vec, and run({id: spec}). strands is imported only inside tools/agent.py behind the new [tools] extra. The diff matches the title and description; changelog fragment and drawings are in place.

Reviewed the PR-specific diff at 67511f44; head befc3f1d is that commit plus a dev merge (#4788) with no PR-specific changes. Spot-checked locally: tests/tools 40 passed / 1 skipped; whole tests/ green apart from Isaac EULA-gated environment errors; import strands_robots pulls neither strands nor PIL; every new module imports alone in a fresh interpreter; the three prior-review threads (record/push model-callable, CodeQL cyclic-import notes) are resolved in this head or answered on the thread.

What's good

• The operator gate is enforced structurally, not by trust: argument validation runs against handler(device, rpc) before target()'s confirm_act substitution, so no tool argument can stand in for the operator (pinned on a fake bus in tests/tools/test_tool_gate.py with zero goal frames on refusal).
• Tool errors are reported to the model as status: error text, never swallowed into a default action (AGENTS.md > Key Conventions 5/6).
• start() lost its **kw sink so the manifest and the tool see the full keyword set, and contextvars.copy_context() keeps a background run gated.
• reset(seed=) is removed loudly (TypeError), not silently; no in-tree caller.

Verification suggestions

• The description says 12 actions on sim.tool; the generated list is 14 (add_robot, add_object, camera, contacts, control_step, describe, pose, randomize, remove, render, reset, run, set_pose, step). Print it with python -c "from strands_robots.simulation import Simulation; from strands_robots.tools.world import WorldDevice; print([r.name for r in WorldDevice(Simulation()).manifest.rpcs])" and confirm contacts/control_step are meant to be model-callable.
• hold through the tool feeds the measured pose to the full safety chain, so on a real arm resting slightly past a declared joint limit it is refused by JointLimitValidator rather than holding where it is; worth a spot-check on hardware that this is the intended fail-closed posture.

[yinsong1986]

Summary

Adds strands_robots/tools/ and the robot.tool / sim.tool properties: one strands PythonAgentTool per device whose input schema is {rpc: <enum>, <rpc>: {args}}, generated by an ast pass over each verb's signature (tools/schema.py) so callables/events fall off the wire, Action becomes a per-joint number object, and a policy becomes a spec string (as_policy now accepts str | PathLike via the factory). Dispatch (tools/call.py) refuses resume/calibrate with PermissionError before any lookup, refuses record/push by name, validates arguments against the declared rpc before swapping act for confirm_act, and gates hold through gate() so torque never comes on without the operator. run/source.py adds a from_agent() contextvar that the run loop reads to pick confirm_act over act on every step, and start() copies the context into the Task thread. Robot.run/Robot.start become the loop's own functions (explicit keywords, no **kw), lockstep moves to run/lockstep.py, and the world grows pose(body), live poses in describe(), vec()-validated fixed-length vectors, remove of objects/cameras, randomize returning what it drew, and reset(keyframe=) refusing an unknown keyframe (seed= dropped, loudly). strands is imported only inside .tool.

What's good

• The gate semantics hold on every path a model can reach: act -> confirm_act, run gated per step via is_agent(), background start carries the context to its thread, hold gated, resume/calibrate never resolved, confirm/confirmed refused as tool arguments. sim.tool cannot touch hardware (lockstep rejects a robot not in the world; add_robot exposes placement args only).
• Checked locally against the head tree: every new module imports alone in a fresh interpreter, pytest tests/tools is green (40 passed, 1 skipped), and the spec probes above confirm 7 robot actions / 14 world verbs with close, add, manual_time, default, unique_id and the tool property itself excluded.
• Scene edits keep the rollback contract (_grow pops on a failed rebuild), and malformed model inputs (../ names, missing files, negative step, a list for policies) come back as error text rather than escaping the tool.

Verification suggestions

• The PR is currently DIRTY against dev: #4789 landed after the merge base and edits the except PolicyFault clauses in both _loop and the old in-file _lockstep to also catch NonFiniteTarget. When resolving, make sure the catch lands in run/lockstep.py (where _lockstep now lives) as well as run/loop.py, then re-run pytest tests/tools tests/test_run.py tests/test_operator_gate.py.
• Optional spot-check of the thread hand-off: with from_agent(): task = real.start(policy, steps=3) on a fake-bus robot with confirm=None should raise RequestRefused at task.stop() and leave zero goal frames on the bus (covered by test_a_background_start_inside_an_agent_call_carries_the_gate_to_its_thread).

[yinsong1986]

Summary

Adds strands_robots/tools/ and the robot.tool / sim.tool properties: one strands PythonAgentTool per device whose input schema is {rpc: <enum>, <rpc>: {args}}, generated by an ast pass over each verb's signature (tools/schema.py) so callables and events fall off the wire, Action becomes a per-joint number object and policy a spec string (as_policy now routes str | PathLike through factory.make). tools/call.py validates a model's arguments against the declared rpc before swapping act for confirm_act, refuses resume/calibrate outright, refuses record/push by name, and gates hold through gate(). run/source.py adds the from_agent() contextvar the loop samples once per _loop/_lockstep, and start() copies the context into its Task thread. Robot.run/Robot.start become the loop functions themselves (explicit keywords, no **kw), lockstep moves to run/lockstep.py, and the world grows pose(), live poses in describe(), vec()-typed vectors, remove of objects/cameras, randomize returning what it drew, reset(keyframe=) refusing an unknown keyframe (seed= dropped loudly), and run({id: spec}). strands is imported only inside tools/agent.py behind the new [tools] extra. Diff matches the title and description; changelog fragment and drawings are in place. Head ac8a5de0 is a single commit on top of current dev (#4789 included; the NonFiniteTarget catch is present in run/lockstep.py as well as run/loop.py).

What's good

• The motion gate is structural: argument validation runs against handler(device, rpc) first, so confirm=/confirmed= from the model are unknown arguments rather than honoured; a refused act, run, start or hold on a fake-bus robot puts zero goal frames on the bus (tests/tools/test_tool_gate.py).
• is_agent() is read once per loop and contextvars.copy_context() carries it into the Task thread, so a background run is gated the same as a foreground one.
• Checked on the head tree: every new module imports alone in a fresh interpreter (robot.run.loop, robot.run.lockstep, robot.robot, tools, tools.agent, simulation), pytest tests/tools 40 passed / 1 skipped, the rest of tests/ green apart from 5 torchcodec shared-library failures that are the box, not the PR; ruff check, ruff format --check clean; mypy clean apart from a pre-existing yaml stub note in policy/lerobot/contract.py.
• Adversarial sim.tool inputs (size=[0,0] camera then render, negative/str step, negative object size, /etc/...xml as a robot name, a non-MJCF path, reversed randomize range, unknown keyframe/body) all come back as status: error text; nothing escapes the tool and nothing segfaults.

Verification suggestions

• The inline comment is about the run.policy spec on a hardware robot; the shape is reproducible without a server: python -c "from strands_robots.policy.factory import make; print(make('zenoh://203.0.113.9:7447/x'))" (with the [lerobot] extra) shows the form is accepted with no gate on the host, and strands_robots/tools/call.py::dispatch never looks at the value.

[yinsong1986]

Summary

Adds strands_robots/tools/ and the robot.tool / sim.tool properties: one strands PythonAgentTool per device whose input schema is {rpc: <enum>, <rpc>: {args}}, generated by an ast pass over each verb's annotations (tools/schema.py) so callables and events fall off the wire, Action becomes a per-joint number object and a policy a spec string (as_policy now routes str | PathLike through factory.make). tools/call.py::dispatch is the single choke point: resume/calibrate are refused as operator-only, record/push and a policy naming a network address are refused as the developer's, act is rewritten to confirm_act, hold is gated at the measured pose, and the whole call runs under from_agent() (a ContextVar) so run's loop and lockstep write through confirm_act on every step, with start copying the context into its thread. Robot.run/Robot.start become the loop's own functions so the spec reads their full signatures; lockstep moves to run/lockstep.py; Simulation gains pose(), live poses in describe(), remove of objects and cameras, a Literal shape enum, vec()-validated vectors, randomize returning its draws, reset(keyframe=) validation (dropping the unused seed=, which now fails loudly), and run() keyed by robot id. strands and pillow form the new [tools] extra, imported only when a tool is built.

Verified on the head commit: pytest tests 426 passed / 48 skipped (the tools suite 42 passed / 1 skipped), ruff and mypy clean, all 7 import-linter contracts kept, and no heavy module in sys.modules after import strands_robots. The earlier policy="zenoh://..." concern is closed by the REMOTE_SPEC check that runs before as_policy()/attach, pinned by test_an_agent_never_chooses_where_the_cameras_stream.

What's good

• The gate is structural rather than per-action: from_agent() + is_agent() at the write site means act, run, lockstep and a background start all inherit the operator gate without each tool action re-implementing it, and test_tool_gate.py pins it against a fake bus with zero goal frames written on refusal.
• The spec is derived, not maintained: test_a_manifest_change_changes_the_tool_with_no_tool_code_edit and the NotOnTheWire refusal for a required un-wireable parameter make the "new rpc = new action" claim testable, and test_the_models_own_claim_is_not_a_confirmation closes the obvious confirmed=True smuggling path.
• Lazy import is pinned in a fresh interpreter (test_tool_imports.py), so the [tools] extra stays optional for core users.
• Changelog fragment present; loop.py stays under the layout cap by splitting lockstep out rather than trimming it.

Verification suggestions

• The one gate CI cannot run: STRANDS_ROBOTS_LIVE=1 pytest tests/tools/test_live_model.py with Bedrock credentials, to confirm a real model still produces the observe then act ordering against the current spec wording.

[yinsong1986]

Summary

Adds strands_robots/tools/ and the robot.tool / sim.tool properties: one strands PythonAgentTool per device whose input is {rpc: <enum>, <rpc>: {args}}, with each rpc's parameter object generated by an ast pass over the method's annotations (tools/schema.py), so callables, events and live objects fall off the wire, Action becomes a per-joint number object and a policy becomes a spec string (as_policy now routes str | PathLike through factory.make). dispatch (tools/call.py) is the single choke point: resume/calibrate are operator-only, record/push developer-only, an agent's act is rewritten to confirm_act, hold on a gated robot asks the operator first, and a zenoh:// policy spec is refused before attach. from_agent() (run/source.py) is a ContextVar the run loop reads once to pick confirm_act over act, and start() now copies the context into the task's thread. Robot.run/Robot.start become the loop's own functions so the spec reads their full signatures; lockstep moves to run/lockstep.py and Simulation.run accepts ids as keys. On the world side, vec() in geometry.py makes every vector argument typed and length-checked, remove takes an object or camera, pose(body) and describe() report live poses, randomize returns what it drew, and reset drops seed and refuses an unknown keyframe. Tests cover the spec shape, the gate on a fake-bus SO-101 (act/run/start/hold/calibrate/resume), the record/push and network-address refusals, world verbs, and a scripted-model agent turn; tests/tools is green at the head (42 passed, 1 skipped).

What's good

• The gate test matrix is the right one for a model-in-front-of-hardware component: every refusal asserts zero goal frames on the fake bus, and confirmed=/confirm= from the model are rejected at decode because the declared act signature, not confirm_act, is what the arguments are checked against.
• start() carrying copy_context() into the Task thread closes the obvious hole where a background run would silently fall back to act.
• strands and PIL are imported only when a tool is built, pinned by a fresh-interpreter test, and the import-linter contracts are updated rather than loosened.

[cagataycali]

Head a3883c8b (dev @ 3ebd17a merged in): call-test-lint green. Corrected the body count you flagged: sim.tool is 14 actions, not 12 (contacts/control_step are read-only world queries, intentionally model-callable); robot.tool stays 7. Re-measured: pytest tests/tools 43 passed/1 skipped, test_world_verbs 13, live Bedrock turn green (observe then act). Squash auto-merge armed.

[yinsong1986]

Summary

Adds strands_robots/tools/ and the robot.tool / sim.tool properties: one strands PythonAgentTool per device whose input is {rpc: <enum>, <rpc>: {args}}, with each rpc's parameter object generated by an ast pass over the method's annotations (tools/schema.py) so callables, events and live objects fall off the wire, Action becomes a per-joint number object and a policy a spec string (as_policy now routes str | PathLike through factory.make). resume/calibrate are never offered, record/push are refused, and a policy spec naming a network address is refused both in dispatch and in bind() under from_agent(). Inside the tool an act becomes confirm_act, hold is gated on the measured pose, and run/lock-step write through confirm_act when is_agent(); start copies the contextvar into its thread. Robot.run/start become the loop's own functions (no **kw swallowing), lockstep moves to run/lockstep.py, Simulation gains pose(), live poses in describe(), remove() of objects/cameras, randomize() returning what it drew, reset(keyframe=) validation (and drops the unused seed= loudly), and run() keyed by robot id; vectors go through geometry.vec so a string or wrong-length vector is refused by name. strands is imported only in tools/agent.py on first .tool access, behind the new [tools] extra.

Both prior [MUST FIX] threads on tools/call.py (model-chosen remote policy spec via policy= and via policies={id: spec}) are resolved at this head: the refusal gathers both shapes before target() and bind() refuses the same under from_agent(), each pinned by a test with ZenohPolicy.attach spied. No new blockers found.

What's good

• Fail-closed by construction: dispatch validates against the declared rpc's signature before substituting the gated form, so a model's confirmed=True / confirm=True is refused as an unknown argument rather than honoured.
• _SceneOps.lock is an RLock, so the new nested _grow() under add()'s lock is safe; _grow rolls the scene back on a failed rebuild.
• At this head on a MuJoCo-only box: pytest tests/tools 43 passed / 1 skipped; pytest tests (Isaac files excluded, EULA-gated here) 661 passed / 5 skipped; ruff check, ruff format --check and mypy strands_robots clean.

Verification suggestions

• STRANDS_ROBOTS_LIVE=1 pytest tests/tools/test_live_model.py with Bedrock credentials, to confirm the generated schema is accepted end-to-end by a real model (the scripted-model tests do not exercise provider-side schema validation, e.g. enum containing null alongside type: ["string", "null"]).
• Note the manifest's RPCS does not include start, so robot.tool exposes 7 actions (describe, observe, act, run, stop, hold, estop); the description's "run/start" wording refers to the from_agent() propagation, which test_a_background_start_inside_an_agent_call_carries_the_gate_to_its_thread covers directly.