From the Hugging Face Hub to robot hardware with Strands Agents and LeRobot

Back to Articles From the Hugging Face Hub to robot hardware with Strands Agents and LeRobot Enterprise Article Published June 17, 2026 Upvote 3 Sundar Raghavan rsundaraws Follow amazon Cagatay Cali cagataydev Follow amazon A walkthrough of the LeRobot integration in Strands Robots - one agent loop, from a Hub dataset to a physical robot, with sim-to-real datasets in the same on-disk format and policies you swap with a string. You have a robot, a folder of demonstration data on the Hugging Face Hub, and a new task you want it to learn. Today that takes five separate tools: one to record new demonstrations, another to train, a third to test in simulation, custom code to deploy on hardware, and yet another to coordinate when you have more than one robot. The pieces work on their own. They don't talk to each other. Strands Robots is an open source SDK from AWS (Apache 2.0) that exposes robot abstractions, simulation, and the LeRobot stack as AgentTools that you compose into a single Strands agent. The integration is deliberately thin: LeRobot's own scripts handle hardware recording and calibration, and the Strands AgentTools come in for the parts an agent actually orchestrates. The simulation tool records LeRobotDatasets in the same format LeRobot writes on hardware. GR00T and LerobotLocal serve policy inference behind a common interface, and MolmoAct2 checkpoints run through the LerobotLocal path. A peer mesh fans the agent out to remote robots. The dataset format stays exactly as LeRobot wrote it; the agent loop is the glue. This post walks you through five steps inside a single agent: build the agent over the LeRobot AgentTools, record a demonstration as a LeRobotDataset in simulation, run a policy on the same robot, deploy the same agent code to a physical SO-101 with one keyword argument change, and broadcast commands across a fleet over the Zenoh mesh. At the end, you can clone the working sample application from GitHub and run it on your laptop in simulation. No hardware, no GPU, no Hugging Face credentials needed for the default path. The runnable companion to this post lives at examples/lerobot/hub_to_hardware.py and hub_to_hardware.ipynb. The notebook is sim-only and Mock-policy by default. What you'll build The Strands Robots SDK exposes the LeRobot stack as AgentTools that you compose into one Strands agent. The example agent in this post does four things: record new demonstrations in simulation, push the result to the Hub as a LeRobotDataset, run a policy in simulation against that same format, and deploy the same agent code to a physical robot with one keyword argument change. When you have more than one robot, the agent can coordinate the whole fleet through a built-in peer mesh. For hardware recording and calibration, LeRobot's own CLIs (lerobot-record, lerobot-calibrate) handle the bring-up; the agent picks up from there. Figure 1. Robot("so100") defaults to a MuJoCo-backed simulation; mode="real" returns a hardware robot driven by LeRobot. Both modes share the same DatasetRecorder and the same policy providers, so a dataset captured in sim and a dataset captured on hardware use the same on-disk LeRobotDataset format. Two design choices make this work. First, Robot("so100") returns a simulation by default (no hardware, no risk), and mode="real" returns a hardware-backed robot driven by LeRobot. The agent code is identical across both modes. Second, the DatasetRecorder that writes a LeRobotDataset is shared between the simulation path and LeRobot's own hardware recording, so a dataset captured in MuJoCo and one captured from a physical SO-101 are in the same format. The whole workflow in five lines of Python: from strands_robots import Robot from strands import Agent arm = Robot("so100") # mode="sim" (default - safe, no hardware) agent = Agent(tools=[arm]) agent("Pick up the red cube") What follows is what's actually happening inside that call, step by step. Prerequisites Minimal (default simulation path) Python 3.12+, on Linux or macOS (Apple Silicon supported for the MuJoCo backend). A Strands-compatible model provider for the agent's reasoning. Amazon Bedrock with AWS credentials, the Anthropic API, OpenAI, or Ollama running locally. Strands Robots installed with the install extras: uv pip install "strands-robots[sim-mujoco,lerobot,mesh]" That's it. The example in this post runs end-to-end on a laptop with these three. Advanced (hardware deployment, real policies, Hub push) A Hugging Face account and token with write permission, for pushing datasets and pulling policy checkpoints from the Hub. For the hardware path: an SO-101 follower and leader pair, or any other LeRobot-supported robot. Both devices need calibration files under ~/.cache/huggingface/lerobot/calibration/. For local GR00T inference: an NVIDIA GPU with at least 16 GB of video memory and Docker installed. The post uses the gr00t_inference tool's lifecycle="full" action, which pulls the image, downloads a checkpoint, and starts the container in one call. Step 1 - Set up the example Install Strands Robots and get the example files: uv pip install "strands-robots[sim-mujoco,lerobot,mesh]" git clone https://github.com/strands-labs/robots.git cd robots Export your Hugging Face token if you want the agent to push datasets or pull policies from the Hub. This is optional for the default simulation path in this post; the example runs end-to-end with the Mock policy and writes the dataset to your local cache without needing Hub access. export HF_TOKEN=hf_... The runnable example lives at examples/lerobot/hub_to_hardware.py (Python script) and hub_to_hardware.ipynb (notebook), in the strands-labs/robots repository alongside the MuJoCo and LIBERO examples. The notebook is the recommended starting point: open it in JupyterLab and run cells top-to-bottom in simulation mode without any hardware connected. Step 2 - Record demonstrations and push to the Hub The simulation tool records LeRobotDatasets in the same format LeRobot writes on hardware. No hardware required. The Simulation tool's start_recording action writes through the same DatasetRecorder class: same parquet schema for joint states and actions, same per-camera MP4 layout. The agent prompt is almost identical: from strands import Agent from strands_robots import Robot robot = Robot("so100") # mode="sim" by default agent = Agent(tools=[robot]) agent( "Record a demonstration of 'pick the red cube and place it in the box' " "using the Mock policy provider at FPS 30. Write the dataset to " "my_user/cube_picking_sim and push to the Hub when done." ) Figure 2. The recording scene in MuJoCo simulation: the SO-100 arm reaching toward a red cube on the ground plane, captured to a LeRobotDataset. No hardware, no GPU, no Hugging Face credentials needed for this default path. The Mock policy is intentional: it generates placeholder joint actions so the workflow runs end-to-end without a trained checkpoint. The robot moves through random motions rather than completing the grasp, and the recording is structurally complete (valid joint states, valid camera frames, a well-formed LeRobotDataset episode), but the demonstration itself isn't useful as training data. Step 3 below swaps in GR00T or LerobotLocal for real grasping behavior. To see actual cube-picking in this step, run --policy lerobot_local --checkpoint allenai/MolmoAct2-SO100_101 (a MolmoAct2 checkpoint, auto-detected from its config.json and routed through the LerobotLocal path); the prompt, dataset format, and agent code stay the same. The proof is what happens next. LeRobot's own dataset loader reads the sim-recorded data with no Strands-specific code path: from lerobot.datasets.lerobot_dataset import LeRobotDataset dataset = LeRobotDataset("my_user/cube_picking_sim") print(dataset.features) # {'observation.state': Sequence(...), # 'observation.images.front': VideoFrame(...), # 'action': Sequence(...), # 'episode_index': Value(...), 'frame_index': Value(...), ...} This feature