Claw machine
How can we bridge frontier intelligence and robot actions?
Five interfaces for robot control, distinguished by what the model outputs and how it becomes physical action.
Two bridges between reasoning and execution
01Learn from frontier demonstrationsGenerate Train Execute
02Learn to follow richer instructionsAnnotate Understand Execute
Frontier guidanceFast local control
Bridge 1 · Autonomous demonstration generation
Autonomous rollouts via in-context learning
We provide five human demonstrations, then let the model control the robot.
Stack cups
Fold a cloth
Close a lid
Successes out of 10 independent trials per model and task.
Successful examples alone are not enough for precise contact.
Show the model how to correct errors
Corrective demonstration segments teach small adjustments when execution goes wrong.
With corrective examples, autonomous generation succeeds more often across all four tasks.
Generation efficiency with accumulated examples
Reuse successful segments. Spend less time and API cost on the next rollout.
Frontier guidance. Faster local execution.
51.25% 65%Success: local policy → harness
40–60% less timeVersus direct frontier control, on successful trials
Bridge 2 · Dense language supervision
One trajectory. Many ways to guide it.
Teach the local policy at every scale, with varied language.
From task to movement
- Composite · Task
Open the cabinet, transfer the fruit, close the cabinet.
- Atomic · Subtask
Transfer the apple into the bowl.
- Primitive · Movement
Reach → grasp → lift → carry → lower → release → retract.
Different words, the same goal
- “Put the apple in the bowl.”
- “Move the apple from inside the cabinet into the bowl on the counter.”
- “Take the apple out of the cabinet and pop it into the bowl.”
Fine-grained commands. More reliable execution.
Compliance with individual movement instructions.
“Lift the object up”
“Move the arm down”
“Move the arm back”
Illustrative demonstrations · Separate compliance evaluation: 36 trials per command, one training seed.
Across planners and phrasings
Highest reported success across all three tested planners, with original and rephrased instructions.
Opus 5
48.8%→47.1%
Gemini-3.1-Pro
46.7%→45.4%
Fable 5
49.2%→48.3%
Original → rephrased success rate · Hierarchical + multi-aspect supervision · RoboCasa 365, seed 42.
Compare all supervision settings
Both bridges together
Reason, place, and build a physical crossword
Frontier planning and local-policy execution on the physical crossword task.
Demo preview · full video coming soon
Changing the instruction
“Put the cube on 1A.”
“Put the cube to the left of the B cube.”
Dense supervision supports varied spatial descriptions.
More time acting. Less time waiting.
Direct control2m 40s
Frontier harness9m 8.55s
3.4×time executing actions
Cumulative action time over one hour · one run per configuration.
BibTeX
@misc{kim2026bridging,
title = {Bridging Frontier Reasoning and Robot Execution:
From Autonomous Demonstration Generation to Dense Language Supervision},
author = {Kim, Bosung and Trevithick, Alexander and Wang, Ruiyi
and Ammanabrolu, Prithviraj},
year = {2026},
eprint = {2610.03615},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.03615}
}