From Reasoning to Action: Benchmarking Frontier AI Models on LIBERO-10 Robot Manipulation Tasks

Run frontier LLMs (GPT, Claude, and other OpenAI-compatible models) as closed-loop control policies on the LIBERO-10 manipulation benchmark.

UpdatedOctober 15, 2026

Model rollout results

How the evaluation works

1. ObserveAgent-view and wrist-camera images from the two most recent observations, plus the robot state.
2. DecideThe LLM calls one tool: move_to, done, or give_up.Output limit per call, including reasoning: 3,072 tokens for Low and Medium; 8,192 for High. GPT-6 Luna High used 3,072.
3. ActAn adapter turns the target pose into controller steps.Limit: 80 calls or 900 sim steps (45 s at 20 Hz)
4. ScoreLIBERO's checker confirms success. Rate = successes ÷ 100 planned episodes; stopped, incomplete, errored, or missing runs count as failures.Logged per episode: outcome, steps, calls, tokens, latency, cost

Success, costs, and calls

Individual states

Click a row to inspect its rollout and details.