s

* Equal Contribution       The University of Texas at Dallas      

IEEE International Conference on Robotics and Automation (ICRA), 2024
Arxiv Supplementary Code Scenes
Scene Replication Grasping Leaderboard Purchase Link

Abstract

We present a new reproducible benchmark for evaluating robot manipulation in the real world, specifically focusing on the task of pick-and-place. Our benchmark uses the YCB objects, a commonly used dataset in the robotics community, to ensure that our results are comparable to other studies. Additionally, the benchmark is designed to be easily reproducible in the real world, making it accessible for researchers and practitioners. We also provide our experimental results and analyses for model-based and model-free 6D robotic grasping on the benchmark, where representative algorithms for object perception, grasping planning and motion planning are evaluated. We believe that our benchmark will be a valuable tool for advancing the field of robot manipulation. By providing a standardized evaluation framework, researchers can more easily compare different techniques and algorithms, leading to faster progress in developing robot manipulation methods.

Scenes

20 scenes in our SceneReplica benchmark with 5 YCB objects in each scene

20 scenes in our SceneReplica benchmark with 5 YCB objects in each scene

Scene Replication

The process of replicating a scene in the real world. The reference scene image is overlaid to the real camera image to guide how to place objects into the real-world scene.

Grasping

medal LeaderBoard

Adding your results: There are two ways to add your results to the leaderboard. (1) Use the 20 scenes to run grasping experiments and then provide us videos of these experiments to verify your results. (2) Provide source code of your method and we will run grasping experiments for you. Please contact the authors if you are interested in.
# Perception Grasp Planning Motion Planning Control Ordering Grasping Type Pick & Place Success medal Grasping Success Videos

References

Official Code: Source code from the authors of the method
SceneReplica Version: Our maintained version (upgrade dependencies, add ROS interface, etc.)

    BibTeX

    Please cite SceneReplica if it helps your research: 
    @article{khargonkar2023scenereplica,
  title={SCENEREPLICA: Benchmarking Real-World Robot Manipulation by Creating Replicable Scenes}, 
  author={Ninad Khargonkar and Sai Haneesh Allu and Yangxiao Lu and Jishnu Jaykumar P and Balakrishnan Prabhakaran and Yu Xiang},
  journal={arXiv preprint arXiv:2306.15620},
  year={2023}}

    Contact

    Send any comments or questions to Ninad | Sai:
    ninadarun.khargonkar@utdallas.edu | saihaneesh.allu@utdallas.edu

    Acknowledgements

    This work was supported in part by the DARPA Perceptually-enabled Task Guidance (PTG) Program under contract number HR00112220005.