RobotFingerPrint: Unified Gripper Coordinate Space for Multi-Gripper Grasp Synthesis and Transfer​

Teaser Image

Abstract

We introduce a novel grasp representation named the Unified Gripper Coordinate Space (UGCS) for grasp synthesis and grasp transfer. Our representation leverages spherical coordinates to create a shared coordinate space across different robot grippers, enabling it to synthesize and transfer grasps for both novel objects and previously unseen grippers. The strength of this representation lies in the ability to map palm and fingers of a gripper and the unified coordinate space. Grasp synthesis is formulated as predicting the unified spherical coordinates on object surface points via a conditional variational autoencoder. The predicted unified gripper coordi- nates establish exact correspondences between the gripper and object points, which is used to optimize grasp pose and joint values. Grasp transfer is facilitated through the point-to-point correspondence between any two (potentially unseen) grippers and solved via a similar optimization. Extensive simulation and real-world experiments showcase the efficacy of the unified grasp representation for grasp synthesis in generating stable and diverse grasps. Similarly, we showcase real-world grasp transfer from human demonstrations across different objects.



Videos

Supplementary Video Presentation




Real World Experiments on SceneReplica Benchmark




Grasp Transfer Examples

Code

IRVLUTD => robot-finger-print

Code repository for the project, adapted from GenDexGrasp. Please follow their instructions for the isaac gym grasp evaluation setup. We used a learning rate of learning_rate=0.1 and step_size=0.02 for the grasp evaluation params defined under the env script for each gripper.

Dataset

Dataset: Gripper Coordinates and Surface Points

Box.com (no login required) link for the gripper surface points coordinates and other metadata files. Please refer to the README provided in the folder for overall setup (since these files are supposed to be used with the dataset provided by GenDexGrasp).

Citation (BibTeX)

Please cite RobotFingerPrint if this work helps in your research:
@inproceedings{khargonkar2024robotfingerprint,
      title={RobotFingerPrint: Unified Gripper Coordinate Space for Multi-Gripper Grasp Synthesis and Transfer​},
      author={Khargonkar, Ninad and Casas, Luis Felipe and  and Prabhakaran, Balakrishnan and Xiang, Yu},
      journal={arXiv preprint arXiv:2409.14519},
      year={2024}
    }

Contact

Send any comments or questions to Ninad Khargonkar: ninadarun.khargonkar@utdallas.edu or
Luis Felipe Casas: Luis.CasasMurillo@UTDallas.edu

Acknowledgements

This work was supported by the Sony Research Award Program and the National Science Foundation (NSF) under Grant No. 2346528.