FewSOL: A Dataset for Few-Shot Object Learning in Robotic Environments

1 The University of Texas at Dallas      2NVIDIA
IEEE International Conference on Robotics and Automation (ICRA), 2023
Publication arXiv Supplementary Code Data paperswithcode

Abstract

We introduce the Few-Shot Object Learning (FewSOL) dataset for object recognition with a few images per object. We captured 336 real-world objects with 9 RGB-D images per object from different views. Object segmentation masks, object poses and object attributes are provided. In addition, synthetic images generated using 330 3D object models are used to augment the dataset.

We investigated (i) few-shot object classification and (ii) joint object segmentation and few-shot classification with the state-of-the-art methods for few-shot learning and meta-learning using our dataset. The evaluation results show that there is still a large margin to be improved for few-shot object classification in robotic environments.

Our dataset can be used to study a set of few-shot object recognition problems such as classification, detection and segmentation, shape reconstruction, pose estimation, keypoint correspondences and attribute recognition.

Annotations

Multi-View RGB-D Object Poses and Masks Object Attributes Synthetic Data

Data

FewSOL-Dataset FewSOL-Experiment-Datasets

FewSOL is licensed under MIT license.

Once you successfully download and extract the dataset, you should have a folder with the following structure: 
├── FewSOL-Dataset/
                  ├── synthetic_objects.zip
                  ├── google_scenes.zip
                  ├── google_scan_selected.zip
                  ├── real_objects.zip
                  ├── OCID_objects.zip
                  ├── LICENSE
                  └── README.txt

More information can be found in the README.txt. For instructions about using the dataset please see FewSOL-Toolkit.

Note: Use FewSOL-experiment-datasets for experiments mentioned in the paper. README.md provides more details.

Code

FewSOL Toolkit

A toolkit for the FewSOL dataset.

FewSOL-Meta-Dataset

The code for (i) few shot classification, (ii) joint object segmentation and few shot classification and (iii) real world experiments conducted by ingesting FewSOL dataset into Meta-Dataset benchmark.

FewSOL PyTorch Dataloader

A PyTorch dataloader for the FewSOL dataset. Contributed by Jesse Musa.

BibTeX

Please cite FewSOL if it helps your research: 
@INPROCEEDINGS{padalunkal2023fewsol,
  title={FewSOL: A Dataset for Few-Shot Object Learning in Robotic Environments}, 
  author={P, Jishnu Jaykumar and Chao, Yu-Wei and Xiang, Yu},
  booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)}, 
  doi={10.1109/ICRA48891.2023.10161143},
  pages={9140-9146},
  year={2023}
}

Contact

Send any comments or questions to Jishnu: jishnu.p@utdallas.edu

Acknowledgements

This work was supported in part by the DARPA Perceptually-enabled Task Guidance (PTG) Program under contract number HR00112220005.