FewSOL: A Dataset for Few-Shot Object Learning in Robotic Environments

1The University of Texas at Dallas  2NVIDIA

IEEE International Conference on Robotics and Automation · ICRA 2023

Publication arXiv Supplementary Code Data Citation
TL;DR A dataset for teaching robots new objects from a few RGB-D views.
  1. 01336
    real objects
  2. 029 RGB-D
    views each
  3. 03330 synthetic
    3D models
  4. 04Masks · poses
    · attributes

Benchmarks few-shot classification and joint segmentation & classification — with clear headroom in robotic environments.

Abstract

We introduce the Few-Shot Object Learning (FewSOL) dataset for object recognition with a few images per object. We captured 336 real-world objects with 9 RGB-D images per object from different views. Object segmentation masks, object poses and object attributes are provided. In addition, synthetic images generated using 330 3D object models are used to augment the dataset.

We investigated (i) few-shot object classification and (ii) joint object segmentation and few-shot classification with the state-of-the-art methods for few-shot learning and meta-learning using our dataset. The evaluation results show that there is still a large margin to be improved for few-shot object classification in robotic environments.

Our dataset can be used to study a set of few-shot object recognition problems such as classification, detection and segmentation, shape reconstruction, pose estimation, keypoint correspondences and attribute recognition.

336 real objects 9 RGB-D views / object 330 3D models Masks · Poses · Attributes

Overview

FewSOL overview video thumbnail

Annotations

Explore the four annotation modalities provided per object.

Multi-View RGB-D Object Poses & Masks Object Attributes Synthetic Data

Code & Data

Code
FewSOL Toolkit Utilities for loading, visualising, and exporting the dataset FewSOL-Meta-Dataset Few-shot classification + joint segmentation on Meta-Dataset FewSOL PyTorch Dataloader Drop-in PyTorch Dataset wrapper (contrib. Jesse Musa)
Data · FewSOL
FewSOL-Dataset Full release — 336 real objects · 330 synthetic models · RGB-D + poses + masks FewSOL-Experiment-Datasets Curated splits used for the paper's experiments README Folder layout, MIT license, usage notes
Folder layout Once extracted, the dataset looks like this 
├── FewSOL-Dataset/
│   ├── synthetic_objects.zip
│   ├── google_scenes.zip
│   ├── google_scan_selected.zip
│   ├── real_objects.zip
│   ├── OCID_objects.zip
│   ├── LICENSE
│   └── README.txt

BibTeX

Please cite FewSOL if it helps your research.

@INPROCEEDINGS{padalunkal2023fewsol,
  title   = {{FewSOL: A Dataset for Few-Shot Object Learning in Robotic Environments}},
  author  = {P, Jishnu Jaykumar and Chao, Yu-Wei and Xiang, Yu},
  booktitle = {2023 IEEE International Conference on Robotics and Automation (ICRA)},
  doi     = {10.1109/ICRA48891.2023.10161143},
  pages   = {9140--9146},
  year    = {2023}
}

Contact

Send any comments or questions to Jishnu — jishnu.p@utdallas.edu

Acknowledgements

Supported by

DARPA
Perceptually-enabled Task Guidance
HR00112220005