Persistent semantic monitoring of indoor spaces such as warehouses, hospitals, and offices requires a robot to repeatedly monitor an environment and track how objects change over time. Running full simultaneous localization and mapping (SLAM) with dense semantic reconstruction from scratch on every visit is redundant when the environment geometry stays the same and only the objects move. We present a modular two-stage system that separates geometric mapping from semantic updating. In the first stage, a frontier-based exploration method with a dynamic search window builds a 2D occupancy grid. In the second stage, the robot relocalizes in this map and builds a semantic object graph using an open-vocabulary object detector and a promptable segmentation model. Only the lightweight semantic stage is repeated on later visits, so the system scales well to frequent revisits. The object graph uses a category and distance based association rule to update objects, which lets the map reflect both intra-session changes (object changes within a single traversal) and inter-session changes (changes across revisits), such as objects being moved, removed, or added. We validate the system on a Fetch robot in two real indoor environments of about 8,500 sq.m and 117 sq.m, and report precision, recall, and F1 scores across multiple update iterations.
The video demonstrates a robot autonomously exploring a large scale 96m x 93m area and a medium scale 9m x 13m area using a Dynamic Window Frontier Exploration strategy. In the large scale envrionment, the robot completes the exploration in approximately 150 minutes, reaching a maximum speed of 0.6 m/s. During this process, it covers a total distance of over 800 meters.
Object detection and segmentation are performed in real time to build the semantic map. Our system utilizes GroundingDINO to detect objects from the robot’s RGB image observation, providing labels and bounding boxes, which are then used as prompts for MobileSAM, a faster version of SAM, to generate segmentation masks.
@misc{allu2026buildoncemonitorcontinuously,
title={Build Once, Monitor Continuously: Persistent Semantic Mapping via Autonomous Exploration and Open-Vocabulary Object Updates},
author={Sai Haneesh Allu and Itay Kadosh and Tyler Summers and Yu Xiang},
year={2026},
eprint={2409.15493},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2409.15493},
}Send any comments or questions to Sai Haneesh Allu: saihaneesh.allu@utdallas.edu
This work was supported by the DARPA Perceptually- enabled Task Guidance (PTG) Program under contract num- ber HR00112220005, the Sony Research Award Program, the National Science Foundation (NSF) under Grant Nos. 2346528 and 2520553, and the NVIDIA Academic Grant Program Award. The work of T. Summers was supported by the United States Air Force Office of Scientific Research under Grant FA9550-23-1-0424 and the National Science Foundation under Grant ECCS-2047040. We would like to thank our colleague, Jishnu Jaykumar P, for his assistance during the experiments