Localization and Mapping based on Semantic and Multi-Layer Maps Concepts

Abstract

The ONU's 2030 Agenda for Sustainable Development aims to promote sustainable agriculture. The demand for natural resources such as food is expected to continue to grow due to the continuous increase of the world's population. Thus, there is the need to make agriculture more e cient and sustainable. Robotics can play a key role in agriculture since robots can collect real-time information, perform autonomous operations, trigger action plans in case of diseases, among others. Agriculture imposes several challenges to agriculture. Namely, the high extension of the crops, the terrain irregularities, the high symmetry in crops characterized by natural corridors, and harsh inclinations in mountain crops. In this environments, semantic perception and mapping are essential concepts. With semantics, robots can learn how to assign meaning to data, learn to coexist with humans and provide useful information for further human use. To implement autonomous navigation systems in these conditions, robots should be able to operate fully autonomously during large periods.Thus, Simultaneous Localization and Mapping algorithms should be able to work in large-scale and long-term operating conditions. One of the main challenges is maintaining low memory resources while mapping extensive environments. This work proposes a novel solution called VineSLAM that allows robots to localiz themselves in challenging environments while creating di erent types of maps of the crop - metric, semantic, and topological. The rst mapping layer (metric) uses Light Detection And Ranging sensors to map feature points and polygon-based features such as the ground plane or the canopy walls. The second mapping layer (semantic) uses Deep Learning-based object detection models to detect natural elements in different growth stages in images, and maps them. In this thesis, this semantic perception system was tested through the detection of vine trunks and grapes bunches. Finally, the third mapping layer (topological) enables robots to work continuously in the crop without time or scale restrictions through a graph-based topological map. On top of the entire mapping architecture, VineSLAM provides a localization algorithm based on a modular Particle Filter that is able to use the different sources of features and maps to compute the six degrees of freedom of the robot's pose. The usage of different sensors together for localization and mapping is possible through a LiDAR-to-camera extrinsic calibration algorithm proposed in this thesis. As a framework, VineSLAM opens the door for robots to operate autonomously in challenging agricultural conditions through a robust localization algorithm combined with a multi-layer mapping approach that provides a rich and varied representation of the crop. Compared with state-of-the-art algorithms, VineSLAM presents an overall better performance since it can localize the robot with similar or higher precision, and can operate for unlimited time frames. Results show that in terms of semantic perception, was obtained an average precision of 84.16% for trunk detection and 44.93% for grape bunch detection. The localization and mapping algorithm was tested in multiple environments, demonstrating, for example, an absolute pose error of 0.9 meters in a sequence placed in a mountain vineyard with 337.1 meters long and an altitude di erential of 7 meters. It was shown that this formulation allows the autonomous navigation without time or scale restrictions. The algorithm was tested in agricultural robots in di erent real environments, allowing them to move autonomously.