Abstract
Robot localization is the problem of finding a robot’s pose using a map and sensor measurements, like LiDAR scans or camera images. It is crucial for any moving autonomous vehicle to interact with the physical world correctly. However, finding injective mappings between measurements and poses is difficult because sensor measurements from multiple distant poses can be similar.
To solve this ambiguity, Monte Carlo Localization (MCL), the widely adopted method, uses random hypothesis sampling and sensor measurement updates to infer the pose. Other common approaches are to use Bayesian filtering or to find better-distinguishable global descriptors on the map. Recent developments in localization research usually propose better measurement models or feature extractors within these frameworks. On contrary, this project we propose a radically new approach to frame the localization problem as an ambiguous inverse problem and solve it with an invertible neural network (INN). We claim that INN is naturally suitable for the localization problem with many benefits, in terms of high accuracy (within 0.25m for city-scale maps), high-speed operation (>150Hz) and operates on low-cost embedded system hardware. We will demonstrate this on point-cloud and camera datasets with evaluation on indoor and outdoor localization benchmarks, and also deploy it on 1/10th scale and 1/2 scale autonomous vehicles to show real-time and scalable operation.
INN stands for Invertible Neural Network
Description
Timeline
Strategic Description / RD&T
Precise and fast localization of the autonomous system with respect to obstacles and landmarks is essential to maintaining safety in future transportation systems. The work proposed here is generally applicable across both advanced driver assistance systems, self-driving vehicles, infrastructure-assisted safety systems and is platform agnostic.
Deployment Plan
We will develop this into an open-source toolkit for the robotics and transportation community to use. We will deploy it on a variety of scaled and real autonomous vehicles. We will benchmark the reliability and efficiency on real hardware across multiple driving scenarios.
Q1: Develop 3D Lidar-based Local_INN with benchmarking for Ouster and Velodyne Lidars
Q2: Develop Camera-based Pose_INN with benchmarking on common datasets and demonstration on indoor and outdoor autonomous vehicles
Q3: Release open-source toolbox and host tutorial in robotics and transportation conferences
Q4: Develop a complete INN-based approach which will outperform the current approaches of ROS2-based SLAM implementations.
INN stands for Invertible Neural Network
Expected Outcomes/Impacts
Benefits of Local_INN -
1. Cheap, Fast and Low latency - It employs a Small neural network-based method
2. Accurate localization: Comparable to particle filter at low speed; Higher precision than particle filter at high speed.
3. Expandable from 2D Lidar to 3D Lidar and camera
4. No map file needed. Local_INN compresses the map in the neural network.
5. Fast convergence in Global Localization - this is very important when the vehicle loses localization or just starts up in a new localization.
We will develop this into an open-source toolkit for the robotics and transportation community to use. We will deploy it on a variety of scaled and real autonomous vehicles. We will benchmark the reliability and efficiency on real hardware across multiple driving scenarios.
Expected Outputs
This project, called Local_INN, has four components to the deployment plan:
1. Map Compression: Local_INN provides an implicit map representation and a localization method within one neural network. Map files are no longer needed when localizing. We will develop the basic INN structure for pose inference and map regeneration in this step to demonstrate the functionality for indoor and ooutdoor localization.
2. Uncertainty Estimation: Local_INN outputs not just a pose but a distribution of inferred poses, the covariance of which can be used as the confidence of the neural network when fusing with other sensors, enhancing the overall robustness. We will use this to provide safety guarantees for localization and pose estimation in high-risk driving scenarios.
3. Fast and Accurate: We demonstrate that the localization performance of Local_INN is comparable to particle filter at slow speed and better at high speed with much lower latency with 2D LiDAR experiments. We will conduct extensive experiments to deploy Local_INN on real and scaled autonomous vehicles.
4. Ability to Generalize: We demonstrate that the framework of Local_INN can learn complex 3D open-world environments and provides accurate localization. We also provide an algorithm for global localization with Local_INN. We will work with partners to show how this scheme can work for infrastructure mounted sensors and how multiple of them can be combined.
TRID
In this project we propose a radically new approach to frame the vehicle localization problem as an ambiguous inverse problem and solve it with an invertible neural network (INN). We claim that INN is naturally suitable for the localization problem with many benefits, in terms of high accuracy (within 0.25m for city-scale maps), high-speed operation (>150Hz) and operates on low-cost embedded system hardware. We will demonstrate this on point-cloud and camera datasets with evaluation on indoor and outdoor localization benchmarks, and also deploy it on 1/10th scale and 1/2 scale autonomous vehicles to show real-time and scalable operation. While the TRiD has several projects in the general topics of robot localization and vehicle localization, they do not use invertible neural networks. Most use Monte-Carlo and Bayesian iterative approaches and require the map data at runtime. Our approach does not require the map data during inference.
Individuals Involved
| Email |
Name |
Affiliation |
Role |
Position |
| rahulm@seas.upenn.edu |
Mangharam, Rahul |
University of Pennsylvania |
PI |
Faculty - Tenured |
Budget
Amount of UTC Funds Awarded
$100000.00
Total Project Budget (from all funding sources)
$100000.00
Documents
Match Sources
No match sources!
Partners
| Name |
Type |
| The Autoware Foundation |
Deployment & Equity Partner Deployment & Equity Partner |
