i.c.sensJob OpportunitiesJob Advertisement 3rd Cohort
Uncertainty modelling in LiDAR scene segmentation using neural networks

Uncertainty modelling in LiDAR scene segmentation using neural networks

This dissertation project addresses the modelling of uncertainty for semantic and scene segmentation, based on neural network architectures, with a focus on LiDAR data.

A key aspect of sensor data processing systems with integrity requirements is the ability to detect their own failure to provide correct information. Especially with contemporary sensors like cameras and laser scanners, producing massive amounts of data, it is very difficult to partition incoming data into ‘good’, supporting the correct solution, and ‘bad’, being detrimental to the correct solution. Typical examples for the latter are excessive noise, leading to a bad signal-to-noise ratio, and massive outliers, leading to plausible, but wrong, alternative hypotheses. In case of LiDAR data, these cases may for example be caused by scanning artefacts, vegetation, and occlusions or temporal scene changes.

High-level information, such as a semantic or scene segmentation, can provide a useful prior on how much trust can be put into data. For example, in vegetation areas, one may expect large noise, while moving objects, like vehicles, typically cause occlusions. However, when such a segmentation becomes part of the overall pipeline, it has to fulfil integrity requirements as well. Since the most successful contemporary segmentation approaches are based on neural networks, uncertainty and dependability in neural networks are currently highly relevant research topics.

The dissertation shall especially work on the modelling of uncertainty and thus extend earlier work on LiDAR semantic segmentation in the i.c.sens group [Peters et al., 2020]. It can use publicly available datasets or draw on large datasets which have been acquired by the group so far. Moreover, as the group operates a LiDAR mobile mapping van and several spinning LiDAR sensors, data can be collected for specific requirements which may arise during the dissertation project.


Feng, D., Rosenbaum, L., Timm, F., Dietmayer, K., 2019: Leveraging heteroscedastic aleatoric uncertainties for robust real-time lidar 3d object detection, 30th IEEE Intelligent Vehicles Symposium, 2019.

Feng, D., 2021: Uncertainty Estimation for Object Detection Using Deep Learning Approaches, Schriftenreihe des Instituts für Mess-, Regel- und Mikrotechnik, Band 37, Universität Ulm.

Kendall, A., Gal, Y., 2017: What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?, Proc. 31st International Conference on Neural Information Processing Systems, 5580–5590.

Liu, Z., Amini, A., Zhu, S., Karaman, S., Han, S., Rus, D. L., 2021: Efficient and Robust LiDAR-Based End-to-End Navigation, 2021 IEEE International Conference on Robotics and Automation (ICRA), 13247-13254.

Peters, T., Brenner, C., Song, M., 2020: Improving deep learning based semantic segmentation with multi-view outlier correction, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020