# Project 1: Bounding and propagating observation uncertainty with interval mathematic (Prof. Dr.-Ing. Steffen Schön)

Intervals (Jaulin et al 2001) can be seen as a natural way to bound observation uncertainy in navigation systems such as GPS, IMU or optical sensors like LIDAR, since they are in principle free of any assumption about probability distributions and can thus describe adequately remaining systematic effects (Schön 2016, Schön and Kutterer 2006). In an ongoing project (Dbouk and Schön 2019) we used intersections of observation intervals to compute feasible positioning domains and to exclude large outliers, cf. Fig. 1. However, all results are based on - meaningful - assumptions about the respective intervals.

In this project, we intent to experimentally investigate in more details the actual size of observation intervals. To this end, the mapathon data will be re-analyzed and new dedicated experiments in urban environments as well as laboratory experiments in the new HITEC building, are designed and carried out with high-end as well as typical GNSS and IMU equipments. Special focus will be on the inclusion of remaining deviations in GNSS code and phase observations. For inertial sensors, methods should be developed to propagate the interval uncertainty through the strap-down differential equation without artificially blowing up the interval size.

### Publications

(2019): *Reliability and Integrity Measures of GPS Positioning via Geometrical Constraints**. *Proceedings of the 2019 International Technical Meeting of The Institute of Navigation, Reston, Virginia, January 2019, pp. 730-743. DOI: 10.33012/2019.16722

(2001): *.* Springer, London

**Schön S.** (2016): *Interval-based reliability and integrity measures. *Proc. ESA Navitec 2016

(2006): *Uncertainty in GPS Networks due to Remaining Systematic Errors: The Interval Approach*. Journal of Geodesy 80(3):150-162, DOI: 10.1007/s00190-006-0042-z

# Project 2: Validation and quality assurance concepts for collaborative multi-sensor-systems (Prof. Dr.-Ing. Ingo Neumann)

Nowadays, the judgement of the quality of results (including integrity) for dynamic sensor networks is an important issue. This is of great importance when the results are used, e.g., in the context of (relative) positioning of autonomous cars. A typical example is the uncertainty of the mapped environment (point cloud) from a multi-sensor system (MSS) which is not homogeneous and depends on many different influencing factors.

Within this project a new methodology for the validation and quality assurance for collaborative interacting MSS shall be developed. Especially the gain by the collaboration for the uncertainty and integrity shall be judged. The starting point of this process is a reference product (e.g. 3D mapping information) and trajectory, which is compared with the results of the collaborative systems (backward modelling). The quality parameters of the interacting MSS are then obtained by an inverse global optimization. This includes the sensor properties, the trajectory and other relevant parameters. The obtained results can be later used for the forward quality modelling of the collaborative MSS.

The algorithms developed for the filtering of the trajectory from Sun et al. (2018) provides a basic input and the relevant quality parameters are numerically determined by the newly developed inverse method.

### Publications

**Stenz, U.; Hartmann, J.; Paffenholz, J.-A.; Neumann, I.** (2017): *A Framework Based on Reference Data with Superordinate Accuracy for the Quality Analysis of Terrestrial Laser Scanning-Based Multi-Sensor-Systems*, Sensors 2017, 17(8), 1886. DOI: 10.3390/s17081886.

**Sun, L.; Alkhatib, H., Paffenholz, J.-A.; Neumann, I. **(2018):** ***Geo-Referencing of a Multi-Sensor System Based on Set-membership Kalman Filter*. In: 21st International Conference on Information Fusion (FUSION) 2018, Cambridge, United Kingdom, July 10-13, 2018, p. 889 - 896. DOI: 10.23919/ICIF.2018.8455763, ISBN: 978-0-9964527-6-2.

**Sun, L.; Alkhatib, H.; Kargoll, B.; Kreinovich, V.; Neumann, I. **(2018):** ***A new Kalman filter model for nonlinear systems based on ellipsoidal bounding*. Submitted to Journal of Optimization and Theory and Applications.

arXiv: 1802.0297

# Project 3: Integrity measures for hierarchical and incremental map data acquisition (Prof. Dr.-Ing. Monika Sester)

Data acquired collaboratively by several traffic participants is characterized by different levels of detail, accuracy and completeness. Integrating and assembling this information to a consistent dynamic map requires integrating and propagating quality and integrity measures across the different levels of representation. The project involves the definition of multiscale representations of quality measures, as well as mechanisms for the propagation of those measures across the different representations. Methods from cartographic generalization, machine learning and optimization will be required.

### Publications

**Sester, M., Arsanjani, JJ., Klammer, R., Burghardt, D. & Haunert, JH.** (2014): *Integrating and Generalising Volunteered Geographic Information. Abstracting Geographic Information in a Data Rich World*, Springer, 119‐155.

**Schulze, M.; Brenner, C. & Sester, M.** (2012): *Cooperative information augmentation in a geosensor network, Advances in Geo‐Spatial Information Science*, CRC Press

**Hampe, M.; Sester, M. & Harrie, L.** (2004): *Multiple representation databases to support visualisation on mobile devices*. In: Altan, O. (Hg.): XXth ISPRS Congress, Technical Commission IV, B4. XXth ISPRS Congress. Istanbul (ISPRS International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XXXV), S. 135–140.

**Sester, M.** (2005): *Optimizing Approaches for Generalization and Data Abstraction*, International Journal of Geographic Information Science, vol. 19 Nr. 8-9, pp. 871-897

# Project 4: Collaborative Navigation: Being more resilient against outliers (Prof. Dr.-Ing. Steffen Schön)

In collaborative navigation, a group of dynamic nodes (vehicles, pedestrians, etc.) equipped with different time synchronized navigation sensors (GNSS, IMU, LIDAR, stereo cameras, odometer,...) increase the positioning quality by exchanging navigation information as well as performing measurements between nodes or to elements of the environment such as urban furniture or buildings (Garcia and Schön 2018). Using a centralised approach, a common filter can be set up that analyses all measurements of all participants together and estimates their states in an optimal manner. Urban areas are very challenging for navigation sensors, like GNSS multipath, diffraction and occlusions of satellite signals. Subsequently, the strength of the positioning geometry is significantly weakened and the data quality is restricted.

In this projects, the use and limits of collaborations for strengthening the reliability of positioning will be investigated. First, a new mathematical framework of reliability and integrity for collaborative sensor networks should be developed and tested in simulation studies. This includes partial GNSS observations (<4 visible satellites), use of atomic clocks, aspects of traffic flow and parked vehicles as anchors. As a result, general statements on the impact of collaboration on integrity should be derived as well as the driving quality factors. In addition, advanced concepts for outlier detection in collaborative scenarios will be developed (e.g. based on Consensus scores) and tested with real data.

### Publications

**Garcia-Fernandéz N., Schön S.** (2018). *Evaluating a LKF Simulation Tool for Collaborative Navigation Systems*, Proceedings of IEEE/ION PLANS. DOI: 10.1109/plans.2018.8373539

# Project 5: Detection and collaborative tracking of vehicles considering UAV based aerial images (apl. Prof. Dr. tech. Franz Rottensteiner)

In this dissertation project the contribution of aerial photographs taken by UAVs to the collaborative positioning of vehicles is examined. A UAV captures aerial photos of the street, depicting vehicles that can communicate with each other and with the UAV. The vehicles have stereo cameras and can thus position themselves relative to each other. Although these relative poses can increase the relative accuracy of the positioning, due to visibility restrictions, an unfavorable block configuration is to be expected. The aerial photographs can support the block geometry due to providing a better overview and linking multiple vehicles. The incorporation of UAV images requires suitable methods to recognize the vehicles in these images and to reconstruct the corresponding vehicle geometry. The corner points of the reconstructed vehicles can on the one hand serve as tie points for positioning the vehicle models in the object space; on the other hand, in this way the vehicle model can be consistently estimated from the total available information (stereo images from the vehicles, UAV images). At the same time, a collaborative tracking of the objects over time during which they can be observed by the UAV should take place. Compared to the vehicle reconstruction process already developed in the context of i.c.sense, the challenge lies in the absence of stereo information as well as in the unfavorable viewing direction for the use of pre-trained classifiers. Furthermore, the problem of collaborative tracking has to be solved. For evaluation, real-world data will be recorded at street intersections with a UAV and several vehicles equipped with stereo cameras as part of the central experimentation facility.

### Publications

**Molina, P.; Blázquez, M.; Cucci, D.A.; Colomina, I.** (2017). First Results of a Tandem Terrestrial-Unmanned Aerial mapKITE System with Kinematic Ground Control Points for Corridor Mapping. Remote Sens. 2017, *9*, 60.

**Coenen, M.; Rottensteiner, F.; Heipke, C.** (2018). Recovering the 3D pose and shape of vehicles from stereo images.. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences IV-2, pp. 73-80, 2018.

# Project 6: Collaborative positioning using imaging sensors (Prof. Dr.-Ing. Christian Heipke)

In diesem Dissertationsprojekt wird untersucht, wie für die integre, kollaborative Positionierung mit GNSS und Bildsensoren in dynamischen Sensornetzen, in denen sich die Knoten gegenseitig erkennen und über ihren Zustand austauschen, die ursprünglichen Sensordaten (z. B. Grauwerte), daraus abgeleitete Größen (z. B. Bildmerkmale) und die relativen Posen am besten kombiniert werden können. Grundlage des Projekts sind Ansätze der photogrammetrischen Punktbestimmung mittel Bündelausgleichung, die geeignet zu erweitern sind. Dabei muss die Architektur der Gesamtlösung (zentral vs. dezentral) und die prinzipielle Bestimmbarkeit einer Lösung inklusive deren Qualität auch unter eingeschränkten Bedingungen berücksichtigt werden. Unter Nutzung der digitalen Karte ist durch Simulationen und mit echten Daten für verschiedene Szenarien der Experimentierstube, wie z.B. eine dicht befahrene Straße mit GNSS-Abschattung, zu untersuchen, in welchem Umfang durch die Kollaboration eine ausreichend genaue und zuverlässige Positionierung auch in Bereichen möglich wird, in denen kein ausreichende GNSS-Information vorhanden ist.

### Publications

**Molina, P.; Blázquez, M.; Cucci, D.A.; Colomina, I. **(2017).** ***First Results of a Tandem Terrestrial-Unmanned Aerial mapKITE System with Kinematic Ground Control Points for Corridor Mapping*. Remote Sens. **2017**, *9*, 60.

**Yu H., Li H., Yang Z.** (2019). Collaborative Visual SLAM Framework for a Multi-UAVs System Based on Mutually Loop Closing, Wireless and Satellite Systems. In: 10th EAI International Conference, WiSATS 2019, Harbin, China, January 12–13, 2019, Proceedings, Part I – LNCS, Springer.

# Projekt 7: Integrity contained navigation based on vehicle data and constrained colaborative information (Prof. Dr.-Ing. Ingo Neumann)

The positioning and navigation of vehicles is usually based on the combination of global navigation satellite systems (GNSS) and additional inertial sensors (IMU). I.e. for long periods, the results rely mainly on the GNSS measurements. This means, that in case of GNSS interruptions and also outliers the positioning and navigation is not a process with integrity any more. This applies i.e. to difficult town environments and indoor application. To overcome this problem, a new methodology was developed which introduces constrained information to the vehicle data. Examples are vertical facades and horizontal street surfaces within a given tolerance zone. This information acts as inequality constraints in a trajectory estimation (Vogel et al. 2018). The determination of the facades and their geometric characteristic can e.g. be determined by high-end mobile mapping systems or by a collaborative mapping from a large number of autonomous vehicles. Furthermore, construction standards from ISO are available to judge the tolerance values of the buildings.

Within the new project, this methodology is extended to constrained collaborative positioning and navigation in difficult environments. Furthermore, the algorithms are transferred to mass date. The main extension lies in the integrity checking of the network. Each vehicle and their sensors together with their collaborative observations are checked within a constrained network. The methods and algorithms shall be applied to the Mapathon data. This implies the judgement of the usability of the façade information for the above task.

### Publications

**Schön, S.; Brenner, C.; Alkhatib, H.; Coenen, M.; Dbouk, H.; Garcia-Fernandez, N.; Fischer, C.; Heipke, C.; Lohmann, K.; Neumann, I.; Nguyen, U.; Paffenholz, J.-A.; Peters, T.; Rottensteiner, F.; Schachtschneider, J.; Sester, M.; Sun, L.; Vogel, S.; Voges, R.; Wagner, B. **(2018):** ***Integrity and Collaboration in Dynamic Sensor Networks*, In: Sensors 2018, 18(7), 2400, p.21.

DOI: 10.3390/s18072400

**Vogel, S.; Alkhatib, H.; Neumann, I.** (2018): *Iterated Extended Kalman Filter with Implicit Measurement Equation and Nonlinear Constraints for Information-Based Georeferencing*, In: 21st International Conference on Information Fusion (FUSION) 2018, Cambridge, United Kingdom, July 10-13, 2018, p. 1209-1216.

# Project 8: Localization and mapping using maximum consensus (apl. Prof. Dr.-Ing. Claus Brenner)

For mobile systems, robust localization and mapping is of great importance. Often, parametric distributions are used for the representation of uncertainties, both for the pose of the mobile systems and for the map of the environment. Although they can be computed efficiently, they are unable to model deviations from the chosen distribution function. A common source of such deviations are wrong associations in the context of localization and mapping problems, due to erroneous measurements, wrong assumptions about the ego position, or changing environments. If these are not detected as outliers, they will result in an erroneous estimate of the system state that is often difficult to detect.

Recently, *maximum consensus* techniques have been discussed for state estimation. They are very robust against outliers, but usually lead to a high computational effort. For systems with high integrity requirements, however, they can be an interesting solution, not only because of their robustness, but also because they allow the identification of erroneous information sources.

In this dissertation project, the applicability of such techniques for problems of filtering, smoothing and adjustment shall be considered, in particular with regard to the localization of mobile sensors using maps of the environment. To model the dependencies, probabilistic graphical models are a suitable tool, and for inference, message passing approaches may be used. As a computational model, the parallel computation, for example on massively parallel clusters or on GPUs, can be considered. The use of machine learning methods, in particular deep learning, can also be investigated, for example, to derive quality measures for consensus situations.

### Publications

**Li, H.** (2009). *Consensus set maximization with guaranteed global optimality for robust geometry estimation*. In: IEEE international conference on computer vision (ICCV), pp. 1074–1080.

**T.-J. Chin and D. Suter **(2017). *The maximum consensus problem: recent algorithmic advances*. Synthesis Lectures on Computer Vision (Eds. Gerard Medioni and Sven Dickinson), Morgan & Claypool Publishers, San Rafael, CA, U.S.A.

# Project 9: Combining probabilistic and interval-based error models for sensor fusion on mobile robots (Prof. Dr.-Ing. Bernardo Wagner)

Interval-based methods are used in the field of mobile robotics to put reliable bounds on the results of different approaches (e.g. localization, map building, obstacle detection, …). These methods return an interval (lower and upper bound) containing the true solution. However, often such an interval is not sufficiently precise as it does not provide a working point as input for a controller.

Nevertheless, methods relying on interval analysis have many advantages over probabilistic approaches (e.g. guaranteed results under given assumptions, ability to model systematic errors, no linearization errors, …). They can be used to eliminate definitely wrong results while making no explicit statement about the true solution, which is only guaranteed to reside in the resulting interval. Naively, the midpoint of an interval can serve as a point-valued solution.

On the other hand, probabilistic approaches are widely used for different problems in the field of mobile robotics. By design, they return point-valued results, which, however, cannot be guaranteed and may be significantly wrong due to systematic or linearization errors. The aim of this project is to combine probabilistic and interval-based methods for robot control to guarantee the integrity of the solution. On the other hand, this enables us to provide point-valued results for further computations.

### Publications

**L. Jaulin, M. Kieffer, O. Didrit, and É. Walter** (2001), *Applied Interval Analysis*. Springer London, 2001.

**R. Neuland, R. Maffei, L. Jaulin, E. Prestes, and M. Kolberg** (2014). *Improving the precision of AUVs localization in a hybrid interval-probabilistic approach using a set-inversion strategy*, Unmanned Systems, vol. 02, no. 04, pp. 361–375, 2014.

**M. Langerwisch and B. Wagner**. *Guaranteed mobile robot tracking using robust interval constraint propagation*, Lecture Notes in Computer Science, pp. 354–365, 2012. DOI:dx.doi.org/10.1007/978-3-642-33515-0_36.