Supervisor: Prof. Dr.-Ing. C. Heipke
This dissertation project deals with tracking and re-identification of multiple persons from moving stereo cameras. The scenario it is placed in is a set of moving cars which collaboratively carry out the task: a person or a group of persons, typically walking on a sidewalk of a road or crossing a road at a zebra crossing, is being detected by one stereo system and tracked as long as the persons are visible, resulting in information about existence, number of persons, position, speed, appearance and possibly also their behavior in the immediate future. This information is then transferred to other platforms equipped with stereo sensors, which subsequently need to re-identify and further track the person. Broadcasting the results to nearby cars, these can change their driving characteristics accordingly, which will render traffic more safe.
Detection and tracking should be carried out in a tracking-by-detection scheme in 3D and extend prior work carried out in the Research Training group (Schön et al., 2018), Nguyen et al, 2019) in terms of integration of pedestrian behavior. In this way, not only real-time capability is ensured, but at each epoch during the computations an integrity measure is available for the results. Particular challenge are the constantly changing occlusions coupled with the need for a motion/interaction and behavior prediction model for groups of people.
Another challenge is given by re-identifying problem. It is particularly difficult in outdoors environments and for largely different points of view. In this regard prior work carried out for indoors surveillance, largely based on convolutional neural networks (Blott et al, 2019) is to be extended in a suitable way.
Fig. 1: Stereo tracking and re-identification in 3D
Fig. 2: Person re-identification: probe (left), 2 candidates (correct), another 8 candidates (incorrect)
References
Blott G, Yu J., Heipke C., 2019: Multi-View Person Re-Identification in a Fisheye Camera Network with different viewing directions. PFG, in print.
Nguyen, U.; Rottensteiner, F.; Heipke, C., 2019: Confidence-aware pedestrian tracking using a stereo camera. In: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences IV-2/W5, 53-60. 10.5194/isprs-annals-IV-2-W5-53-2019
Schön S., Brenner C., Alkhatib H., Coenen M., Dbouk H., Garcia-Fernandez N., Fischer C., Heipke C., Lohmann K., Neumann I., Nguyen U., Paffenholz J.-A., Peters T., Rottensteiner F., Schachtschneider J., Sester M., Sun L., Vogel S., Voges R., Wagner B., 2018: Integrity and Collaboration in Dynamic Sensor Networks, Sensors, 18, 2400, 21 pages; doi:10.3390/s18072400.
