Stereo-Based Object Tracking and Pedestrian Recognition in Traffic environments
Project founded by Volkswagen AG, Germany (2006-2007)
The pedestrians are the most vulnerable participants to the urban traffic. The first step toward protecting pedestrians is to reliably detect them. However, although recognizing the humanoid shape proves very easy for humans, it is very difficult, at the moment, for computer vision systems. This is especially true in the highly cluttered urban environment. The high variance of appearance, occlusions and different pose and distance are problems in pedestrian classification.
Development of a real-time dense-stereo camera based system able to
detect, classify and track pedestrians in urban environments. Based on
the detection parameters the system is able to alert the driver of a
imminent accident or take control o f the car to avoid a collision with
a pedestrian. The system needs to have a high degree of accuracy and
speed to be able to handle difficult situations and react faster than a
human is able to.
• detection range: 0 to 30 m
• Field of view: 72 degrees
• Maximum traveling speed: 80 km/h
• Processing rate: 20 frames/second
The acquisition system is composed from two digital grayscale cameras integrated in a calibrated stereo head rigid. The camera parameters are calibrated using a dedicated methodology. Dense stereo reconstruction is performed by a dedicated hardware board (DeepSea by TYZX) or by a software algorithm. The 3D points are used to determine the surface of the road in front of the car. The 3D points above the detected road surface are grouped into pedestrian hypothesis, taking into account the density of the 3D points, vicinity criteria and 2D image information such as similar texture and connecting edges. Tracking using Kalman filtering is used in order to improve and stabilize the detected pedestrian hypothesis. The pedestrians are classified based on shape and motion features Real time implementation using Visual C++ framework and MMX instructions for code optimization was used. Detection results can be further fused with information provided by other sensors (radar and laser scanner).
Detection results of the DESPED application are presented in fig. 1:
|Fig. 1. DESPED output.|
Real-time dense stereo reconstruction is performed using a dedicated board (DeepSea by TYZX) (fig. 2). A software algorithm was also developed for off-line validation purposes (fig. 3).
Detection of pedestrian hypothesis
Shape Based Classification
Motion Based Classification
Multifeature Object Classification
We have developed a system based on stereovision able to perform in real time pedestrian detection and pedestrian classification in urban scenarios. The system functions are integrated into a dedicated stereovision framework, which can be easily extended with other capabilities (e.g. image analyzes tasks) or to build a specific application for an active security or driving assistance system for automotive industry.
The obtained expertise of the research team has reached the state-of-the-art level in some specific fields of the computer vision usable in robot or automotive applications: camera calibration, high-resolution stereo reconstruction, stereo measurements, object detection and tracking, pattern matching, motion estimation (control, programming, design).
S. Nedevschi, C. Tomiuc, S. Bota, "Stereo Based Pedestrian Detection for Collision Avoidance Applications", Proceedings of ICRA 2007 Workshop: Planning, Perception and Navigation for Intelligent Vehicles, April, 2007, Roma, pp. 39-44.
S. Bota, S. Nedevschi, Multi-Feature "Walking Pedestrian Detection Using Dense Stereo Motion", WIT 2007, 20-21 March, 2007, Hamburg, pp. 113-118
Corneliu Tomiuc, Sergiu Nedevschi, "Real time object classification exploiting 2D and 3D information", Proceedings of IEEE 2nd International Conference on Intelligent Computer Communication and Processing, 1-2 September, Cluj-Napoca, Romania, 2006, pp. 129-134.
Silviu Bota, Sergiu Nedevschi, "Walking Pedestrians Detection Using Motion Field and Dense Stereo", Poster volume of IEEE 2nd International Conference on Intelligent Computer Communication and Processing, 1-2 September, Cluj-Napoca, Romania, 2006, pp 45-51.
S. Nedevschi, C. Vancea, T. Marita, T. Graf, On-Line Calibration Method for Stereovision Systems Used in Vehicle Applications, Proceedings of the IEEE Intelligent Transportation Systems Conference (ITSC 2006), Toronto, Canada, September 17-20, 2006, pp. 957-962.