Dense Stereo-Based Object Recognition system for automatic cruise control (ACC) in urban traffic environments
Project founded by Volkswagen AG, Germany (2005-2007)
Automotive companies are investing a lost of interest in intelligent systems which can assist the driving process. Radar and laser sensors based applications are dedicated for precise pose measurements but they cannot give a complete description of the driving environment due to their nature. Vision sensors can compensate the information lack of their measurements. Stereovision has been proved the only method able give reliable results both for measurements and description of the driving environment.
Development of a realtime densestereo camera based system able to
detect, classify and track relevant objects, detect the lane and
estimate the road geometry, detect and recognize traffic signs painted
on the roads. Based on the detection parameters the system is able to
perform specific urban traffic ACC tasks: stop[&]go (follow- to-stop
and stop-inhibit), vehicle lateral control, set maximum ACC velocity
• detection range: 0 to 50 m
• Field of view: 72 degrees
• Maximum traveling speed: 80 km/h
• Processing rate: 15 frames/second
The acquisition system is composed from two digital b/w cameras integrated in a rigid stereo head. The camera parameters are calibrated using a dedicated methodology. Dense stereo reconstruction is performed by a dedicated hardware board (DeepSea by TYZX) or by a software algorithm. Features belonging to lane delimiters are classified and their 3D coordinates are used in the lane detection process. The 3D points above the detected road surface are grouped into objects, taking into account the vicinity criteria, density variation with the distance, and 2D image information such as similar texture and connecting edges. Tracking using Kalman filtering is used both for lane and objects parameters improving. Real time implementation using VisualC framework and MMX instructions for code optimization was used. Detection results can be further fused with information provided by other sensors (radar and laser scanner).
Detection results of the DESBOR application are presented in fig. 1:
Realtime dense stereo reconstruction is performed using a dedicated board (DeepSea by TYZX) (fig. 2). A software algorithm was also developed for offline validation purposes (fig. 3).
Detection of elevation maps
3D lane detection
Object detection and tracking
We have developed a system based on stereovision able to perform in real time many tasks needed for driving environment description and measurements. The system functions are integrated into a dedicated stereovision framework, which can be easily extended with other capabilities (e.g. image analyzes tasks) or to build a specific application for an active security or driving assistance system for automotive industry.
The obtained expertise of the research team has reached the state-of-the-art level in some specific fields of the computer vision usable in robot or automotive applications: camera calibration, high-resolution stereo reconstruction, stereo measurements, object detection and tracking, 3D lane detection, stereo image acquisition systems (control, programming, design).
S. Nedevschi, R. Danescu, T. Marita, F. Oniga, C. Pocol, S. Sobol, C. Tomiuc, C. Vancea, M. M. Meinecke, T. Graf, T. B. To, M. A. Obojski, "A Sensor for Urban Driving Assistance Systems Based on Dense Stereovision", Proceedings of Intelligent Vehicles 2007, 13-15 June, 2007, Istambul, pp.278-286
R. Danescu, S. Sobol, S. Nedevschi, T. Graf, "Stereovision-Based Side Lane and Guardrail Detection", Proceedings of the IEEE Intelligent Transportation Systems Conference (ITSC 2006), Toronto, Canada, September 17-20, 2006, pp. 1156-1161.
S. Nedevschi, T. Marita, R. Danescu, F. Oniga, C. Pocol, S. Sobol, C. Tomiuc, C. Vancea, S. Bota, "Stereovision Sensor for Driving Assistance", in Proceedings of IEEE International Conference on Intelligent Computer Communication and Processing, 1-2 Sept. 2006, Cluj-Napoca, Romania, pp. 105-112.
T. Marita, F. Oniga, S. Nedevschi, T. Graf, R. Schmidt, "Camera Calibration Method for Far Range Stereovision Sensors Used in Vehicles", Proceedings of IEEE Intelligent Vehicles Symposium, (IV2006) , June 13-15, 2006, Tokyo, Japan, pp. 356-363.
S. Nedevschi, F. Oniga, R. Danescu, T. Graf, R. Schmidt, "Increased Accuracy Stereo Approach for 3D Lane Detection", Proceedings of IEEE Intelligent Vehicles Symposium, (IV2006) , June 13-15, 2006, Tokyo, Japan, pp. 42-49.
Stefan Sobol, Sergiu Nedevschi, "Driving area detection", Poster volume of IEEE 2nd International Conference on Intelligent Computer Communication and Processing, 1-2 September, Cluj-Napoca, Romania, 2006, pp. 89-94