Multi-scale multi-modal perception of dynamic 3D environments based on the fusion of dense stereo, dense optical flow and visual odometry information

Project Code: PN-II-ID-PCE-2011-3-1086;
Contract No: 344 din 01/01/2012

Accurate artificial sensing of complex dynamic environments has been the long time focus of many research teams, this field receiving constant attention from governments and industry. A popular passive sensor for environment perception, stereovision delivers large quantities of information, but the density and the accuracy of the 3D data depends on the quality of the processing algorithms. This project’s aim is to bring significant improvements to the field of stereovision-based perception of complex dynamic environments, by establishing the mathematical foundation and by designing and implementing original solutions for multi-scale multi-modal perception and representation of structured and unstructured environments, based on the fusion of high accuracy dense stereo, high accuracy dense optical flow and highly accurate visual odometry information. The project will bring original contributions in the field of stereovision and optical flow computation from image sequences, and will combine these results towards extracting accurate 3D features and their 3D speed vectors. The increased accuracy and density of the dynamic 3D features will be exploited in new solutions for higher level perception of the environment, featuring new models for dynamic world representation, and new probabilistic perception and tracking techniques. The original results will be proven on mobile demonstrators, and disseminated in high impact publications.