Automatic Recognition and Understanding of the Driving Environment for Driver Feedback

Source Organization:
University:Carnegie Mellon University
Principal Investigator:Martial Hebert
PI Contact Information:
Project Manager:Courtney Ehrlichman
Funding Source(s) and Amounts Provided (by each agency or organization):Research and Innovative Technology Administration, University Transportation Centers Program, US Department of Transportation
Total Dollars:$211,205
Agency ID/Contract/Grant Number:DTRT12GUTC11
Start and End Dates:February 2012 - December 2013
Project Status:Complete
Subject Categories:Feedback
Abstract:A smart driving system must include two key components to be able to generate recommendations and make driving decisions that are effective and accurate: 1) The environment of the car and 2) the behavior of the driver. We will investigate advanced concepts for both. Our long-term goal is to develop techniques for building internal models of the vehicle’s static environment (objects, features, terrain) and of the vehicle’s dynamic environment (people and vehicle moving in the vehicle’s environment) from sensor data, which can operate online and can be used to provide the information necessary to make recommendations, to generate alarms, or to take emergency action. Our overall approach is to combine recent progress in machine perception with the rapid advent of onboard sensors, and the availability of external data sources, such as maps.

Given input (images and 3D) from sensors, the first component of our approach illustrated in Figure 3 relies on recent development in the general area of scene understanding. Specifically, our approach is to extend state-of-the-art machine perception techniques in three areas: 1) scene understanding from images in which objects, regions, and features are identified based on image input; 2) scene understanding from the type of 3D point clouds acquired from, for example, stereo of LIDAR systems; and 3) analysis of moving objects which includes the ability to predict likely future motions in addition to modeling the current trajectory.

The second key component of our approach is to extend the machine perception techniques to incorporate a complete ensemble of constraints from this application and environments. The technical challenge is to combine data of a statistical and “continuous” nature such as sensor signals and low-level features with knowledge of a symbolic and discrete nature. In fact, our group is leading state-of-the-art research in formal methods to combine statistical and symbolic sources of data in many domains.

A key development in the automotive industry is the availability of massive amounts of data from a variety of external sources. Accordingly the third component of our approach is to develop techniques to maximize the use of external sources of information. We propose to start by using current map data from navigation systems to generate priors on distribution of features and objects in the environments, and to generate priors of pedestrian and tracking activity. We will anchor the development of this part of our approach on our experience using contextual sources of information.

While using domain knowledge and contextual information from many sources is not new in machine perception, the exciting revolution afforded by the proliferation of in-car IT and communication elements make it possible to implement this vision. Moreover, the structure of the Center provides the needed expertise in data analytics, communication, trust in data, and other issues, toward a realistic design of a perception system that is truly integrated with all the sources of information.
Describe Implementation of Research Outcomes (or why not implemented):
Impacts/Benefits of Implementation (actual, not anticipated):Research in progress
Project URL: