Principal Investigator

Martial Hebert

Status

Completed

Start Date

Feb. 1, 2012

End Date

Dec. 31, 2013

Project Type

Research Advanced

Grant Program

MAP-21 TSET - Tier 1 (2012 - 2016)

Grant Cycle

2012 TSET UTC

Visibility

Public

Abstract

Abstract:	A smart driving system must include two key components to be able to generate recommendations and make driving decisions that are effective and accurate: 1) The environment of the car and 2) the behavior of the driver. We will investigate advanced concepts for both. Our long-term goal is to develop techniques for building internal models of the vehicle’s static environment (objects, features, terrain) and of the vehicle’s dynamic environment (people and vehicle moving in the vehicle’s environment) from sensor data, which can operate online and can be used to provide the information necessary to make recommendations, to generate alarms, or to take emergency action. Our overall approach is to combine recent progress in machine perception with the rapid advent of onboard sensors, and the availability of external data sources, such as maps. 

Given input (images and 3D) from sensors, the first component of our approach illustrated in Figure 3 relies on recent development in the general area of scene understanding. Specifically, our approach is to extend state-of-the-art machine perception techniques in three areas: 1) scene understanding from images in which objects, regions, and features are identified based on image input; 2) scene understanding from the type of 3D point clouds acquired from, for example, stereo of LIDAR systems; and 3) analysis of moving objects which includes the ability to predict likely future motions in addition to modeling the current trajectory. 

The second key component of our approach is to extend the machine perception techniques to incorporate a complete ensemble of constraints from this application and environments. The technical challenge is to combine data of a statistical and “continuous” nature such as sensor signals and low-level features with knowledge of a symbolic and discrete nature. In fact, our group is leading state-of-the-art research in formal methods to combine statistical and symbolic sources of data in many domains. 

A key development in the automotive industry is the availability of massive amounts of data from a variety of external sources. Accordingly the third component of our approach is to develop techniques to maximize the use of external sources of information. We propose to start by using current map data from navigation systems to generate priors on distribution of features and objects in the environments, and to generate priors of pedestrian and tracking activity. We will anchor the development of this part of our approach on our experience using contextual sources of information. 

While using domain knowledge and contextual information from many sources is not new in machine perception, the exciting revolution afforded by the proliferation of in-car IT and communication elements make it possible to implement this vision. Moreover, the structure of the Center provides the needed expertise in data analytics, communication, trust in data, and other issues, toward a realistic design of a perception system that is truly integrated with all the sources of information.    

Description

-

Timeline

February 2012 - December 2013

Year 1: (a) Algorithms for estimating environment representations from cameras, stereo and Lidar. (b)
Detailed performance analysis on a set of typical scenarios.
Year 2: (a) Framework for incorporating domain knowledge and sources of contextual information. (b)
Implementation using map data to predict prior distribution of object labels. (c) Detailed performance
analysis documenting improvement afforded by the information-access components of the car.

http://utc.ices.cmu.edu/utc/Hebert%20project%20description.pdf 

Strategic Description / RD&T

Deployment Plan

-

Expected Outcomes/Impacts

-

Expected Outputs

TRID

Individuals Involved

Email	Name	Affiliation	Role	Position
hebert@cs.cmu.edu	Hebert, Martial	Robotics Institute	PI	Faculty - Tenured

Budget

Amount of UTC Funds Awarded

$268735.48

Total Project Budget (from all funding sources)

$268735.48

Documents

Type	Name	Uploaded
Final Report	Automatic_Recognition_and_Understanding_of_the_Driving_Environment.pdf	March 21, 2018, 8:11 a.m.

Match Sources

No match sources!

Partners

No partners!