Principal Investigator

Luis E. Navarro-Serment

Status

Completed

Start Date

Jan. 1, 2016

End Date

Dec. 31, 2016

Project Type

Research Advanced

Grant Program

MAP-21 TSET National (2013 - 2018)

Grant Cycle

2016 TSET UTC

Visibility

Public

Abstract

Effective traffic monitoring systems must take into account all moving objects. While
vehicle detection systems are common, sensor-based solutions that provide
awareness of pedestrian activity have yet to become an integral part of a smart and
effective infrastructure capable of protecting these most vulnerable traffic
participants. Such a system could alert incoming vehicles about dangerous situations
involving pedestrians, or provide adaptive traffic light control systems with
information about the motion of people, so that they can operate and make decisions
cognizant of all moving objects.

In our previous research work we have developed a vision-based framework which is
capable of detecting, tracking, and predicting the trajectories of people. These
abilities constitute the core of a system capable of monitoring any developments
concerning pedestrian motion. In this project, we aim at positioning our research for
deployment. We plan to enhance our algorithms to make them suitable for operation
in real traffic situations, in terms of accuracy and robustness against adverse
operating conditions. Additionally, we will address the difficulties of bringing a new
site into effective action, such as camera calibration and the initial identification of a
context to make predictions. 

The expected outcome is an implementation of the
enhanced framework which can be deployed in the field, and is capable of providing
pedestrian information to Surtrac1—a real time traffic signal control system.
The milestones envisioned, to accomplish in two years, are: 1) Development of
enhanced approach for pedestrian detection; 2) Implementation of prototype suitable
for operation in real time; and 3) Testing and characterization of prototype in the
field. Our team is composed of a Master's student, and researchers with many years
of experience in computer vision for autonomous systems.    

Description

This project addresses the need for timely and accurate information about pedestrian
traffic in urban areas. This is particularly important at locations where it is not
uncommon to find more pedestrians than vehicles during certain times of the day.

Through the work proposed in this project, we move to position our research towards
deploying a system for detecting, tracking, and forecasting human behavior in
dynamic environments. This information can be used by other systems as part of an
infrastructure-based framework to effectively protect the more vulnerable traffic
participants.

During recent efforts (Metro21), we have transitioned our software infrastructure
from using LIDARs to video cameras as the primary sensor. We have developed a
software framework where a static camera that has an unobstructed view of a road
intersection is used to detect and track the motion of pedestrians. This represents the
first level of awareness, i.e. how many people are present in the area, and their
current location, as well as their speed and direction of motion. Furthermore, by
learning from training data relationships between pedestrian paths and features in
the environment, we can model explicit preferences in motion patterns, such as
destinations and preferred routes. These models are used to predict the trajectories
of pedestrians further in the future than what it is achievable using simple motion
models. This constitutes the second level of awareness: the most likely paths to be
traversed by the pedestrians. A sample output from our framework is shown in Fig. 1.
Vision sensors are commonly used at traffic intersections: they have small space and
power requirements, and their performance and robustness continues to improve as
their cost becomes lower. They are suitable for widespread use. However, these
sensors are more susceptible to extreme (i.e. too bright or too dim) or adverseillumination conditions (e.g. cast shadows induce color and illumination
discontinuities in the image). Moreover, the accurate and robust detection of
pedestrians from images is more challenging as the distance from the camera
increases, or when cameras have wider fields of view (as is frequently the case with
traffic cameras), which make objects on the scene to appear smaller in size. To
address these issues, work still needs to be done to integrate our current algorithms
in a way that its successful deployment is feasible. Hence, the goal of this project is
the design and implementation of a software framework that processes video in real
time to detect, track, and predict the trajectories of pedestrians at the site; that is
robust and accurate over a wide range of operating conditions; that is easily installed
and calibrated at every site; that is easy to maintain and diagnose on-site; and that
can communicate the detection, tracking and prediction results to other users.

To accomplish this goal, we will leverage different elements developed in other
projects. We will use algorithms designed to detect poor or adverse illumination
conditions, and to compensate for them to support pedestrian detection. We will also
apply methods to extract priors from external sources of information, which can
provide a richer initial context for predictions at a particular site. Additionally, we will
adapt methodologies for intrinsic and extrinsic camera calibration for use at the site
of each traffic camera. Finally, we will leverage our recent work on modeling
interactions between pedestrians to extend our prediction framework, originally
capable of producing predictions for each mover independently of other pedestrians,
to generate multi-agent predictions using simple heuristics.

Timeline

The work is planned over a 24-month time frame.

Strategic Description / RD&T

Deployment Plan

To understand and solve the issues involved in the deployment of our technology,
and to evaluate the beneficial effects of pedestrian-related information, we have
approached Rapid Flow Technologies, creators of Surtrac. We will collaborate with
them in several ways: obtain selected examples of data for analysis; discuss the
issues involved with the deployment of your system using actual traffic
infrastructure;collaborate with the design of an interface with Surtrac; and
receive feedback on the usefulness and practicalities of our system.

Expected Outcomes/Impacts

By the end of the project we expect to have a functional implementation of the
enhanced algorithmic framework for pedestrian detection, tracking, and
prediction, which is also capable of publishing information. This will enable us to run
reliability tests and to carry out experiments to evaluate its performance, and to
study and evaluate its usefulness by other applications.
As for evaluation metrics, the primary objective is to detect the presence and
behavior of people accurately and robustly in real time. The pedestrian detection
accuracy will be measured using the percentage of false and missed detections
over time. The accuracy of the predictions will be assessed by comparing the
probabilistic distribution over paths generated by our models with the actual paths
demonstrated by pedestrians.
The secondary objective is to provide information to other systems (e.g.
SURTRAC, driver assistant systems) to enhance their performance. Therefore, we will
compare each system’s performance measures with and without our application to
evaluate and quantify the improvements.

Expected Outputs

TRID

Individuals Involved

Email	Name	Affiliation	Role	Position
hebert@cs.cmu.edu	Hebert, Martial	Robotics Institute	Co-PI	Faculty - Tenured
lenscmu@ri.cmu.edu	Navarro-Serment , Luis E.	Robotics Institute	PI	Faculty - Research/Systems

Budget

Amount of UTC Funds Awarded

$79752.00

Total Project Budget (from all funding sources)

$79752.00

Documents

Type	Name	Uploaded
Final Report	065_finalReport.pdf	Aug. 17, 2018, 4:04 a.m.

Match Sources

No match sources!

Partners

No partners!