Principal Investigator

Srinivasa Narasimhan

Status

Completed

Start Date

Jan. 1, 2016

End Date

Dec. 31, 2016

Project Type

Research Advanced

Grant Program

MAP-21 TSET National (2013 - 2018)

Grant Cycle

2016 TSET UTC

Visibility

Public

Abstract

Annual crash statistics continue to reveal the disturbing trend that driving at night is very
dangerous despite nearly a century of automotive headlight development. Even with recent
advances in adaptive headlight technology, a majority of accidents occur at night when there is
less traffic on the road [1]. Our programmable headlight overcomes some of their functional and
performance limitations by being versatile, i.e., capable of being programmed to perform many
different types of tasks to increase safety for all drivers on the road [2]. With previous support
from the T-SET UTC, we have developed a single headlight prototype capable of operating at
highway speeds. We have also developed several application algorithms and demonstrated them
on the road.

The goal of the proposed research is to design and develop a stereoscopic programmable
headlight. The addition of a second headlight will enable the use of 3D computer vision methods.
More accurate algorithms for anti-glare high beams, seeing through rain and snow, and obstacle
spotlighting will be developed. Algorithms for new applications such as dynamic beam forming
and scene reconstruction will also be developed. We will build a prototype and develop software
for the stereoscopic headlight system. All algorithms developed will be demonstrated on the
prototype. For the duration of the award period, a custom embedded solution will be developed to
reduce the cost, size, and energy consumption of the headlight.    

Description

Background

With previous support from the T-SET UTC, we have made significant improvements in the design,
performance, and functionality of our single programmable headlight design. In the past year, the reaction
time is now deterministic and 50% faster. We performed thorough evaluation of the system’s latency and
conducted experiments demonstrating that the system can react to high-speed events [10]. The physical
size of the prototype has been significantly reduced permitting easier installation on a vehicle for road
testing and demonstrations (Fig. 1). We have also demonstrated the potential safety benefits of our
headlight with four separate application algorithms: reduced high-beam glare, improved visibility in
snow, improved driving lane visibility, and early visual warning of bicyclists (Fig. 2). We propose
developing a system with two programmable headlights, which will permit distance estimation
resulting in more accurate algorithms for improved road safety.

Safety Impact

The U.S. National Highway Safety Administration reports that more than half of the vehicle crashes and
fatalities occur at night despite significantly less traffic during those [1]. In many scenarios, for example,
dark and narrow rural roads, bright headlights are required to safely see the driving environment (e.g.,
edge of the road, wildlife, pedestrians, etc.) especially when traveling at high speeds. Unfortunately,
bright headlights also cause significant glare to other drivers, bicyclists, and pedestrians on the road.
During rain and snowstorms, they also cause distracting bright flickering streaks. Thus, a headlight that
adaptively illuminates the road environment without causing distractions would be expected to improve
driver visibility and safety at night and during poor visibility conditions.

Research Objectives

In previous work, a prototype of a single programmable headlight was designed, built, and road-tested. In
the proposed research (2 years) emphasis will be on developing a stereoscopic programmable
headlight with improved algorithm accuracy and new application areas. To achieve these goals, in
the first year, we will address both the hardware and software components of the headlight design
beginning with a focus on designing and building a dual-headlight system. A second programmable
headlight (i.e., camera) is advantageous because the distance to detected objects can be estimated to assist
in classifying objects and reducing false positives in our current algorithms (Fig. 3). Furthermore, threedimensional computational methods such as reconstruction and scene understanding can be employed.
When necessary, methods for fusing data from other sensors (GPS, RADAR, etc.) will be developed and
integrated with the system. During the second year of the award, additional application algorithms will be
developed. All algorithms will be tested on the road when possible. Over the entire 2-year award period,
an embedded FPGA-based system will be under development to reduce the size, cost, and power
consumption of the system.

Research Objectives for Year 1

Stereoscopic Headlight Prototype: Previous work building single headlight prototypes will be leveraged
to expedite production time allowing for more time to develop algorithms and perform road tests. The
stereoscopic headlight system will consist of two single headlight prototypes connected to a computer
with newly developed system software. Software will take as input images from each headlight, process
the images and control the headlights independently. We will develop a systematic method for calibrating
the headlights with epipolar geometry, which will permit computationally efficient implementations of
3D computer vision methods.

Distance Estimation: A computationally efficient algorithm for estimating distance for the headlight pair
will be developed using passive stereoscopic vision, which will 1) remove distortion from the image pair,
2) project the images to a common plane, and 3) estimate the position of features in the two images.

Glare-Free High Beams: Driving faster than 45 miles per hour surpasses the reach of low-beam headlights
seriously compromising reaction time. High beams provide an additional 100 feet of visibility, but are
used less than 25% of the time under ideal conditions (dark, rural roads) [11]. Several reasons for high
beam underuse include fear of glaring oncoming drivers and not wanting to switch between high and low
beams. Approximately 30% of drivers are stressed by glare from oncoming vehicle headlights causing
significant fatalities every year [3, 4]. Glare is especially problematic for the elderly whom take eight
times longer to recover from glare as compared to a 16-year old [5]. We have demonstrated with road
tests that our programmable headlight can address all of these issues. Rather than toggle between high and
low beams, our headlight automatically disables light directed towards oncoming drivers. LED-based
headlights perform similar function, but due to the lack of spatial resolution have limited light throughput
and do not work well in the presence of many oncoming vehicles (Fig. 4). The algorithm developed in
previous work had a tendency to incorrectly identify vehicles. In this work, we will develop an algorithm
to better spatially localize and classify vehicles by utilizing 3D information and data from a RADAR
sensor. The algorithm will be deployed with the prototype in multiple road environments.

Early Visual Warning of Pedestrians: Every year, there are hundreds of thousands of crashes with objects
on the road. In 2013, there were 71,000 crashes with pedestrians and 69.8% of pedestrian fatalities
occurred at night [1]. Crashes with pedestrians often occur due to poor peripheral visibility on the side of
the road and slow reaction times of the driver. We have shown that our previous headlight design reacts
fast enough (750x faster than the average persons reaction time) to act as a peripheral spotlight drawing
the driver's attention to objects on the road. During this award period we will collaborate with
Bernardo Pires (CMU, RI) to develop an accurate vision-based algorithm for pedestrian detection.
The algorithm will be optimized for speed and tested on a moving vehicle in various road environments.

Research Objectives for Year 2

Improving Visibility in Poor Weather: Annually, more than 300,000 crashes and thousands of fatalities
are caused by rain and snow at night [1]. Driving in a snowstorm at night is incredibly difficult and
stressful. Snowflakes are illuminated brightly and distract the driver from observing the entire road.
Researchers in computer vision have proposed methods for removing snow from videos [6 - 8]. We have
previously demonstrated improved visibility outside at night with free-falling artificial snowflakes. While
we have previously demonstrated that avoiding illumination of artificial snowflakes is feasible,
experiments were not conducted in driving conditions where motion and turbulence make snowflake
prediction more challenging. Algorithms will be developed to incorporate better prediction methods.

Improving Road Visibility: Sometimes the road is not clearly visible and no amount of illumination from
a standard headlight can assist the driver. A few examples of such situations are snow-covered roads,
roads without lane markings or shoulders, and poorly lit roads. Our current prototype can be used to
brightly illuminate only the driver's lane to provide them with a visual guide to their destination. In
previous demonstrations, the road topology was built into the system. In this work, we will integrate a
global positioning sensor (GPS) and geographic maps into the system for real-time localization updates
anywhere with the availability of GPS satellite connectivity and high-resolution mapping data.

Early Visual Warning of Bicyclists: In 2013, there were 49,000 crashes with bicyclists [1]. Crashes with
bicyclists are likely to grow with a growing interest in biking and electric vehicles [12]. During this award
period, we will focus on developing algorithms to accurately detect and spotlight bicyclists. We will
collaborate with Anthony Rowe (ECE, CMU) to leverage his experience with bicycle safety.

Timeline

Year 1:
Hardware Prototype (Jan-Mar)
System Software (Mar-Apr)
Calibration (Apr)
Distance Alg (May)
First Embedded FPGA System (Jan-Aug)
Glare Free Algorith (June-Sept)
Pedestrian Spotlighting Algorithm (Sept-Dec)
Second Embedded FPGA System (Sept-Dec)
FPGA Implementation of Glare-Free (Sept-Dec)

Year 2: 
Road Vis Alg (Jan-Feb)
Road Vis. (Mar-Jun)
Bicyclist Algorithm (Apr-July)
Second Embedded FPGA System (Jan-July)
3D Object Reconstruction (July-Dec)
FPGA Implementation of Select Algs (Aug-Dec)
Snow Algorithm (Aug-Dec)

Strategic Description / RD&T

Deployment Plan

We already have collaborations with Traffic 21 (UTC TSET), NSF funded CPS project partners in RI and
ECE (Hebert, Bagnell, Mertz), collaborations with industry (funding from Intel, Ford and likely GM), and
endorsements for collaborations with Pittsburgh city and local partners like GASP and CMU CREATE
Labs to study environmental effects (air quality) of traffic and industrial sources. The funding from the
UTC will help put together a larger and longer-term effort in solving the transportation safety problems in
America.

Expected Outcomes/Impacts

At the end of the year, we expect to have a prototype of a stereoscopic programmable headlight system
with software optimized for speed. Algorithms will be developed for estimated the distance to object, and
for performing a variety of tasks. The prototype will be mounted to a vehicle and each of the algorithms
will be road tested. Results will be published and demonstrations will be given.

Expected Outputs

TRID

Individuals Involved

Email	Name	Affiliation	Role	Position
srinivas@cs.cmu.edu	Narasimhan, Srinivasa	Robotics Institute	PI	Faculty - Research/Systems
ssankar1@andrew.cmu.edu	Sankar, Srihari	ECE	Other	Student - Masters
rtamburo@cmu.edu	Tamburo, Robert	Robotics Institute	Co-PI	Faculty - Research/Systems

Budget

Amount of UTC Funds Awarded

$80000.00

Total Project Budget (from all funding sources)

$80000.00

Documents

Type	Name	Uploaded
Progress Report	30_Progress_Report_2016-12-31	Oct. 5, 2017, 10:03 a.m.
Final Report	30_final_report_stereo.pdf	Sept. 28, 2018, 5:20 a.m.

Match Sources

No match sources!

Partners

No partners!