Login

Project

#104 In-Vehicle Vision-Based Cell Phone Detection


Principal Investigator
Bernardo Pires
Status
Completed
Start Date
Jan. 1, 2015
End Date
Dec. 31, 2015
Project Type
Research Advanced
Grant Program
MAP-21 TSET National (2013 - 2018)
Grant Cycle
2015 TSET UTC - National
Visibility
Public

Abstract

According to the NOPUS survey, at any given daylight moment across America approximately 660,000 drivers are using cell phones or manipulating electronic devices while driving. According to the same survey, this number has held steady since 2010, despite major investments into awareness programs and numerous changes in legislation. Recently, there has been significant interest into automatic detection of driver distraction. Such research often focuses on the driver’s eyes in an attempt to detect gaze direction (determine where the driver is looking at.) The difficulty with such approach is that it either requires active infrared illumination, which can be “blinded” by the sun, or requires significant computation to recognize the driver’s face, determine pose, and estimate gaze. Furthermore, this approaches often requires high-resolution cameras in order to be able to accurately observe the user’s eyes. Instead of focusing on the driver’s eyes, we propose to obtain an overhead or over-the-shoulder view of the car interior with the objective of determining if the driver is holding or using a cell phone or other electronic device. We expect this to be a superior method because the screen is often illuminated, relatively large and, when in use, turned directly towards the user’s head and, consequently, to our over the shoulder camera.
    
Description
We have all heard the statistics about distracted driving and how that endangers everyone on the road. In 2012 alone, more than 3,300 people were killed in distraction-affected crashes. For the same year, an estimated 421,000 people were injured in motor vehicle crashes involving a distracted driver [1]. Secretary of Transportation Foxx has called this problem a “deadly epidemic of distracted driving.” The National Occupant Protection Use Survey estimated that the percentage of drivers text-messaging or visibly manipulating hand-held devices stood at 1.3% in 2011, while driver hand-held cell phone use stood at 5% for the same year. [2]

Multiple governmental agencies, automakers and suppliers have dedicated significant resources to detecting driver distraction and implementing warning systems. For example, Volvo has recently announced a driver state estimation system, “which casts infrared light upon the driver’s face, monitored by sensors that detect the driver’s eye gaze, head movement, head angle, and how open his or her eyes are” [3]. These types of systems attempt to determine if the driver is sleepy or inattentive and adjust automatic systems such as lane keeping, collision warning, and adaptive cruise control to match the driver’s state.


Current Driver Monitoring Systems

Most driver-monitoring systems focus on the driver and, in particular, on the driver’s eyes. The eyes give important queues to the driver’s internal state and perception. Eye closing can be an early sign of fatigue or drowsiness, and often road signs are missed unless the driver directly gazes at them. On the other hand, it can be challenging to produce accurate estimates of the gaze direction in all driving environments and for all possible drivers.

In car gaze estimation systems can be divided in two categories. Active systems (such as the one by Volvo described above) use infrared light to illuminate the driver. The presence of this illumination facilitates pupil detection, but leaves the system subject to interference from other infrared light sources such as the sun. Passive detection, on the other hand, uses one or more “regular” camera pointed at the driver and often requires that the head pose be estimated before the gaze direction can be computed. Such estimation under arbitrary illumination conditions can be a difficult computer vision problem and require a significant amount of computation resources (which might not be available in a consumer grade vehicle.) 

Further, all gaze estimation approaches require fairly high-resolution cameras trailed at the driver so that the eyes can be correctly imaged, which further increases their deployment cost. Additionally, in either case eyewear, especially shaded, can confound the system and prevent estimation.


Proposed approach

We focus on the detection of active manipulation of electronic devices by the driver. This is often the most dangerous driver behavior and the one that leads to many of the distracted-driver accidents. Instead of focusing on the driver’s eyes, we propose to obtain an overhead or over-the-shoulder view of the car interior with the objective of determining if the driver is holding or using a cell phone or other electronic device. 

Compared with the approaches that focus on the driver’s gaze, we expect this to be a superior method because the detection of the unsafe behavior is much easier. Specifically there are three advantages in detecting a screen versus tracking the user’s eyes. First, because the screen is illuminated its detection is much easier as it will stand out versus the background. Second, the screen is much larger than the user’s eye and thus our method does not require as high-resolution cameras. And finally, because our camera is placed overhead, when the device is in use it will be turned directly towards our camera.



References

[1] Official U.S. Government Website for Distracted Driving. www.distraction.gov. Accessed November 2014.

[2] Pickrell, T. M., & Ye, T. J. (2013, March). Driver Electronic Device Use in 2011. (Report No. DOT HS 811 719). Washington, DC: National Highway Traffic Safety Administration.

[3] Steve Siler. Volvo Tests Driver Alertness With Face-Monitoring Technology—Yeah, It’s Creepy. Car and Driver. March 18, 2014
Timeline
January to December 2015
Strategic Description / RD&T

    
Deployment Plan
N/A
Expected Outcomes/Impacts
- Collection of a dataset of over-the-shoulder views of users while driving and also when parked and using their cell phone (naturally, we will not record instances of cell phone usage while driving.)
- Creation of a fully automatic in-vehicle monitoring system capable of detecting cell phone usage and evaluating it on the compiled dataset.
Expected Outputs

    
TRID


    

Individuals Involved

Email Name Affiliation Role Position
bpires@cmu.edu Pires, Bernardo Robotics Institute PI Faculty - Research/Systems

Budget

Amount of UTC Funds Awarded
$100027.00
Total Project Budget (from all funding sources)
$100027.00

Documents

Type Name Uploaded
Data Management Plan Pires_Phone_Detection_Proposal_6Qy8MCT.doc Oct. 13, 2017, 7:45 a.m.
Final Report 104_-_Pires_PhoneDetection_FinalReport.pdf Feb. 21, 2019, 4:39 a.m.

Match Sources

No match sources!

Partners

No partners!