Project: #19 Monitoring and Predicting Pedestrian Behavior Using Traffic Cameras Progress Report - Reporting Period Ending: March 31, 2018 Principal Investigator: Luis E. Navarro-Serment Status: Completed Start Date: Jan. 1, 2017 End Date: Aug. 31, 2018 Research Type: Advanced Grant Type: Research Grant Program: MAP-21 TSET National (2013 - 2018) Grant Cycle: 2017 TSET UTC Progress Report (Last Updated: March 18, 2019, 10:57 a.m.) % Project Completed to Date: 20 % Grant Award Expended: 0 % Match Expended & Document: 0 USDOT Requirements Accomplishments We have developed a video processing pipeline to detect people from images, which is customized for operation with the type of cameras currently used to monitor vehicular traffic. We have also developed an approach to calibrate traffic cameras on-site, which is inexpensive in terms of time and logistics; does not require expensive instruments or software packages; uses a low cost custom-made laser scanner; and can be performed by personnel with minimal training. Using these elements, we have created a software framework that allows us to further develop the algorithms for pedestrian detection and tracking. Additionally, we have applied concepts from Inverse Reinforcement Learning to construct predictive models of how pedestrians traverse an environment in the presence of certain features. These models allow us to forecast the paths that pedestrians may follow near the intersection. Similarly, since the predictive models do not consider the influence of other pedestrians (this is a highly complex problem) we developed a heuristic approach based on the analysis of potential interactions in the direction of motion, which is capable of dealing with environments involving multiple pedestrians and runs in real time. These algorithms were implemented as a software program. Impacts The approaches developed in this effort move us closer to provide traffic intersections with the ability to monitor pedestrian activity. Most traffic intersections currently lack awareness of pedestrian traffic: their perception abilities—when available—are usually limited to the detection of vehicles at very specific places. Video cameras can be used to monitor pedestrian traffic in a setting where a static camera that has an unobstructed view of the road is used to detect and track pedestrians. Typically, a single camera cannot cover the entire area, so multiple cameras are used at each intersection. However, simply detecting pedestrians is not enough: it is also necessary to accurately determine their location within the area surrounding the intersection. The work done in 2016 is key to determine locations using the monocular cameras used for traffic monitoring. We anticipate that our research will have an impact on adaptive traffic light control systems, which currently operate entirely based on information pertaining vehicular traffic. Our work will alleviate the need for timely and accurate information about pedestrian traffic. This is particularly important at locations where it is not uncommon to find more pedestrians than vehicles during certain times of the day. Other A design concept of a low-cost 3D scanner for geometric modeling of traffic intersections. A methodology for traffic camera calibration using a low-cost 3D scanner. Outcomes New Partners No new partners Issues The award for this project was contingent on 1:1 cost share being provided. It was difficult to find sources for these matching funds during 2017. This situation was finally resolved, and the funds were just released in March, 2018. The cost share comes from Prof. Kris Kitani, and it includes his teaching effort; percentages of tuition for two of his students; and of effort of two staff members. All of these people work on topics which are related to this project (specifically, pedestrian detection, localization, activity recognition). The new people involved in this effort are Prof. Kris Kitani (Assistant Research Professor, Robotics Institute); Aashi Manglik and Pengju Jin (graduate students, RI); Eshed Ohn-Bar and Tatsuya Ishihara (staff). The work carried out by them will compensate for the late start with respect to the project's original timeline. No changes are expected in terms of use of funds.