Project: #304 Image Processing Approaches to Traffic Situation Understanding, Risk Assessment, and Safety Progress Report - Reporting Period Ending: March 31, 2020 Principal Investigator: Keith Redmill Status: Active Start Date: March 1, 2019 End Date: June 30, 2022 Research Type: Advanced Grant Type: Research Grant Program: FAST Act - Mobility National (2016 - 2022) Grant Cycle: Mobility21 - The Ohio State University Progress Report (Last Updated: March 31, 2020, 3:07 p.m.) % Project Completed to Date: 38 % Grant Award Expended: 33 % Match Expended & Document: 48 USDOT Requirements Accomplishments This project is exploring several potential applications of image processing, including neural network/deep learning technologies, to the analysis of traffic scenes involving passenger and transit vehicles. Topic 3: Optical Flow for Automated Vehicle Control We have created a working online pipeline for vehicle control that is solely based on optical flow information obtained from monocular camera frames. In this period we completed the obstacle detection and avoidance portion of the study. A visual potential field that combined image plane (vertical plane) information and motion plane (horizontal plane) information was successfully implemented and tested in simulated driving environments with and without obstacles. Latest results show it can be a low cost and easy to implement method supplementary to other control actions when the vehicle loses, for example, localization information. Nor surprisingly, there are limitations with this approach, especially with respect to obstacle detection and avoidance. Optical flow works reasonably well at vehicles speeds above 15mph, but is not reliable for vehicle control at low speeds, so it can not be the sole sensing and control approach for an automated vehicle. In addition, optical flow does not provide absolute depth information, so additional constraints, and in particular a time-to-collision formulation, are needed for successful obstacle avoidance. Topic 1: Traffic Scene Assessment and Driver Behavior Understanding Currently working on creating a large subset of public driving data-sets for training a neural network based unsupervised behavior extractor. This behavior extractor will be used for traffic scene risk assessment and driver behavior understanding. Ekim Yurtsever, the postdoctoral researcher working on this task, was the recipient of the Young Researcher Award from the IEEE ITS Society Nagoya Chapter. Topic 2: Extracting Traffic Information from Transit Vehicle Video The postdoctoral researcher who is assigned to this topic arrived at the end of 2019. Activity on this task has recently begun with a series of meetings between the Civil Engineering Traffic faculty and the postdoc. After exploring sample video, they are exploring with Christoph Mertz and his student at CMU the use of their video processing techniques for this application. At this point, initial discussion and technology transfers have occurred. Impacts During this period: Topic 3- two conference papers have been submitted on this activity. Topic 1- one conference paper has been published on this activity. Topic 2- none at present Other none Outcomes New Partners Topic 2- Christoph Mertz and Justin Chiang (CMU Robotics Institute) Issues Adjusting to working from home and a lock-down of OSU buildings is an ongoing challenge, especially for researchers using specialized computing resources that must remain on-campus. OSU has still not received the year-4 funding increment. The postdoc supported by this project has joined OSU in late 2019.