Abstract
The current approach to connected and autonomous driving function development and evaluation uses model-in-the-loop (MIL) simulation, hardware-in-the-loop (HIL) simulation and limited proving ground use, followed by public road deployment of the beta version of software and technology. The rest of the road users are involuntarily forced into taking part in the development and evaluation of these beta level connected and autonomous driving functions. This is an unsafe, costly and inefficient method and has resulted in many problems in the deployment of autonomous vehicles with an associated loss of trust. Motivated by these shortcomings, this project focuses on the Vehicle-in-Virtual-Environment (VVE) method of safe, efficient and low-cost connected and autonomous driving function development, evaluation and demonstration. The VVE method places the vehicle inside a highly realistic virtual environment with realistic virtual sensor feeds while the actual vehicle is physically running inside a large and empty test area. This is as if the vehicle is using a virtual reality headset. It is possible to easily change the virtual development environment and also inject rare and difficult events which can be tested very safely. This is a two-year project and will focus on the use of the VVE method for development and application of Vulnerable Road User (VRU) safety functions.
Pedestrian safety will be treated in the first year of the project and bicyclist safety will be treated in its second year. Our research will start by considering the following five pedestrian crash scenario use cases of: Crossing Roadway – Vehicle Not Turning (FARS 750), Walking/Running Along Roadway (FARS 400), Dash / Dart-Out (FARS 740), Crossing Roadway – Vehicle Turning (FARS 790), Crossing Expressway (FARS 910) during year 1 and the five bicyclist crash scenario use cases of: Motorist Overtaking Bicyclist (FARS 230), Bicyclists Failed to Yield – Midblock (FARS 310), Bicyclist Failed to Yield – Sign – Controlled Intersection (FARS 145), Bicyclist Left Turn / Merge (FARS 220), Motorist Left Turn / Merge (FARS 210) in year 2 where FARS is short for NHTSA’s Fatality Analysis Reporting System.
Vehicle-to-VRU communication-based pedestrian/bicyclist detection which also works for non-line-of-sight cases will be combined with camera and lidar based detection within the VVE method. A data-driven approach will be used to predict the vulnerable road user trajectory which will be compared with the trajectory of the vehicle to predict a future collision possibility. Vehicle trajectory modification to avoid a possible future collision will be developed and evaluated safely using the VVE approach with the vehicle and VRUs at separate locations physically but on a collision risk path in the virtual environment which will enable very realistic evaluation of the designed VRU safety function. Robust and delay tolerant trajectory control will be developed and evaluated using the VVE method also, for executing the calculated collision free modified vehicle trajectory which may involve slowing down, braking or braking and steering. Virtual environments and collision risk scenarios will be developed and evaluated first in MIL and HIL, followed by development and evaluation using the VVE method.
Description
Timeline
Strategic Description / RD&T
This proposal is highly aligned with the Safety research priorities outlined in Chapter 2 (pages 14-22) of the US DOT Research, Development and Technology Strategic Plan 2022-2026 goals. The proposed work will contribute to the zero transportation related fatalities goal of the Safety Grand Challenge introduced on page 15 of the strategic plan. To help contribute to that goal, the proposed project focuses on vehicle automation and connectivity and vulnerable road user safety that are highlighted as critical research topics for the Safety Grand Challenge on page 16 of the strategic plan. This proposal uses connected and automated vehicles that communicate with vulnerable road users to contribute to reducing vehicle and vulnerable road user collisions. Under the heading of Research Priority: Data-Driven System Safety, this proposal contributes to “how technological innovations can reduce and mitigate crashes” on page 18 of the strategic plan by augmenting current perception sensor detection and tracking of vulnerable road users with Vehicle-to-VRU connectivity, by developing vehicle automation controls to avoid possible collisions with VRUs and by developing and using the VVE method to safely develop and evaluate VRU safety functions before public road deployment. This proposal aligns well with the “Identify and support strategies to increase vulnerable road user safety” priority on p. 19 under the Safe Design heading as it develops a VRU safety system that uses Vehicle-to-VRU communication and focuses on improving pedestrian safety (year 1) and bicyclist safety (year 2). This proposal also aligns well with the Safe Technology heading on p. 19 of the strategic plan as it introduces and develops the VVE method to “advance transportation safety by evaluating the safety of existing transportation technologies and supporting the safe integration of emerging technologies” and to “develop test tools, procedures, and performance measures that enable improved safety… evaluations of … vehicles”.
Deployment Plan
Expected Outcomes/Impacts
The VVE method can be used for pre-deployment testing of connected and automated vehicles and their functions in a safe, efficient, low-cost and repeatable manner and has the potential to replace the current testing and certification procedures. It has high potential for use in regulation testing by NHTSA and similar regulating agencies. Its adoption and use by automotive OEMs and technology companies that develop and deploy connected and autonomous driving will help alleviate many of the problems experienced by the public where these technologies are currently deployed on public roads. The VVE method will enable these companies to develop their connected and autonomous driving functions and evaluate them against rare and unexpected events in a very realistic environment safely and efficiently. This will result in their public deployments becoming safer and resulting in less problems and thus improving the safety of transport. The Vehicle-to-VRU communication based VRU safety functions will improve the safety of VRUs in traffic and are expected to have a positive impact fast as readily available mobile phone-based connectivity will be used in implementation.
Expected Outputs
The research and technology outcomes will be the development of the VVE method for evaluating connected and autonomous driving functions with particular focus on VRU safety, a Vehicle-to-VRU communication based VRU safety system, and corresponding conference and journal publications. The project will also contribute to graduate student theses. The VVE environments and scenarios will be created in Unreal Engine and will be simulation environment related outputs of the project. Vehicle and VRU data from VVE development and evaluation of the VRU safety function will also be outputs of the project. Project results will be integrated into the relevant parts of the OSU courses ECE 5553 Autonomy in Vehicles (undergraduate and graduate) and ME 8322 Vehicle System Dynamics and Control (graduate). Exemplary project results will be shared using our lab’s Youtube site. OSU has a utility patent application on the VVE method. More invention disclosures and patent applications on VRU safety are expected.
TRID
We searched TRID using the query “vehicle in virtual environment AND vulnerable road user safety”. The first reference, workshop on safety of powered two wheelers, the third paper, a survey of trust in autonomous vehicles, and the fifth paper on improvements in driving skills of first-time motorcyclists were not seen to be relevant to this proposal. The second reference: “Study of Vulnerable Road-User Choices and Effect of V2P-based Alert on Crossing Behavior through Analysis of Virtual Environment Crossing Events” used a VR simulation for pedestrians to understand their choices of using a crosswalk or jaywalking and tested the effectiveness of V2P based alerts to pedestrians. Our approach is completely different and is based on modifying the behavior of the autonomous vehicle as opposed to alerting the VRU. We also immerse the vehicle in the virtual environment. The fourth paper: “The Reality of Virtual Reality: A Comparison of Pedestrian Behavior in Real and Virtual Environments” compared pedestrian behavior in real and virtual environments. In comparison, we immerse the vehicle in the virtual environment. While these and many other references we investigated in the TRID search are interesting and useful, this search showed the uniqueness of our approach.
Individuals Involved
| Email |
Name |
Affiliation |
Role |
Position |
| aksunguvenc.1@osu.edu |
Aksun-Guvenc, Bilin |
Ohio State University |
Co-PI |
Faculty - Untenured, Tenure Track |
| guvenc.1@osu.edu |
Guvenc, Levent |
Ohio State University |
PI |
Faculty - Tenured |
| redmill.1@osu.edu |
Redmill, Keith |
The Ohio State University |
Other |
Other |
Budget
Amount of UTC Funds Awarded
$57584.00
Total Project Budget (from all funding sources)
$155867.00
Documents
| Type |
Name |
Uploaded |
| Project Brief |
ProjectIntroSlides_Guvenc_2023.pptx |
Oct. 15, 2023, 2:05 p.m. |
| Project Brief |
ProjectIntroSlides_Guvenc_2023.pdf |
Oct. 15, 2023, 2:06 p.m. |
| Data Management Plan |
Data_Management_Plan_VVE.pdf |
Oct. 15, 2023, 2:37 p.m. |
| Project Brief |
AnticipatedDeploymentActivities-422_Guvenc.pdf |
Oct. 16, 2023, 5:26 a.m. |
| Publication |
Deep Reinforcement Learning Based Collision Avoidance of Automated Driving Agent |
March 23, 2024, 3:58 p.m. |
| Publication |
Vehicle-in-Virtual-Environment Method for ADAS and Connected and Automated Driving Function Development, Demonstration and Evaluation |
March 23, 2024, 3:58 p.m. |
| Progress Report |
422_Progress_Report_2024-03-31 |
March 25, 2024, 4:16 p.m. |
| Final Report |
Guvenc_L_422.pdf |
Sept. 12, 2024, 6:52 a.m. |
Match Sources
No match sources!
Partners
| Name |
Type |
| City of Marysville Public Service Department |
Deployment & Equity Partner Deployment & Equity Partner |
