#342 Training Drivers to Automated Vehicles

Principal Investigator
Helen Loeb
Start Date
July 1, 2020
End Date
June 30, 2023
Project Type
Research Applied
Grant Program
FAST Act - Mobility National (2016 - 2022)
Grant Cycle
Mobility21 - University of Pennsylvania


Let’s face it, this is the 21st Century and cars are changing. With the advent of self-driving technology, drivers need to adapt. We propose with this project to tackle a daunting issue:  trust in automation. To that effect, an In Car Mixed Reality Driving Simulator will be used to assess the need to train drivers to ADAS and the tools and methods needed to encourage skeptical drivers of various ages to try and trust automation and self-driving technology.     
For this project, an in-car Mixed Reality Driving Simulator will be used. A training module will be developed to teach the basics of automation from a users perspective. Drivers will be provided with a hands on opportunity to experience various ADAS features such as Emergency Braking, Adaptive Cruise Control, Lane Keep Assist. They will also be able to drive the simulator both in manual and autonomous mode. This one year project will consist in a first technical when the educational component and user interface of the driving simulator are developed. A pilot cohort of 6 participants will then be recruited. We will maintain male/female parity and create 3 age groups: novice drivers (age 16 to 20), adult drivers (age 35 to 50), senior drivers (age 65+). An IRB protocol will be established with the University of Pennsylvania. Participants will be recruited through a questionnaire. We will retain drivers who are skeptical of ADAS and self-driving. Study will consist of 4 steps. 

Step1: semi structured interview on knowledge and perception of self-driving technology. 
Participants will be asked general questions on what they understand of the self-driving technology, what they are willing to try, what they are willing to buy.

Step2: training to ADAS/self-driving on an In Car Mixed Reality Driving Simulator. 
Participants will be seated in a stationary In Car Driving Simulator. They will wear a Virtual Headset and will be taught how to drive normally and how to activate ADAS and autopilot function. They will be taught how to regulate speed while on autopilot and how to return to normal driving. Their interaction with the driving simulator will be recorded for data analysis.

Step 3: semi structured interview for debriefing session.
Participants will be asked to comment on their interaction with driving simulator, what they would be comfortable trying on the road. Participants will be asked if their view of self-driving evolved. 

Step 4: data analysis and research report
Data collected will consist in semi structured interview before simulator experience, interaction with driving simulator, VR eye tracking headset data , semi structured interview after simulator experience. Participants will be video taped during all clinical steps of the study.
A one year project is proposed here:
Q1: Development of Research Protocol with University of Pennsylvania Institutional Research Board (IRB)
Q1: Development of driving simulator scenario and training syllabus 

Q2: Development of driving simulator scenario and training syllabus  (continued)
Q2: Recruitment of participants according to IRB Protocol.

Q3: Testing of participants. 
Q3: Report to Mobility21

Q4: Testing of participants (continued)
Q4: Data analysis, report, and presentation to conferences (AVS, TRB, SAE…)
Strategic Description / RD&T

Deployment Plan
The Research Team will consist in a partnership between University of Pennsylvania with Rahul Mangharam as Pi, and JITSIK LLC which will provide access to simulators.
Penn Team: Dr. Rahul Mangaharam (Professor), Mike Coralluzzi (project manager)  and student
Jitsik Team: Dr. Helen Loeb (CEO), Todd Avery (Driving Instructor), and student

The team will include expertise in embedded systems (Rahul Mangharam), Virtual Reality {Helen Loeb). Dr. Helen Loeb has extensive experience recruiting and testing over 100 participants for driving simulation through her appointment as PI at the Children Hospital of Philadelphia. Driving Instruction will be provided by Todd Avery who is owner and Principal instructor at SafeDrivingCoach.com 

JITSIK LLC has developed immersive in car Mixed Reality Driving Simulators which will provide a realistic immersive in car training environment for ADAS and self-driving instruction. Chroma Key Technology will be used using a pass thru with Virtual Reality Headset HTC Vive Pro.

Research Year will consist in the development of the IRB protocol. While this is developed, a curriculum will be built with Virtual Reality Unity3D. A pilot study of 6 participants will start 6 months into the study. We will maintain Male/Female parity and focus on 3 age groups (16 to 19, 35 to 54, 65 and older). Participants will be recruited through a phone interview. A critical aspect will be their apprehension of the self-driving technology.  Participants will go through: 1) pre drive semi structured interview on their knowledge and perception of ADAS and self-driving 2) virtual ride on stationary In Car Mixed Reality Driving Simulator and training to ADAS and self-driving 3) exit interview on knowledge and perception of ADAS and self-driving.
Expected Outcomes/Impacts
Data analysis will compare Gender and Age Groups (young, adult, senior) for perception and trust in automation. Metrics will be the level of comfort with automation attained by participants and the evolution of their trust in the technology. The team will provide mid year and final report to the UTC, as well as presentations to relevant international conferences such as SAE, TRB, AVS.
Expected Outputs



Individuals Involved

Email Name Affiliation Role Position
LoebH@email.chop.edu Loeb, Helen Children's Hospital of Philadelphia PI Other
rahulm@seas.upenn.edu Mangharam, Rahul Universty of Pennsylvania Co-PI Faculty - Tenured


Amount of UTC Funds Awarded
Total Project Budget (from all funding sources)


Type Name Uploaded
Data Management Plan Data_management_plan_-_Penn_Jitsik_-_Driving_Training_for_AV_9BCA0ep.pdf July 2, 2020, 7:14 p.m.
Presentation Regaining Control of a Vehicle when the Autopilot fails: the age and gender factors Sept. 30, 2020, 6:31 a.m.
Publication 2021_Loeb_VehicleAutomationEmergencyScenario-UsingaDrivingSimulatortoAssesstheImpactofHandandFootPlacementonReactionTime.pdf April 12, 2021, 3:06 p.m.
Publication 10205459.pdf April 12, 2021, 3:06 p.m.
Publication Building trust in self-driving - developing a safe hands on experience for drivers of all ages and abilities April 12, 2021, 2:36 p.m.
Publication Near_Crash.pdf April 12, 2021, 2:43 p.m.
Publication 10205321.pdf April 12, 2021, 2:29 p.m.
Publication Efficacy of automatic emergency braking among risky drivers using counterfactual simulations from the SHRP 2 naturalistic driving study. April 12, 2021, 2:36 p.m.
Presentation Vehicle automation emergency scenario: using a driving simulator to assess the impact of hand and foot placement on reaction time April 12, 2021, 3:06 p.m.
Progress Report 342_Progress_Report_2021-03-31 April 12, 2021, 3:46 p.m.
Progress Report 342_Progress_Report_2021-09-30 Oct. 6, 2021, 10:03 a.m.
Publication Evaluation of Driver’s Sense of Control in Lane Change Maneuvers with a Cooperative Steering Control System Oct. 24, 2021, 8:31 p.m.
Publication Loeb_SAE_2022.pdf Dec. 2, 2021, 5:24 a.m.
Publication Seacrist_AAAM_2021.pdf Dec. 2, 2021, 5:25 a.m.
Publication The Impact of driver distraction and secondary tasks with and without other co-occurring driving behaviors on the level of road traffic crashes. Accident Analysis & Prevention Dec. 2, 2021, 5:26 a.m.
Publication Loeb_SAE_2021.pdf Dec. 2, 2021, 5:28 a.m.
Progress Report 342_Progress_Report_2022-03-30 March 30, 2022, 6:01 a.m.
Presentation SMC_Loeb_Penn_Jitsik_3_17_2022.pdf March 30, 2022, noon
Progress Report 342_Progress_Report_2022-09-30 Sept. 30, 2022, 8:12 a.m.
Publication Drive Right: Autonomous Vehicle Education through an Integrated Simulation Platform March 29, 2023, 2:38 p.m.
Publication shaping.pdf March 29, 2023, 2:38 p.m.
Publication promoting.pdf March 29, 2023, 2:38 p.m.
Presentation 123_Metadrive_flyer.png March 29, 2023, 2:38 p.m.
Presentation Rehab Robotics Workshop March 29, 2023, 2:38 p.m.
Presentation 123_Metadrive_flyer_LftJxir.png March 29, 2023, 2:38 p.m.
Presentation HCPS23_paper_8.pdf March 31, 2023, 12:47 p.m.
Progress Report 342_Progress_Report_2023-03-31 March 31, 2023, 12:47 p.m.
Final Report Final_Report_-_342.pdf Sept. 15, 2023, 11:12 a.m.

Match Sources

No match sources!


Name Type
JITSIK LLC Deployment Partner Deployment Partner