Abstract
The FHWA zero death vision is a strategic goal to eliminate all traffic fatalities and severe injuries. One of the pillars of this strategy is to advance life-saving technology in vehicles and infrastructure. Many such in-vehicle technological advances have been developed and deployed in the past decade such as backup cameras, lane departure warning systems, adaptive cruise control, emergency brake systems, driver attention monitoring, etc. However, deployments of technology in the infrastructure have lagged. Most infrastructure advances have centered around redesigning roads to control traffic and improve visibility of vulnerable road users (VRUs), i.e., anyone in the road environment unprotected by a vehicle's frame. Ideally, the infrastructure should also monitor road activity for unsafe conditions and communicate those conditions to drivers and VRUs. To accomplish this goal, the infrastructure needs to be instrumented with sensors to capture data for real-time analysis and devices for relaying information to all road users. With prior UTC support, we have addressed some of the building blocks needed for a ‘smart’ infrastructure such developing and deploying methods for reliably detecting and tracking vehicles, classifying anomalous activity, and estimating 3D vehicle parameters (speed and direction). These methods were developed using images collected from cameras deployed at intersections. The main challenges that we faced were difficulty identifying a variety of ‘unsafe’ events because they rarely occurred at the deployment locations and difficulty tracking pedestrians and bicyclist because they were rarely captured.
In the proposed work, we will address these challenges by developing a system to simulate unsafe vehicle, pedestrian, bicyclist, and road worker activity from previously acquired images and large language models (LLMs). The proposed work also enables safety technology development without exhausting resources for camera installation. Focus will be on generating visual data of normal and anomalous activities such as near misses, erratic driving, blocked bicycle lanes, etc. Methods will be developed to model realistic driver behavior based on observations from datasets acquired from prior UTC projects in collaboration with deployment partners City of Pittsburgh Department of Mobility & Infrastructure and Shaler Township. Realistic bicyclist behavior will be modeled using bicyclist-centric data provided by partner dashcam.bike. Methods will be developed to leverage these models with LLMs to generate unsafe activity involving vehicles and VRUs based on text entered by users. The system will be deployed to a website so that anyone can create their own dataset. The benefit of such a road activity system will enable safety research and permit data driven evaluation of road or intersection safety without installing cameras and storing/analyzing hours upon hours of images per camera.
Description
Timeline
Strategic Description / RD&T
Section left blank until USDOT’s new priorities and RD&T strategic goals are available in Spring 2026.
Deployment Plan
Q1: Define parameters needed for simulator. Begin collecting and pre-processing vehicle and bicycle data.
Q2: Develop methods to synthesize realistic trajectories based on computed trajectories.
Q3: Begin development of website backend and frontend. Evaluate simulated data in comparison to real data.
Q4: Deploy simulator to website for use by public.
Expected Outcomes/Impacts
It is anticipated that the road activity simulator will be a useful tool for infrastructure safety planning and developing road safety methods without acquiring or using images. Safety is generally evaluated by reviewing crash reports, but the standard for reporting is fairly high. Thus, some crashes are not reported and unsafe interactions are never reported. A text-based simulation environment provides an easy-to-use way of evaluating safety for any road environment.
Expected Outputs
We anticipate the following outputs:
- Methods to simulate unsafe road activity based on real visual data of vehicles and bicycles.
- Easy to use web-based interface to simulate data from text prompts.
TRID
A search for “simulated unsafe road activity” yielded two results [1] The Effects of Instruction and Environmental Demand on State Anxiety, Driving Performance and Autonomic Activity: Are Ego-Threatening Manipulations Effective? and [2] IDENTIFYING UNSAFE DRIVER ACTIONS THAT LEAD TO FATAL CAR-TRUCK CRASHES. Neither publication is directly related to the generation of simulated data generation for unsafe road activity. Publication [1] studies the effect of instruction and environmental demand on anxiety and driving performance. Publication [2] studies unsafe actions by drivers in fatal car-truck crashes. While not directly related to simulating data, their findings can support the modeling of driver behavior for various outcomes.
Individuals Involved
| Email |
Name |
Affiliation |
Role |
Position |
| srinivas@cs.cmu.edu |
Narasimhan, Srinivasa |
Carnegie Mellon University |
Co-PI |
Faculty - Tenured |
| rtamburo@cmu.edu |
Tamburo, Robert |
Carnegie Mellon University |
PI |
Other |
Budget
Amount of UTC Funds Awarded
$90000.00
Total Project Budget (from all funding sources)
$180000.00
Documents
Match Sources
No match sources!
Partners
| Name |
Type |
| dashcam.bike |
Deployment Partner Deployment Partner |
