Abstract
The proposed project aims to prevent fatal rear and side crashes related to heavy-duty tractor-trailer combinations. Specifically, we propose to develop and test smart trailer sensors/vision systems that infer "dynamic safety zones” and use lighting signals (or other communication modes) to alarm following and overtaking vehicles, pedestrians, or other non-occupant situations. The proposed trailer sensors/vision systems automatically analyze videos, vehicle size, and loading and brake data to infer collision risks between tractor-trailer combinations and approaching vehicles and people. From 2019 to 2021, fatal rear crashes with large trucks with trailers, where passenger vehicles travel under the rear of the truck, increased from 16.8% to 18.0%. In 2021, other vehicles in the large truck lane (26.5%) and others encroaching into the large truck lane (36.0%) were the two critical pre-crash events that caused such crashes. Drivers usually underestimate the required distance when the safe distance suddenly increases because of the large weights and sizes of the vehicles, unexpected pavement conditions, and terrains that require extra separations between vehicles. Inter-vehicle dynamic safety zones change and differ by situations and changes over time, so manually estimating the safe following and overtaking distances could be unreliable. Sometimes, illusions, slippery caused by weather, and poor lighting conditions can bias human estimates and make the reaction too late to stop. The recent integration of computer vision and motion sensors has shown the potential to improve passenger vehicles. However, heavy-duty vehicles, especially trailers, need special consideration of vehicle size, motion planning, road conditions, and occlusions to ensure a reliable assessment of side and rear collision risks in different positions of the tractors and trailers.
The proposed project will integrate the expertise of the project team and two industry partners in developing and testing an intelligent tractor-trailer sensor and vision system and provide benchmark datasets. In construction and airport safety, the project team has integrated computer vision, robotic motion simulation, and spatiotemporal analyses to implement dynamic safety zone estimation solutions for aircraft and construction equipment. The project team has also developed the technique to find safe actions when there is uncertainty in the dynamic system models or environments. The proposed project will adapt these intelligent dynamic safety zone estimation solutions to implement the proposed smart sensors and vision systems on tractor-trailer combinations. An industry collaborator, Clarience Technologies, will work with the project team to use their tractor and trailer fleet to collect video, vehicle, and telematics data to support the development and testing of the proposed smart safety system. Clarience will also leverage its automotive and vehicular engineering background to support the 4D simulation and motion analysis of heavy-duty vehicles in given road and terrain conditions. Another industry partner, Safety Emissions Solutions, has collaborated with the team in integrating inspection reports, crash, and telematic data into ‘vehicle deterioration models’ that predict the crash risks of heavy-duty vehicles. Integrating this expertise, software, data, and hardware from the researchers and industry will ensure the timely delivery of the proposed dynamic safety zone estimation solution and the benchmark data sets.
Description
Timeline
Strategic Description / RD&T
The proposed project centered on the goal of “Safety” stated in the US DOT Research, Development, and Technology Strategic Plan (the RD&T Plan hereafter; page numbers below are those in this RD&T Plan) while also helping with the goal of “Transformation.” On the “Safety” aspect, the proposed video, motion, and telematics data-driven prediction of dynamic safety zones of heavy-duty truck-trailer combinations can enhance “Vehicle and aircraft safety, automation, and connectivity” (page 16). The proposed integration of trailer camera data into the telematics and driver assistance system is related to the priority of “Data-Driven System Safety” (page 18). Specifically, this part of the proposed work is “essential for understanding the systemic causes of transportation challenges” and for studying “how technological innovations can reduce and mitigate crashes” (page 18). Multi-source data analytics of multiple moving vehicles in changing environments can reveal critical vehicle attributes, components, and environmental conditions that can trigger losses of separations between heavy-duty vehicles and passenger vehicles. Also, the pilot studies with the tractor-trailer fleets of the participating industry partners can reveal deployment opportunities and help develop policy suggestions to ensure the effectiveness of the deployment of the proposed smart sensor and camera systems at the fleet, state, and national levels. Such information can help investigate “how regulatory and policy tools can reduce fatalities and injuries” (page 18). On the “Transformation” aspect, the proposed smart sensor and cameras for real-time alarms of dynamic safety zones between heavy-duty trucks, other vehicles, and people can fill a significant gap in the “Digital Infrastructure” to “establish a fully functional, reliable, and secure foundation of transportation system digital infrastructure” (page 50). Integrated video and smart sensor data analytics can also generate “Data-driven Insights” to “create timely, accurate, credible, and accessible information to support transportation operations and decision-making” (page 50).
Deployment Plan
For this project, we will collaborate with the following organizations (two confirmed deployment partners, one confirmed equity partner, and three additional partners potentially to develop relationships):
1. Quarter 1: Sensor and video data collection with the tractor-trailer fleet owned by Truck-Lite (Truck-Lite collaboration activities detailed below).
2. Quarter 2: Technology implementation for Truck-Lite vehicles based on the collected data (Truck-Lite collaboration activities detailed below).
3. Quarter 3: Discussion with Safety Emissions Solutions, LLC, and additional industry partners about the opportunities of implementing the developed system for their vehicles (Truck-Lite and Safety Emissions collaboration activities detailed below).
4. Quarter 4: Implementation and policy discussions with DOT and professional organizations, including equipment management organizations (DOT and professional organization collaboration activities detailed below).
Clarience Technologies – Clarience Technologies is a confirmed deployment partner who will directly assist the project team in developing and testing the proposed smart cameras on trailers, focusing on assessing deployment capabilities and challenges. Truck-Lite is a telematic system vendor targeting heavy-duty vehicle fleets. It is a subsidiary of Clarience Technologies LLC, a truck safety component and system company with a global R&D center in Pittsburgh. Truck-lite has been collaborating with the project team for over two years. It has been making lights for medium and heavy-duty trucks for over 60 years and has recently entered the telematics space for commercial trucking through its Road Ready system. The Road Ready system (https://www.roadreadysystem.com) is a new hardware and services platform that collects data from various onboard sensor components (including all safety components, not just lights) on trucks and trailers. This platform communicates the data and status information to the driver and centrally to fleet managers. For example, a typical trailer has about 20 lights, and anyone being inoperable results in a failed inspection and safety concerns. Clarience Technologies, LLC will provide in-kind support regarding staff support time, fleet data collection, camera testing on the tractor-trailer combinations owned by Tuck-Lite, access to the Road Ready platform, etc. (see support letter with cost-sharing commitment).
Safety Emissions Solutions, LLC – Safety Emissions Solutions is a confirmed deployment partner who will assist the project team in analyzing the collected video and vehicle sensor data and contribute knowledge in estimating safety zones around moving tractors and trailers. Based in Valencia, PA, it is a small business that provides web-based inspection record management and reporting software for inspection stations (ranging from small garages to large dealerships). This web-based system records all PA safety inspection criteria, regardless of pass or fail status, and data for some fields are comprehensive, including information such as all actual tire tread measurements (in units of 1/32 of an inch) at the time of inspection, all maintenance work done to meet state inspection program requirements, final pass/fail status, etc. Safety Emissions Solutions has collaborated with the project team for over two years on safety inspection and fleet management studies of commercial trucks and trailers. The project team knows its data architecture well and has analyzed millions of state-mandated vehicle safety inspection records from Safety Emissions Solutions. This partner is helping the project team build relationships with other interested parties such as PennDOT, various state senators and representatives, the Pennsylvania Automotive Association (PAA), and CITA (a global alliance of companies in the safety and emission inspection industry). Safety Emission Solutions will provide in-kind support through all of their historical data related to truck and trailer inspections and substantial staff time supporting our work with their data (see support letter with cost-sharing commitment).
Trade Institute of Pittsburgh (TIP) – The Trade Institute of Pittsburgh (TIP) is a confirmed equity partner that will assist the project team in exploring career opportunities for marginalized groups in heavy-duty equipment inspection and maintenance planning, focusing on understanding vehicle smart vision systems. TIP, founded in 2013, is a non-profit and vocational training provider dedicated to providing opportunities for people with barriers to employment and others who need additional support to begin their careers. It is located in the Homewood community of Pittsburgh, Pennsylvania. It has engaged and empowered men and women throughout Allegheny County to meet a growing need for skilled tradespeople. The project team plans to develop new programs to train construction equipment inspectors or truck fleet managers who work on infrastructure and construction projects.
PA Department of Transportation (PennDOT) – (in the process of developing a relationship) We have negotiated a data usage agreement with PennDOT that allows us to purchase a complete list of all registered vehicles (including commercial trucks and trailers) currently in the state as of the time of the request. The information available includes the vehicle identification number, zip code, county, type of vehicle, etc. We will also work with them and the Pennsylvania State Police to identify interested parties for our system’s pilot “fleet safety dashboard” (ongoing at the time of proposal submission).
CITA - (in the process of developing a relationship) CITA is the international association of public and private sector organizations actively involved in mandatory road vehicle compliance. While much of their members’ activities are focused on passenger vehicle programs, a significant cross-section of members whose operations span commercial vehicles are interested in the results of this work.
The Lindy Group, Inc. – (in the process of developing a relationship) We are approaching Lindy Group to find the best alignment between the team’s vehicle safety studies and the work of Lindy Paving’s construction equipment and heavy-duty truck tracking and management team for potential deployment. Lindy Paving is a subsidiary company of the Lindy Group, which operates asphalt and concrete plants in Western Pennsylvania. Lindy Paving’s equipment team owns a heavy-duty truck fleet (around 250 trucks) that delivers asphalt and concrete to clients and job sites. They have a team of experienced equipment managers and professionals who maintain the mobility of this vehicle fleet with full consideration of vehicle conditions, client requirements, and contextual factors that influence safety, operational efficiency, and client satisfaction. We are considering pilot studies that can better address three critical aspects of vehicle fleet management practices: 1) standardization of the vehicle data collection, processing, and analytics platforms for proactive safety of heavy-duty vehicles; 2) collaboration with telematics system manufacturers in enabling remote inspection, diagnostics, and vehicle settings; and 3) extensive comparison and analysis of various predictive safety alarming systems for identifying critical gaps where new AI techniques can provide stronger support for inspectors and managers. In addition, we will leverage Lindy Paving’s help to build relationships with other interested parties, such as the Association of Equipment Management Professionals (AEMP).
Expected Outcomes/Impacts
1) New patents of smart sensors and vision systems integrated with lighting and braking systems can significantly reduce the losses of separations between heavy-duty tractor-trailer combinations and other vehicles and people in their dynamic safety zones.
2) Improved heavy-duty commercial trucks in western Pennsylvania by engaging three industry partners and multiple (another 3 partners) motor carriers to assess and implement the proposed smart sensors and camera systems.
3) The sensor and video dataset collected by the Truck-Lite fleet in various environments can serve as a critical benchmark data set for calibrating motion sensors and cameras suitable for reliably assessing side and rear collision risks of the tractors and trailers.
4) Leveraging knowledge about the varying safety zones of various tractors-trailer combinations to guide the improvement of driver assistance technologies and safety information communication systems for coordinating the behaviors of multiple vehicles.
5) Best practices and policy suggestions about regulating vision and smart sensors on heavy-duty truck-trailer combinations and vehicle safety program updates.
6) Training programs (focusing on marginalized groups interested in equipment safety inspection jobs) and industry professionals capable of inspecting and repairing heavy-duty truck-trailer combinations equipped with smart sensors and camera systems.
7) Trained Ph.D., master's, and undergraduate students.
Expected Outputs
1) A multi-source computer vision and data analytics algorithm that automatically analyzes videos, vehicle size, and loading and brake data to infer collision risks between tractor-trailer combinations and approaching vehicles and people.
2) A technical plan for integrating the proposed smart sensors and camera systems into existing driver assistance technologies, such as Automatic emergency braking (AEB).
3) Benchmark datasets collected by intelligent sensors and calibrated cameras installed on 5-10 commercial heavy-duty tractor-trailer combinations for other researchers and industry partners who want to test new algorithms that use these data for assessing dynamic safety zones around heavy-duty trucks.
4) Comparative analysis results of 5-10 tractor-trailer combinations show the losses of separations between vehicles when equipped with and without the proposed smart sensor and camera systems.
5) Smart sensor and camera systems deployment results and roadmaps for addressing wide adoption of the proposed smart sensor and camera systems; such a roadmap should specify data security issues and ethical implications.
6) Cost-benefit models that quantify the costs and benefits of using the proposed smart sensor and camera systems in various contexts faced by heavy-duty vehicles, driving environments, and the resource constraints of motor carriers and regulation agencies.
TRID
The project team searched on TRID for studies related to vehicle condition assessment, crash analysis, and technologies for preventing highway accidents and found relevant studies and some research gaps. These studies are from three major categories: 1) vision-based driving assistance system augmented by motion sensor data analytics, 2) damage detection of vehicles, 3) crash analysis with statistics of vehicle conditions, and 4) cost-benefit-waste analysis of safety and tax programs. The gap is the lack of a real-time alarming system that fully considers the vehicle size, motion planning, road conditions, and occlusions to ensure reliable risk updates of the safety zones around the vehicle's side and rear. Such a system is critical for inter-vehicle dynamic safety zone analysis to augment human estimates of the safe following and overtaking distances. Such a system can also overcome risks caused by illusions, slippery roads due to weather conditions, and poor lighting conditions that can bias human estimates and make the reaction too late to stop. The attached search report lists the literature found with a particular focus on the literature on vision-based driving assistance systems augmented by motion sensor data analytics. The proposed study will 1) extend existing vision-based driver assistance technologies for passenger vehicles to heavy-duty tractor-trailer combinations operating in diverse road and natural environments; 2) use a tractor-trailer fleet owned by Truck-Lite in validating the accuracy and effectiveness of the proposed system in estimating inter-vehicle safety zones precisely and reducing the collision risks in various vehicle and environment combinations; 3) extend existing telematics technology cost-benefit analysis to capture deployment challenges of the proposed smart sensor and camera systems for tractor-trailer combinations.
Individuals Involved
| Email |
Name |
Affiliation |
Role |
Position |
| hcain@andrew.cmu.edu |
Cain, Heather |
Carnegie Mellon University |
Other |
Staff - Business Manager |
| bethannh@andrew.cmu.edu |
Hockenberry, Beth |
Carnegie Mellon University |
Other |
Staff - Business Manager |
| ynakahir@andrew.cmu.edu |
Nakahira, Yorie |
Carnegie Mellon University |
Co-PI |
Faculty - Untenured, Tenure Track |
| ptang@andrew.cmu.edu |
Tang, Pingbo |
Carnegie Mellon University |
PI |
Faculty - Untenured, Tenure Track |
Budget
Amount of UTC Funds Awarded
$95000.00
Total Project Budget (from all funding sources)
$210000.00
Documents
Match Sources
No match sources!
Partners
| Name |
Type |
| The Trade Institute of Pittsburgh (TIP) |
Equity Partner Equity Partner |
| Clarience Technologies, LLC |
Deployment Partner Deployment Partner |
| Safety Emissions Solutions, LLC |
Deployment Partner Deployment Partner |
