Principal Investigator

H. Scott Matthews

Status

Completed

Start Date

Jan. 1, 2016

End Date

July 31, 2016

Project Type

Research Advanced

Grant Program

MAP-21 TSET National (2013 - 2018)

Grant Cycle

2016 TSET UTC

Visibility

Public

Abstract

States require inspections on vehicle safety components to be performed with varying frequencies and on various subsets of the fleet. Stakeholders in numerous states have called for modifications or elimination of safety inspection programs.  However, inspection data have not been available, so efforts to improve programs have been challenging and many states have eliminated their programs as a result of the lack of
analysis.

To date (funded by PITA/NSF/UTC) we have analyzed millions of Pennsylvania comprehensive vehicle safety and registration data records. While the commonly reported failure rate is about 2%, our findings suggest the actual rate is about 10 times higher [1]. We also compared fatalities per billion VMT, nationally, between states with and without vehicle safety inspection programs. Findings showed safety inspections were statistically significant in reducing the fatality rate in states on average by 1.8 fatalities per billion VMT. Furthermore, a highly significant urbanity coefficient of about 11 fatalities per billion VMT was found showing statistically higher rural fatality rates (compared to urban) [2]. It was concluded that urbanity must be accounted for in these analyses, and led to additional questions of whether there are other parameters that affect the fatality rates in states, such as vehicle makes or vehicle ages.   

In this next phase, we will continue the state comparisons yet expand them from looking only at fatality rates to also identifying differences in vehicle fleets. This will be accomplished by analyzing national vehicle registration data in order to separately consider whether the benefits of safety inspection programs are the cause or result of any vehicle fleet differences (e.g., fleet age differences and make differences). Our goal is an unbiased study that could help inform stakeholders and the policy process on vehicle safety in both Pennsylvania and the U.S. on whether the vehicle safety inspection program is
worth implementing and how stringent it should be.

Note: Due to the sensitivity and potential political impact of our findings, we have chosen to not pursue funding or official collaboration from PennDOT (aside from permission to receive data) so as to produce
an independent study.  CompuSpections has only provided data for this same reason.    

Description

In recent years, states have been questioning the effectiveness of vehicle safety inspections. The common sentiment of many states that there is lack of data providing evidence that vehicle safety inspections aren’t worth the time and money; yet, it cannot be inferred safety inspections do not reduce fatalities. More analyses are necessary before this statement can be confirmed and conclusions drawn. These questions may help lead toward a helpful conclusion on this topic of vehicle safety inspection program benefits (or lack thereof). In previous research, we have found initial results in support of the Pennsylvania inspection program; however, the results also confirmed the significant differences in fatality rates between urban and rural locations. This caused further questions of whether differences in vehicle fleets between states exist and whether these differences were the cause of safety states having lower fatality rates (or the result from those states having younger, “safer” vehicles initially). 

Personal vehicle safety inspection programs today vary widely across their execution and oversight, making the program challenging to analyze and identify any benefits or disadvantages. One task in this research aims to classify current safety inspection programs in each state by their frequencies and rigorousness to find if there is any advantage in supporting these programs. In parallel, the quality of data used in this study is evaluated. Data used thus far includes: 

PA Department of Transportation (PennDOT) – We have negotiated a contract with PennDOT that allows us to receive ongoing data on the following state records in digital form:  
? Complete list of all registered vehicles currently in the state as of time of request, including
vehicle identification number, zip code, county, type of vehicle, etc.;
? Complete report of all information from the E-SAFETY database, including vehicle identification number, zip code, cost of inspection and repairs, pass/fail status, etc.

CompuSpections, LLC – We have been working with the CEO and Director of CompuSpections LLC for almost 2.5 years.  CompuSpections is a small PA business that provides inspection record management and reporting software for inspection stations (ranging from small garages to large dealerships). Records are extremely comprehensive, including information such as all four actual tire tread measurements (in units of 1/32 of an inch) at time of inspection, all maintenance work done to meet state inspection program requirements, labor and material costs, final pass/fail status, etc.  In short, this data can fill the gap identified above in terms of clearly noting what happens in the workflow of a safety inspection from time of entry to point of exit from the station and not simply whether the vehicle passes when leaving.  CompuSpections will continue to provide us with a large amount of in-kind data (see letter) and expertise. 

Past efforts to justify or modify the inspection program by the various stakeholders involved have been limited since their motivations have been questioned (e.g., by stations to maintain the current system and associated revenues).  A primary goal of this effort is to have a study with multiple data sources but authored by researchers from CMU, a neutral third party. Our partnership with CompuSpections has already provided us with 10 million inspection records over 10 years and helped us to build relationships with other interested parties such as PennDOT, various state legislators, AAA, Pennsylvania Automotive Association (PAA), etc.
In this project to date (partly funded by PITA, NSF, and UTC), we have efficiently organized the CompuSpections archive of inspection records.  

In support of this effort, we have developed data mining and visualization methods (via Python, and R) to operate on the data archive to identify trends in inspection failure rates across make-model-year of vehicle, age, urban-rural location, and miles driven. Supplemental Figure 1 shows an example of our analysis on both sets of inspection data that demonstrates relatively low failure rates (2-3%) for nearly new vehicles, but a much higher failure rate (20%, an order of magnitude higher) as those vehicles are more than 6 years old. The results also show number of failures identified within specific subcategories of inspection tests (e.g., ineffective tires and brakes).  This work was accepted for publication in Transportation Research Part A: Policy and Practice [1].Going forward, we propose to continue to collaboratively work with CompuSpections to receive additional data so as to consider the cost-effectiveness of changes to the state inspection regime in order to maintain overall safety at socially acceptable levels.  We will assess cost of inspections across locations in the state as well as by vehicle models and ages. There is also potential, with the current data provided, to monitor and assess the inspection pass/fail rates by inspector and inspection station in order to track performance and reliability of results for the state of PA. 

We have also expanded our research from Pennsylvania to a nationwide analysis by using data from DOT’s Fatality Analysis Reporting System (FARS), to evaluate and compare states with stringent, annual vehicle safety inspections versus those with less stringent programs by comparing fatal crash rates (fatal crashes per billion VMT). While this analysis was limited to fatal crashes and limited knowledge was available on the intensity of each state’s inspection program, regression models were found to strongly support the vehicle safety inspection program. The results showed that safety inspections were statistically significant in reducing the fatality rate in states on average by 1.8 fatalities per billion VMT and also resulted in a highly significant urbanity coefficient of about 11 fatalities per billion VMT difference in urban versus rural locations (higher rural fatality rate, refer to Supplemental Figure 2). This led to the conclusion that urbanity must be accounted for in these analyses, as well as additional questions of whether there are other parameters that affect the fatality rates in different states, such as vehicle makes or vehicle ages. This work has been submitted to be considered for poster and/or publication for the Transportation Research Board’s Annual Meeting [2]. 

There is no national standard for safety inspections; therefore, even states that do have programs may not execute them the same way. This led to limitations in accurately analyzing the program and resulted in numerous regression models to check any possible scenario. Lack of consistency of the crash attributions across crash data reported by each state and the difficulty in identifying the best way to assign a vehicle to a state (either registration or crash state) was a downfall to this section of the analysis. Improvements in data reporting, collection, and program oversight for both data collection and safety inspection programs are integral in further reducing vehicle crashes and therefore assessments of differences in fatality rates. 

In addition to FARS data and as a result of the previous conclusions we were able to make in our most recent fatality rate analysis, we recently purchased 2015 national vehicle registration data from Hedges and Company. This data allows for additional comparisons between state vehicle fleets, supplementing the current findings in strong urbanity fatality rate differences. From initial work with the 2015 registration data, questions arise as to whether vehicle age differs between states with more stringent safety inspection programs versus those with less stringent programs. Initial results resulted in states with more stringent programs having younger fleets on average. Detailed statistical analyses are necessary in order to determine whether the younger fleet is a result of these states having a more stringent safety inspection program or whether the younger fleet causes the significant results in lower fatality rates in those states with the more stringent safety inspection programs. 

If vehicle age and vehicle make can be separated from the fatality rate results, then the true underlying effects of the safety inspection program can be found. Additionally, if the safety inspection program does lead to a younger vehicle fleet, there are numerous advantages (economic growth, reduced emissions, change in travel patterns) of newer vehicles that can also be examine as a result of the safety inspection program in addition to the safety benefits. The overall goal of this research is to provide an unbiased study on the effects of the safety inspection program that could help inform policy makers in each state and possibly even the U.S. government, depending on the findings.

References
[1] D. Peck, H. S. Matthews, P. Fischbeck, and C. T. Hendrickson, “Failure rates and data driven policies for vehicle safety inspections in Pennsylvania ,” Transportation Research Part A, vol. 78, no. C, pp. 252–265,
Aug. 2015.
[2] D. Peck, H. S. Matthews, P. Fischbeck, and C. T. Hendrickson, “The Effect Of Vehicle Safety Inspections
On Urban/Rural Fatality Rates,” Transportation Research Record.

Timeline

January 1, 2016 – July 31, 2016

Strategic Description / RD&T

Deployment Plan

We plan to submit another paper by mid-2016 outlining the overall benefits of the safety inspection program and any additional benefits resulting from the inspection program. There is also a lot of interest from policy makers in PA and other states on the results of this research. GAO, who was very interested in our work, approached us and asked us to outline our initial findings from the PA study (which they then used in their report). We anticipate further interest from states and policy makers regarding this final research stage and hope to have a peer-reviewed paper in late-2016 that can be shared and referenced regarding the benefits of the safety inspection program.

Expected Outcomes/Impacts

As noted above, we expect the result of this work to be the further development of an analytics-driven engine to help present the right information to stakeholders in the safety inspection domain.  Beyond the information systems work, we will be creating a white paper to be used by stakeholders to inform the state policy process as it pertains to changing the safety program.  We will seek to provide testimony to the transportation committee of the PA legislative bodies, as they again seem ready to take action on reducing the scope of the inspection program. We expect the economic and social benefits within Pennsylvania (and potentially the U.S.) to be substantial, and expect commensurate spillover benefits to other states considering changes to their programs by demonstrating a best-practices solution for inspection data management.  Where suitable,

Expected Outputs

TRID

Individuals Involved

Email	Name	Affiliation	Role	Position
pf12@andrew.cmu.edu	Fischbeck, Paul	SDS	Co-PI	Other
cth@cmu.edu	Hendrickson, Chris	CEE/Heinz	Co-PI	Other
hsm@cmu.edu	Matthews, H. Scott	CEE/EPP	PI	Faculty - Research/Systems

Budget

Amount of UTC Funds Awarded

$87203.00

Total Project Budget (from all funding sources)

$87203.00

Documents

Type	Name	Uploaded
Final Report	TSET_FY16_Project_39_Final_Report.pdf	June 28, 2018, 4:23 a.m.
Publication	Net-societal and net-private benefits of some existing vehicle crash avoidance technologies.	Nov. 27, 2020, 7:51 p.m.

Match Sources

No match sources!

Partners

No partners!