Principal Investigator

H. Scott Matthews

Status

Completed

Start Date

Jan. 1, 2015

End Date

Dec. 31, 2015

Project Type

Research Advanced

Grant Program

MAP-21 TSET - Tier 1 (2012 - 2016)

Grant Cycle

2015 TSET UTC

Visibility

Public

Abstract

States require inspections on vehicle safety components to be performed with varying frequencies and on various subsets of the fleet.  In Pennsylvania, every passenger vehicle is inspected annually. Stakeholders have called for modifications or elimination of safety inspection programs.  However, inspection data have not been available, so efforts to improve programs have been challenging.    

To date (funded by PITA/NSF/UTC) we haveanalyzed millions of Pennsylvania vehicle inspectionsfrom the past 8 years viacomprehensive datasets pertaining to safety inspections and vehicle registrations. One of these datasets is from a collaborating partner, Compuspections, a private PA company focused on IT solutions for inspection data management.  While the conventional wisdom is that the failure rate is about 2%, our findings (recently submitted for publication(Peck 2014)) suggest that the actualrate is about 10 times higher.  We have studied thisfailure rate across many dimensions:  time, age of vehicles, mileage driven, urban-rural, etc. We identifiedthe major causes for inspection failure:brakes, tires, and lights, which are routine maintenance activities.In this next phase, we will continue to collaboratewith Compuspectionsto receive additional data andfurther,to consider the cost-effectiveness of changes to the state inspection regime tomaintain metrics of safety at socially acceptable levels.  For example, we will consider the inspection cost tradeoffs in expected fatalities if the program were relaxed to exempt new or lightly driven cars from annual inspections.  To accomplish this we will combine our analysis of failures with federal accident databases (e.g., FARS) to estimate expected changes in annual fatalities caused by various underlying failures subject to inspection (i.e., accidents caused by brake or tire failures).Additionally, we will evaluate and compare current states with annual vehicle safety inspections versus those without by comparing fatal crashes.Our goal is an unbiased study that could help inform stakeholders and the policy process in Harrisburg.

Note: Due to the sensitivity and potential political impact of our findings, we have chosen to not pursue funding or official collaboration fromPennDOT (aside from permission to receive data) so as to produce an independent study. Compuspections has only provided data for this same reason.    

Description

Various parties within Pennsylvania (and other states) have called for significant modifications and/or elimination of their passenger vehicle safety inspection programs.  Such programs call for specified safety inspection tests to be performed on automobiles and light trucks with varying frequencies (e.g., annually) and on various subsets of the fleet (e.g., exempting new cars).

The main motivation for modifications to inspection programs is a perception that they are very costly to consumers, and provide little or no benefits to society or to parties other than automobile repair business owners, who earn revenue from fixing vehicles that fail tests. Behind these perceptions are sentiments that "cars have never been safer", yet very little data exists so very few analytical studies have been able to prove this.  The handful of past studies completed have been high level analyses of whether states with safety inspection programs have higher crash or fatality rates which provide at best indirect measures of effectiveness. 

Pennsylvania (PA) is fairly unique in policy discussions.  PA requires annual safety inspections for all passenger vehicles.  Total costs from safety inspections and repairs in Pennsylvania are more than $500 million per year(this of course is the revenue for inspection stations). 

Five years ago, PennDOT launched the “E-SAFETY” program, where stations voluntarily report detailed results of vehicle inspections. Given alack of incentives to report, only about 1% of all inspections are reported.  Beyond the small data sample, a critical problem exists in how inspection results are captured and reported in the data archive –that is, only the “driving away” status is recorded.  Vehicle systems may have been flagged for repair by the inspection, and repairs made, which leads the vehicle to pass.  But such instances are really “vehicles that would have failed”.  The E-SAFETY dataset suggests only 2-3% of cars fail [1], and this trivial failure rate has been trumpeted by PA legislators trying to cut inefficient programs.  However this rate only means that 2-3% of vehicles leave the inspection station in a failed state.  Lost amongst the data are those various intermediate repairs or adjustments, which certainly could (should) have been classified as failures under the current inspection regime.  There is also a need to set baselines and consider differences in cost and fatality outcomes if the program were modified.  Thus, this data does not help inform the policy debate given its weaknesses.

We have partnered with several entities seeking to promote the “right data” needed to assess the cost-effectiveness of the current PA safety inspection program, and to suggest cost-effective changes to the frequency or fleet required to be inspected.  Data used to date has come from:

PA Department of Transportation(PennDOT) –We have negotiated a contract with PennDOT that allows us to receive ongoing data on the following state records in digital form:  
?Complete list of allregistered vehicles currently in the state as of time of request, including vehicle identification number, zip code, county, type of vehicle, etc.;
?Complete report of allinformation from the E-SAFETY database, including vehicle identification number, zip code, cost of inspection and repairs, pass/fail status, etc.

Compuspections, LLC–We have been working with the CEO and Director of Compuspections LLC for 18months.  Compuspections is a small PA business that provides inspection record management and reporting software for inspection stations (ranging from small garages to large dealerships). Records are extremely comprehensive, including information such as all four actual tire tread measurements (in units of 1/32 of an inch) at time of inspection, all maintenance work done to meet state inspection program requirements, labor and material costs, final pass/fail status, etc.  In short, this data can fill the gap identified above in terms of clearly noting what happens in the workflow of a safety inspection from time of entry to point of exit from the station and not simply whether the vehicle passes when leaving.  Compuspections will continue to provide us with a large amount of in-kind data (see letter) and expertise.  Past efforts to justify or modify the inspection program by the various stakeholders involved have been limited since their motivations have been questioned (e.g., by stations to maintain the current system and associated revenues).  A primary goal of this effort is to have a study with multiple data sources but authored by researchers from CMU, a neutral third party. Our partnership with Compuspections has already provided us with 10 million inspection records over 10 years and helped us to build relationships with other interested parties such as PennDOT, various state legislators, AAA, Pennsylvania Automotive Association (PAA), etc.  

In this project to date (partly funded by PITA, NSF, and UTC), we have efficiently organized the Compuspections archive of inspection records.  In support of this effort, we have developed data mining and visualization methods (via Python, and R) to operate on the data archive to identify trends in inspection failure rates across make-model-year of vehicle, age, urban-rural location, and miles driven. Supplemental Figure 1 shows an example of our analysis on both sets of inspection data that demonstrates relatively low failure rates (2-3%) for nearly new vehicles, but a much higher failure rate (20%, an order of magnitude higher) as those vehicles are more than 6 years old.The results also show number of failures identified within specific subcategories of inspection tests (e.g., ineffective tires and brakes). This work has been recently submitted for publication in Transportation ResearchPart A.

Going forward, we propose to continue to collaboratively work with Compuspections to receive additional data so as to consider the cost-effectiveness of changes to the state inspection regime in order to maintain overall safety at socially acceptable levels.  We will assess cost savings from reduced inspection requirements (in terms of actual costs and value of owner's time) against increases in vehicle-related fatalities from more unsafe vehicles.  Proposed changes to the inspection program (e.g., biennial inspections, exempting cars driven less than 5,000 miles per year, or no longer inspecting tires) could then be assessed by estimating additional numbers of vehicles on the road that would be unsafe due to being inspection exempt, and which would potentially lead to crashes and fatalities.    Given our robust dataset, we can estimate these effects by model year on the road as a result of various scenarios of relaxed inspection program rules(see Figure 2). 

We will use data fromDOT’s Fatality Analysis Reporting System (FARS), which notes whether vehicle safety components (like brakes or tires and many others) were the primary or contributing factors in all fatal vehicle accidents in the US. We will create risk-based measures of expected fatalities as a function of failures in safety components(i.e., accidents caused by brake or tire failures per million vehicle miles driven).We will also need to evaluate and compare states with annual vehicle safety inspections versus those without by comparing fatal crashes.  We will also incorporate the odometer data to consider risk measures on a per-mile driven basis for the PA vehicle fleet. By combining this fatality data with our failure analysis, we will be able to assess the differences in expected costs of inspection and fatalities from alternative inspection program policy designs.We will estimate the net saving sand cost-effectiveness (cost savings per fatality avoided or incurred)to better inform policy discussions.Our goal is an unbiased study that could help inform stakeholders and the policy process in Harrisburg.

References1.Cambridge Systematics, “Pennsylvania'sVehicle Safety Inspection Program Effectiveness Study.” 2.Peck, Dana, H. Scott Matthews, Chris Hendrickson, and Paul Fischbeck,“An Analysis of Vehicle Safety Inspection Data in Pennsylvania: Expected Failure Rates.” Transportation Research Part A: Policy and Practice. Submitted, 2014.

Timeline

Jan 1, 2015 -Dec 31,2015

Strategic Description / RD&T

Deployment Plan

We expect intermediate results of this expanded research scope by summer 2015.  We anticipate publication of the initial failure rate paper in late 2014, and testimony and participation in Harrisburg in early 2015.  We hope to have a peer-reviewed paper on the cost-effectiveness analysis by late 2015 to support testimony in late 2015.

Expected Outcomes/Impacts

As noted above, we expect the result of this work to be the further development of an analytics-driven engine to help present the right information to stakeholders in the safety inspection domain.  Beyond the information systems work, we will be creating a white paper to be used by stakeholders to inform the state policy process as it pertains to changing the safety program.We will seek to provide testimony to the transportation committee of the PA legislative bodies, as they again seem ready to take action on reducing the scope of the inspection program.

Given the safety / information technology focus of TSET, we expect our project to be aligned well and to be competitive for follow on funding (and we expect to only need about one year of support for completion of the work at that point). 

We expect the economic and social benefits within Pennsylvania to be substantial, and expect commensurate spillover benefits to other states considering changes to their programs by demonstrating a best-practices solution for inspection data management. Where suitable, we will also provide input or testimony to policy makers in these other states to support their discussions.

Expected Outputs

TRID

Individuals Involved

Email	Name	Affiliation	Role	Position
pf12@andrew.cmu.edu	Fischbeck, Paul	EPP/SDS	Co-PI	Faculty - Research/Systems
cth@cmu.edu	Hendrickson, Chris	CIT/Heinz	Co-PI	Faculty - Tenured
hsm@cmu.edu	Matthews, H. Scott	CEE/EPP	PI	Faculty - Research/Systems

Budget

Amount of UTC Funds Awarded

$59558.00

Total Project Budget (from all funding sources)

$59558.00

Documents

Type	Name	Uploaded
Final Report	TSET_FY15_Project_99_Final_Report.pdf	June 28, 2018, 4:24 a.m.

Match Sources

No match sources!

Partners

No partners!