#192 Low-Cost 3D model acquisition for rapid accident investigation

Principal Investigator
Christoph Mertz
Start Date
Aug. 1, 2017
End Date
June 30, 2019
Research Type
Grant Type
Grant Program
FAST Act - Mobility National (2016 - 2022)
Grant Cycle
2017 Mobility21 UTC


Vehicular accidents are one of the great societal challenges. Acquiring 3D models from actual or staged crashes is an important investigative tool. Currently this is done with laser scanners. They can provide excellent 3D data, but the devices are expensive, need to be used by experts and the data collection is time consuming. For these reasons they are only used for the most severe accidents. We want to build on our work with Structure-from-Motion (SfM) techniques to create 3D models of the whole accident and of relevant parts. We have already developed the core methods where the investigator only needs to use a standard camera to take pictures of the scene from many locations and perspectives. Given enough images, the SfM software computes a 3D model. In this project we want to refine the techniques, develop more tools and bring the method closer to market. In particular we want to develop a tool that compares the intact 3D model and the damaged 3D model and determines the deformation of the latter with minimal user input. Lastly we want to test our methods in the field, to investigate real accidents and as a tool for in collision research.    

Vehicular accidents are one of the great societal challenges. In some demographics, they are the leading cause of death. It is important that accidents are thoroughly investigated. The immediate reason is to find out who is responsible and liable for the damages. But just as important is it to collect information about accidents to determine if changes to vehicles, infrastructure or policy could prevent or mitigate future accidents. One set of tools in the investigation use 3D models of the scene. They are usually acquired with laser scanners. However, they are very costly in time and money and are therefore only used in severe cases, like fatal accidents. In recent years digital cameras and computer vision algorithms have become so inexpensive, powerful and efficient that it is now possible to create 3D models from a set of digital images at a very low cost. In the past year we have shown that one can do this for vehicle accidents and that these 3D models are useful for accident investigation. In the proposed project we want to refine the techniques, develop more tools and bring the method closer to market.

Recent accomplishments

In the recent past the Navlab group has developed methods to make 3D reconstructions of accident scenes from images (Figure 1). We are using structure-from-motion (SfM) software (http://ccwu.me/vsfm/). Images are taken all around the scene from different angles and different heights. The software simultaneously estimates the position of the cameras and the 3D model of the scene (Figure 1 top right). The model can be color mapped to achieve a colorized 3D model (Figure 1 bottom right). 
More recently we have used the 3D model to determine the crush and thereby determine the speed at impact (Figure 2). We are also able to model hard-to-reach areas like the wheel and suspension (Figure 3). Furthermore, we are now able to make a joint model of the inside and outside of the vehicle (Figure 4). 

Proposed Project

Our current method of measuring crush involves steps that require user input: Alignment of damaged and undamaged model, creating a horizontal slice, and measuring the six distances (see Figure 2). We want to develop methods that will require much less user input. Our starting point will be open source tools like Point Cloud Library (www.pointclouds.org). They have functions to clean point clouds by removing noise, align and register 3D point clouds, reconstruct surfaces and fit models. Ideally the fitted models would be deformable models, where the original model and a deformation function are simultaneously fitted to the crushed model. We are also interested in measuring the location and damage to hard-to-reach parts inside the vehicles. For that we will try to segment the 3D model into smaller parts and fit primitives to them.  
Finally we want to make the technology easy to use for non-experts. We already have a web page (http://www.cs.cmu.edu/~reconstruction/) where interested users can download software and read instructions on how to do 3D reconstruction. We will keep it updated with the new tools we are developing.      
Fall 2017: Selection of master student, initial research
Spring 2018: develop initial analysis methods
Summer 2018: Collection of additional data, testing of methods.
Fall 2018: Refinement of analysis methods
Spring 2019: Final testing, final report/master thesis     
Deployment Plan
We will contact agencies involved in vehicle crash reconstruction and investigation and present our work to them.    
Expected Accomplishments and Metrics
The accident analysis method that will be developed should be able to make a 3D model from images of a crashed vehicle and detect various damaged and undamaged parts. The metrics are detection rate (ROC or precision/recall) of parts and accuracy of model fit (standard deviation of distance between 3D points of real object to model object).      

Individuals Involved

Email Name Affiliation Role Position
cmertz@andrew.cmu.edu Mertz, Christoph Carnegie Mellon University PI Other


Amount of UTC Funds Awarded
Total Project Budget (from all funding sources)


Type Name Uploaded
Data Management Plan DMP_accident_reconstruction_mertz_v1.pdf March 22, 2018, 7:50 a.m.
Publication 1808.00671.pdf Sept. 25, 2018, 3:35 p.m.
Progress Report 192_Progress_Report_2018-09-30 Sept. 25, 2018, 3:38 p.m.
Publication Yuan_2018.pdf Sept. 27, 2019, 1:05 p.m.
Publication Khot_2019.pdf Sept. 27, 2019, 1:05 p.m.
Presentation Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency Sept. 27, 2019, 1:09 p.m.
Presentation Mobility21_2018_v4.pdf Sept. 27, 2019, 2:05 p.m.
Progress Report 192_Progress_Report_2019-09-30 Sept. 27, 2019, 2:29 p.m.
Final Report 192_-_Final_Report.pdf Jan. 21, 2020, 6:32 a.m.
Progress Report 192_Progress_Report_2020-03-30 March 17, 2020, 9:36 a.m.
Publication Monte Carlo sampling based imminent collision detection algorithm Nov. 27, 2020, 6:19 p.m.
Publication A method of objects classification for intelligent vehicles based on number of projected points.(*Christoph Mertz) Nov. 27, 2020, 6:26 p.m.

Match Sources

No match sources!


Name Type
TBD Deployment Partner Deployment Partner