Principal Investigator

Christoph Mertz

Status

Completed

Start Date

Jan. 1, 2016

End Date

Dec. 31, 2016

Project Type

Research Advanced

Grant Program

MAP-21 TSET National (2013 - 2018)

Grant Cycle

2016 TSET UTC

Visibility

Public

Abstract

Vehicular accidents are one of the great societal challenges. In some demographics, they are the leading cause of death. It is important that accidents are thoroughly investigated. In this project we use Structure-from-Motion (SfM) techniques to create 3D models of the whole accident and of relevant parts. The investigator only needs to use a standard camera to take pictures of the scene from many locations and perspectives. Given enough images, the SfM software computes a 3D model. We will develop tools to integrate the models into the investigative procedures, e.g. by comparing the model of the damaged car with an intact model to determine the crush. We will also collaborate with Texas A+M Transportation Institute (TTI) to find additional use cases and make it a useful tool in collision research.    

Description

Introduction

Vehicular accidents are one of the great societal challenges. In some demographics, they are the leading cause of death. It is important that accidents are thoroughly investigated. The immediate reason is to find out who is responsible and liable for the damages. But just as important is it to collect information about accidents to determine if changes to vehicles, infrastructure or policy could prevent or mitigate future accidents. One set of tools in the investigation use 3D models of the scene. They are usually acquired with laser scanners. However, they are very costly in time and money and are therefore only used in severe cases, like fatal accidents. In recent years digital cameras and computer vision algorithms have become so inexpensive, powerful and efficient that it is now possible to create 3D models from a set of digital images at a very low cost. In the past year we have shown that one can do this for vehicle accidents and that these 3D models are useful for accident investigation. In the proposed project we want to further integrate it into the work flow of the Pittsburgh accident investigators, improve the accuracy of the 3D reconstruction, and collaborate with Texas A+M Transportation Institute (TTI). The collaboration with TTI will be threefold. First of all it is for us to be able to observe staged crashes and learn from the experts on how crashes are investigated. Secondly we want to introduce our techniques to them. And the third goal is to jointly develop new use cases.      
Current project

In the recent past the Navlab group has developed methods to make 3D reconstructions of accident scenes from images (Figure 1). We are using structure-from-motion (SfM) software1. Images are taken all around the scene from different angles and different heights. The software simultaneously estimates the position of the cameras and the 3D model of the scene (Figure 1 top right). The model can be color mapped to achieve a colorized 3D model (Figure 1 bottom right). The method is applicable to many other objects (Figure 2), from something as small as a penny to something as large as a building. It works indoors as well as outdoors. When we described this technology to the Pittsburgh Police, they pointed out that they are particularly interested to determine the crush of a vehicle, i.e. how much the impact deformed the vehicle. From the crush one can estimate the velocity of the vehicle. The estimation is illustrated with an example in Figure 3. We construct the 3D model of the damaged car as described above and download a 3D model of an intact car from the internet. We align and compare those two models. A horizontal cross-section is taken and at 6 locations the distances C1-6 from the intact model to the damaged model are measured. A standard formula calculates the impact velocity from C1-6. We anticipate that by the end of the current project (end of 2015) we will have determined the crush of one vehicle from an actual accident and compared it with the current method used by Pittsburgh accident investigators.
Proposed Project

The SfM algorithms assume that the objects have Lambertian surfaces, i.e. that there are no specular reflections. However, there are often specular reflections off the vehicles which can cause some errors. We will test various methods to reduce or even eliminate these errors. These methods are: Polarization filters to suppress specular reflections, spraying the surface with a dull paint, or putting markers on significant locations. The markers will also aid in aligning the damaged and intact models and measuring the distances marked as C1-6 in Figure 3. In a preliminary test we showed that it is possible to suppress specular reflections using polarization filters. Figure 4 shows how the reflection of a bright light off a table can be suppressed. However, one needs to realize that complete suppression works only at one angle (the Brewster angle), at other angles one gets only partial suppression. We will investigate how to best take images and orient the filter to minimize the specular reflections. Pittsburgh police has standard accident investigation software (ARAS 360) which can read in 3D point clouds. We want to further develop the work flow from taking the images at the optimal locations, efficiently creating a 3D model, and converting it to a format best suited for analysis in accident investigation software.  

TTI has a test range where they can stage vehicles accidents, e.g. vehicles crashing into barriers to test barrier design. TTI invited us to observe these experiments and test our 3D reconstruction on the various relevant parts (the vehicle, the barrier, etc.). We can compare our 3D model with the model they get with a laser scanner. We will also learn what the important parts are that need to be observed, the desired resolution, and many other things pertinent to accident investigation and safety research. During this time we will introduce our technology to TTI to find out if it is a useful tool for their research. Together we also want to come up with new use cases. For some things it is very difficult to make a 3D scan with a laser scanner. Examples are the interior of the vehicle or parts like the suspension that are inside the chassis. These are examples of new use cases for our system. As long as one can reach a part with a camera, it should be possible to create a 3D model. Together with TTI we will select the most important and promising use cases and test our system on these.    Beyond Pittsburgh Police and TTI we will reach out to the greater accident investigation community by presenting our technology at the annual PA reconstruction seminar and by making out techniques available online2. Furthermore, Greg Sullenberger (accredited accident reconstructionist, pcarspresident@gmail.com) is the coordinator of WREX2016 and he personally invited Dr. Mertz to use and demonstrate our technology, and possibly participate as an invited speaker. This event has already registered over 300 participants from more than 10 countries and expects to host over 500 participants.
 
Project Tasks

1. Improvement
a. Polarization filter to suppress specular reflections. b. Paint specular surfaces or place  markers at significant locations
2. Integration
a. Automatic matching of significant locations b. Develop and improve the full pipeline from taking images to using the result in the investigation software.
3. Collaboration with TTI
a. Visit 1: Exchange ideas and come up with new use case(s). 2: Test new use case(s). b. Observe stage crashes, learn investigation methods c. Introduce our technology d. Develop and test new use case(s)
4. Outreach
a. Visit the reconstruction seminar and WREX2016 b. Publish our techniques and results on a web site2

1 http://ccwu.me/vsfm/                                                
2 http://www.cs.cmu.edu/afs/cs.cmu.edu/project/reconstruction/www/index.html, website will go live within a few
days.

Timeline

1. Improvement
a) Polarization filter (August - October)
b) Paint, markers (February - March)
2. Integration 
a) Matching
b) Full pipeline (March - April, October - December)
3. TTI
a) Visit (January, July)
b) Staged Crashes (January)
c) Introduce System (January - February)
d) New Use Case (January - February, May - September)
4. Outreach
a) Seminar, expo (May, September)
b) Website (April, August, November - December)

Strategic Description / RD&T

Deployment Plan

We are already testing our system with the Pittsburgh Police (see letter of support). We anticipate that we can recruit a few more partners during the reconstruction seminar or the expo. Ideally they would be able to use our system in their routine work to measure crush by the end of the project. The new use case will be ready for pilot testing at the end of the project. The Pennsylvania Turnpike Commission (PTC) indicated in summer 2014 that they are interested in accident reconstruction from images. As soon as the IGA between PTC and CMU is signed we will connect with PTC again to discuss a possible project. A project with PTC would provide matching funds for this proposal.  

Expected Outcomes/Impacts

We expect that by the end of the project we have the full pipeline develop from taking images to calculating the crush and have the instructions documented on the website. Metric for success: System is actually used by investigators. Metric comparing with previous method: Savings in time or cost over previous method, increase in accuracy We expect to apply out techniques to at least one new use case. Metric for success: TTI adopts our system for the new use case.

Expected Outputs

TRID

Individuals Involved

Email	Name	Affiliation	Role	Position
cmertz@andrew.cmu.edu	Mertz, Christoph	Robotics Institute	PI	Faculty - Research/Systems

Budget

Amount of UTC Funds Awarded

$89742.00

Total Project Budget (from all funding sources)

$89742.00

Documents

Type	Name	Uploaded
Final Report	41_-_Accident_Investigation_with_3D_Models_from_Image_final_report.pdf	Aug. 15, 2018, 4:45 a.m.

Match Sources

No match sources!

Partners

No partners!