#327 Bus on the edge: Continuouse monitoring of traffic and infrastructure - year 2

Principal Investigator
Christoph Mertz
Start Date
July 1, 2020
End Date
Aug. 30, 2021
Research Type
Grant Type
Grant Program
FAST Act - Mobility National (2016 - 2022)
Grant Cycle
2020 Mobility21 UTC


One transit buses from Washington County Transportation Authority will be equipped with cameras and an edge computer connected to a cellular network for time critical and WiFi for high bandwidth communication. The computer detects relevant events and saves data of interest in a privacy preserving manner. This system will enable many mobility relevant observations related to traffic, weather, infrastructure, or disruptions, several of them we will implement and test.     
Data is essential to maximize mobility and make the transportation system resilient. Examples are real time schedules, traffic conditions, weather conditions, or infrastructure inspection. In this project we want to develop an efficient data collection platform that can monitor complete routes on an hourly basis. The idea is to make use of transit buses that travel on main roads on a regular basis. They are often already equipped with cameras and an electronics cabinet that houses recording equipment and the automatic annunciation system. We will use such a commercial system, where the recording device is a full-fledged computer with all the hardware components to use it as an edge computer. All the edge computing software modules will run in parallel to the pre-installed recording system. The edge computing system will analyze the video streams live to detect relevant events and sends these detections over a cellular network to a central location. Data that is interesting but not time critical will be saved and uploaded  via WiFi at the end of the day. Potentially relevant data will be saved for a set amount of time locally and analyzed or uploaded upon request. The edge computer system will also handle privacy: Before data is saved any personally identifiable information will be eliminated, e.g. by blurring out faces or license plates. The data collected by such a system can be used for event detection, traffic modeling and infrastructure monitoring and thereby provide input for up-to-date detailed maps of roads and traffic. Up-to-date detailed maps are one essential component of autonomous driving, but they are also needed for traffic management, planning, and infrastructure maintenance. Concrete examples are traffic counts, counting of parked cars, observations of road construction, pothole detection, detecting landslide precursors, measuring snow cover, or observing crossing of wildlife.  

In this project we will develop and implement the core edge computing, basic detection, and privacy modules. We will make the platform and the collected data available for other projects, and other projects will complement our data. Some of the other projects will also supply software modules. We will modify them to run on the edge computing system. The other projects are:

1. Updating HD maps with videos from vehicles 
This is a Argo sponsored MSCV (Master of Science in Computer vision) project where 3 students use video data to update HD maps. The plan is to run their change-detection software on the edge computer

2. Landslide detection and prediction
Ongoing effort by Amit Acharya, CEE, and Christoph Mertz to find precursors and indicators of landslides like cracks, excessive water or debris on the road. These will be used to model, analyze, and predict landslides. The initial effort was a project sponsored by Metro21 "Landslide prediction and analysis from images". The data collected by the edge computer will be used to train and test the detection algorithms.

3.  Improving rush hour traffic flow by computer-vision-based parking detection and regulations
Just finished UTC project by Christoph Mertz and Sean Qian. The software developed under this project to count parked and moving vehicles will be one of the modules run on the edge computer. 

4. Using municipal vehicles as sensor platforms to monitor the health and performance of the traffic control system.
This is a project from Mark McCord and his colleagues at OSU. They use video from transit buses to obtain traffic flow estimates across extensive urban roadway networks. Their data is collected by a similar system to ours. We have already received data from them and used it to test our parked and moving data detection. We will continue to exchange data and algorithms. 
5. Pedestrian and Cyclists Detection for Transit Buses PHASE II
A NSF-SBIR project with Novateur Research Solutions as the prime and CMU as a sub-contractor. Video data collected by this SBIR project will be used to complement the proposed project. The data will be collected in and around Pittsburgh.  

6. Road surface monitoring
The startup company RoadBotics (Christoph Mertz is co-founder and Chief Scientist) has commercialized road surface monitoring. They have a collection of road images from over 180 customers (towns and cities in several states and countries). RoadBotics will share this data, as permitted by the customer. A data sharing agreement between RoadBotics and the Robotics Institute is in the works.  

The aforementioned projects 1.-6. are independently funded. 
The majority of the edge computing development will be done by Canbo Ye, a Robotics Master's student. He already started in Fall '19 and is supported by the first year of this project until Summer. We request 50% tuition and stipend support for him for Fall '20 and Spring '21. His remaining 50% support will come from a different funding source belonging to Satya. 
John Kozar will help in assembling, installing and maintaining the hardware. Christoph Mertz will be the overall lead of the project. Satya will be the academic advisor of Canbo Ye. 

The first year of this project has already been approved and is currently running. It will finish before the start of the second year. By that time we will have the hardware installed on the transit bus, collected initial data and have preliminary edge computing software. 

By the end of Year 1 (June 30, '20):
1. Edge computer hardware installed on transit bus.
2. Preliminary edge computing software installed on edge computer.
3. Preliminary data collection.
4. First versions of applications tested offline. 

July - Sept. '20
1. Finish edge computing management software.
2. Collection of raw data for development purposes
3. Get first application ready for edge computing

Oct. - Dec. '20
1. Finish implementation and management of applications on the edge computer
2. Collection of raw data for development purposes
3. Collection of analyzed data for evaluation
4. Get other application ready for edge computing

Jan. - June '21:
1. edge computer: test complete system in the field, find and fix bugs, improve performance, evaluate performance
2. Applications:  test in the field, find and fix bugs, improve performance, evaluate performance
3. Commercialization: Present system to potential customers, partners and investors, make plan to scale up deployment
4. Finishing of project: Clean and document all software, document hardware configuration, publish open source software in public repositories, write final report        
Deployment Plan
Deployment is part of the project, the edge computing and its applications will be running on a real transit bus during normal transit operation. Several industry and government entities will be partners during the project: Argo, Washington County Transportation Authority, and RoadBotics. The supplier of the commercial recording system SafetyVision also indicated that they are interested in edge computing and like to be updated on the progress of the project. During the project we will discuss commercialization opportunities with these partners. They could be interested in running such a system to sell the data or be a customer to buy the data.     
Expected Accomplishments and Metrics
system running on a transit buses:
 - edge computer powered by bus and connected to video cameras on the bus
 - management software running on edge computer and management computer at central location
 - at least three applications running live on the edge computer: 
     1. parked and moving vehicle detection and counting 
     2. saving of data in privacy conserving manner 
     4. object detection
 -  successful running of edge system means:
     1. automatic startup and shutdown when bus starts and finishes operations
     2. running for one month with minimal interventions
     3. sending relevant warnings, information, or data through cellular and/or WiFi network
     4. remotely managing the system on the bus by monitoring its health, sending requests, and changing functionality   
 - successful running of system at central location means:
     1. display of relevant results in a timely manner through an interface
     2. display of health indicators of the edge system
     3. issue of requests, updates, and change of functionality to edge system
     4. correctly updating map with new information and removing expired or superseded information  

Metrics for individual applications:
Up-time, CPU and GPU usage, precision-recall 

Individuals Involved

Email Name Affiliation Role Position
cmertz@andrew.cmu.edu Mertz, Christoph Carnegie Mellon University PI Faculty - Researcher/Post-Doc
satya@cs.cmu.edu Satyanarayanan, Mahadev SCS Co-PI Faculty - Tenured
canboy@andrew.cmu.edu Ye, Canbo SCS Other Student - Masters


Amount of UTC Funds Awarded
Total Project Budget (from all funding sources)


Type Name Uploaded
Data Management Plan DMP_bus_on_edge_year2.docx Jan. 2, 2020, 1:48 p.m.
Progress Report 327_Progress_Report_2020-09-30 Oct. 11, 2020, 6:15 a.m.
Publication Towards a distraction-free waze. Nov. 27, 2020, 6:31 p.m.
Progress Report 327_Progress_Report_2021-03-31 March 30, 2021, 9:40 a.m.
Final Report Final_Report_-_327.pdf Aug. 26, 2021, 5:05 a.m.
Publication Lidar and monocular camera fusion: On-road depth completion for autonomous driving April 6, 2022, 5:18 a.m.
Publication Low-Cost 3D Model Acquisition for Rapid Accident Investigation April 6, 2022, 5:19 a.m.
Publication Improving Rush Hour Traffic Flow by Computer-Vision-Based Parking Detection and Regulations April 6, 2022, 5:20 a.m.
Publication CARLA Simulated Data for Rare Road Object Detection April 6, 2022, 5:20 a.m.

Match Sources

No match sources!


Name Type
Ohio State University Deployment Partner Deployment Partner
RoadBotics Deployment Partner Deployment Partner
Freedom Transit Deployment Partner Deployment Partner