Abstract
One transit buses from Washington County Transportation Authority will be equipped with cameras and an edge computer connected to a cellular network for time critical and WiFi for high bandwidth communication. The computer detects relevant events and saves data of interest in a privacy preserving manner. This system will enable many mobility relevant observations related to traffic, weather, infrastructure, or disruptions, several of them we will implement and test.
Description
Data is essential to maximize mobility and make the transportation system resilient. Examples are real time schedules, traffic conditions, weather conditions, or infrastructure inspection. In this project we want to develop an efficient data collection platform that can monitor complete routes on an hourly basis. The idea is to make use of transit buses that travel on main roads on a regular basis. They are often already equipped with cameras and an electronics cabinet that houses recording equipment and the automatic annunciation system. We will use such a commercial system, where the recording device is a full-fledged computer with all the hardware components to use it as an edge computer. All the edge computing software modules will run in parallel to the pre-installed recording system. The edge computing system will analyze the video streams live to detect relevant events and sends these detections over a cellular network to a central location. Data that is interesting but not time critical will be saved and uploaded via WiFi at the end of the day. Potentially relevant data will be saved for a set amount of time locally and analyzed or uploaded upon request. The edge computer system will also handle privacy: Before data is saved any personally identifiable information will be eliminated, e.g. by blurring out faces or license plates. The data collected by such a system can be used for event detection, traffic modeling and infrastructure monitoring and thereby provide input for up-to-date detailed maps of roads and traffic. Up-to-date detailed maps are one essential component of autonomous driving, but they are also needed for traffic management, planning, and infrastructure maintenance. Concrete examples are traffic counts, counting of parked cars, observations of road construction, pothole detection, detecting landslide precursors, measuring snow cover, or observing crossing of wildlife.
In this project we will develop and implement the core edge computing, basic detection, and privacy modules. We will make the platform and the collected data available for other projects, and other projects will complement our data. Some of the other projects will also supply software modules. We will modify them to run on the edge computing system. The other projects are:
1. Updating HD maps with videos from vehicles
This is a Argo sponsored MSCV (Master of Science in Computer vision) project where 3 students use video data to update HD maps. The plan is to run their change-detection software on the edge computer
2. Landslide detection and prediction
Ongoing effort by Amit Acharya, CEE, and Christoph Mertz to find precursors and indicators of landslides like cracks, excessive water or debris on the road. These will be used to model, analyze, and predict landslides. The initial effort was a project sponsored by Metro21 "Landslide prediction and analysis from images". The data collected by the edge computer will be used to train and test the detection algorithms.
3. Improving rush hour traffic flow by computer-vision-based parking detection and regulations
Just finished UTC project by Christoph Mertz and Sean Qian. The software developed under this project to count parked and moving vehicles will be one of the modules run on the edge computer.
4. Using municipal vehicles as sensor platforms to monitor the health and performance of the traffic control system.
This is a project from Mark McCord and his colleagues at OSU. They use video from transit buses to obtain traffic flow estimates across extensive urban roadway networks. Their data is collected by a similar system to ours. We have already received data from them and used it to test our parked and moving data detection. We will continue to exchange data and algorithms.
5. Pedestrian and Cyclists Detection for Transit Buses PHASE II
A NSF-SBIR project with Novateur Research Solutions as the prime and CMU as a sub-contractor. Video data collected by this SBIR project will be used to complement the proposed project. The data will be collected in and around Pittsburgh.
6. Road surface monitoring
The startup company RoadBotics (Christoph Mertz is co-founder and Chief Scientist) has commercialized road surface monitoring. They have a collection of road images from over 180 customers (towns and cities in several states and countries). RoadBotics will share this data, as permitted by the customer. A data sharing agreement between RoadBotics and the Robotics Institute is in the works.
The aforementioned projects 1.-6. are independently funded.
The majority of the edge computing development will be done by Canbo Ye, a Robotics Master's student. He already started in Fall '19 and is supported by the first year of this project until Summer. We request 50% tuition and stipend support for him for Fall '20 and Spring '21. His remaining 50% support will come from a different funding source belonging to Satya.
John Kozar will help in assembling, installing and maintaining the hardware. Christoph Mertz will be the overall lead of the project. Satya will be the academic advisor of Canbo Ye.
The first year of this project has already been approved and is currently running. It will finish before the start of the second year. By that time we will have the hardware installed on the transit bus, collected initial data and have preliminary edge computing software.
Timeline
By the end of Year 1 (June 30, '20):
1. Edge computer hardware installed on transit bus.
2. Preliminary edge computing software installed on edge computer.
3. Preliminary data collection.
4. First versions of applications tested offline.
July - Sept. '20
1. Finish edge computing management software.
2. Collection of raw data for development purposes
3. Get first application ready for edge computing
Oct. - Dec. '20
1. Finish implementation and management of applications on the edge computer
2. Collection of raw data for development purposes
3. Collection of analyzed data for evaluation
4. Get other application ready for edge computing
Jan. - June '21:
1. edge computer: test complete system in the field, find and fix bugs, improve performance, evaluate performance
2. Applications: test in the field, find and fix bugs, improve performance, evaluate performance
3. Commercialization: Present system to potential customers, partners and investors, make plan to scale up deployment
4. Finishing of project: Clean and document all software, document hardware configuration, publish open source software in public repositories, write final report
Strategic Description / RD&T
Deployment Plan
Deployment is part of the project, the edge computing and its applications will be running on a real transit bus during normal transit operation. Several industry and government entities will be partners during the project: Argo, Washington County Transportation Authority, and RoadBotics. The supplier of the commercial recording system SafetyVision also indicated that they are interested in edge computing and like to be updated on the progress of the project. During the project we will discuss commercialization opportunities with these partners. They could be interested in running such a system to sell the data or be a customer to buy the data.
Expected Outcomes/Impacts
system running on a transit buses:
- edge computer powered by bus and connected to video cameras on the bus
- management software running on edge computer and management computer at central location
- at least three applications running live on the edge computer:
1. parked and moving vehicle detection and counting
2. saving of data in privacy conserving manner
4. object detection
- successful running of edge system means:
1. automatic startup and shutdown when bus starts and finishes operations
2. running for one month with minimal interventions
3. sending relevant warnings, information, or data through cellular and/or WiFi network
4. remotely managing the system on the bus by monitoring its health, sending requests, and changing functionality
- successful running of system at central location means:
1. display of relevant results in a timely manner through an interface
2. display of health indicators of the edge system
3. issue of requests, updates, and change of functionality to edge system
4. correctly updating map with new information and removing expired or superseded information
Metrics for individual applications:
Up-time, CPU and GPU usage, precision-recall
Expected Outputs
TRID
Individuals Involved
| Email |
Name |
Affiliation |
Role |
Position |
| cmertz@andrew.cmu.edu |
Mertz, Christoph |
Carnegie Mellon University |
PI |
Faculty - Research/Systems |
| satya@cs.cmu.edu |
Satyanarayanan, Mahadev |
SCS |
Co-PI |
Faculty - Tenured |
| canboy@andrew.cmu.edu |
Ye, Canbo |
SCS |
Other |
Student - Masters |
Budget
Amount of UTC Funds Awarded
$99331.00
Total Project Budget (from all funding sources)
$99331.00
Documents
| Type |
Name |
Uploaded |
| Data Management Plan |
DMP_bus_on_edge_year2.docx |
Jan. 2, 2020, 1:48 p.m. |
| Progress Report |
327_Progress_Report_2020-09-30 |
Oct. 11, 2020, 6:15 a.m. |
| Publication |
Towards a distraction-free waze. |
Nov. 27, 2020, 6:31 p.m. |
| Progress Report |
327_Progress_Report_2021-03-31 |
March 30, 2021, 9:40 a.m. |
| Final Report |
Final_Report_-_327.pdf |
Aug. 26, 2021, 5:05 a.m. |
| Publication |
Lidar and monocular camera fusion: On-road depth completion for autonomous driving |
April 6, 2022, 5:18 a.m. |
| Publication |
Low-Cost 3D Model Acquisition for Rapid Accident Investigation |
April 6, 2022, 5:19 a.m. |
| Publication |
Improving Rush Hour Traffic Flow by Computer-Vision-Based Parking Detection and Regulations |
April 6, 2022, 5:20 a.m. |
| Publication |
CARLA Simulated Data for Rare Road Object Detection |
April 6, 2022, 5:20 a.m. |
Match Sources
No match sources!
Partners
| Name |
Type |
| Ohio State University |
Deployment Partner Deployment Partner |
| RoadBotics |
Deployment Partner Deployment Partner |
| Freedom Transit |
Deployment Partner Deployment Partner |
