Abstract
In the past two UTC cycles, we have built the "Bus on the Edge" as a data gathering and analysis platform that can monitor traffic and infrastructure. In this project cycle, we want to develop several applications that are specifically useful for the transit agency and relate to problems at bus stops. The lead application is to detect snow, ice, or any other hazards at or near the bus stop that would make it difficult for pedestrians to reach the bus. This application will be particularly useful to make bus stops more accessible for people with mobility problems.
Description
In the past 18 months we have developed a bus-on-the-edge system that uses cameras mounted on a bus to monitor infrastructure and traffic. The hardware components are a standard security system for transit buses: 5 cameras observing the surrounding of the bus and a computer to record the videos. We have installed our own software on that computer. The main functions of the software is to analyze and manage the data. The videos from the 5 cameras are far too much data to upload to the cloud. Instead, it needs to be quickly analyzed on the bus itself and then data of interest is sent to a central location where it is analyzed more thoroughly and used for various applications. The computer has a cellular connection for time critical communication and a WiFi link to exchange large amounts of data. The system is scheduled to be installed on a FreedomTransit bus in February 2021 and after that we will test a first set of applications: Hazard detection, high-definition (HD) map updates, traffic count, object example harvesting to train new detectors, road monitoring, and unusual event detection. Several of these applications are developed by other projects, who's sponsors or collaborators are Argo AI, NSF, RoadBotics, and OSU. The bus-on-the-edge has shown to be a great platform for additional projects, e.g. we successfully leveraged it to apply for a multi-year NSF grant.
In the proposed project we want to add more applications to the system, specifically those which will directly benefit the transit agency. All the applications concern issues at or around bus stops. Currently the transit agency finds out about issues through their drivers, complaints from users, or they have to send out staff. None of these are ideal. The bus driver has to operate the bus and deal with passenger and has little time to record issues. Users often complain only about major issues and are inconsistent in their reporting. Sending out staff is reliable, but it is expensive and time consuming. We want to use the bus-on-the-edge system to automate the reporting.
The lead application will be to detect snow, ice, or other obstacles at or near bus stops. It is important for all passengers to have an easy access to the bus, but it is especially important for people with mobility challenges to have a save and unimpeded path to the bus. We propose to develop algorithms that can find the bus stops and sidewalks and then determine if there are snow, ice, or other obstacles. These then trigger a notification to the transit agency or other users who then can act on this information. This application will be jointly developed by Dr. Mertz and a master's student.
The other applications we want to develop are detecting full trash cans, garbage, graffiti, or other damage at bus stops, and we want to determine on-time arrival at bus stops. These applications were suggested by the transit agency and will help them in their day-to-day operations. Developing these applications are good semester or Summer projects for students.
One other staff engineer will be part of the project. He will help to maintain and update the system and assist with data collection and training of detection models. The upkeep of the system is not only needed to develop the proposed applications, it will also make the system available to other projects, some which are ongoing (Argo AI and NSF) and others that we hope to propose in the near future.
The overall system we are proposing to develop will have several benefits to the transit agency. Applications like HD map update or traffic counts will give them a new revenue stream, as the users of these applications will have to pay for the service. Detection of problems at bus stops will make their operations more efficient. Noticing when there is snow, ice, or other obstacles at or around the bus stop will help them with their core mission: making transportation accessible to all users.
Timeline
July 2021: Initial data collection and selecting relevant data from previous collections
August 2021: Hiring of students for semester projects
September 2021: Development of module that regularly collects bus stop images
September-December 2021: Development of initial detectors for snow, ice, obstacles, graffiti, etc.
December-February 2022: Live testing of snow and ice detectors
December-April 2022: Live testing of obstacles, graffiti, trash, etc. detectors
January-April 2022: Development of final detectors for snow, ice, obstacles, graffiti, etc.
March 2022: Initial concept of displaying results to transit agency
April 2022: Presentation of initial results to transit agency for feedback
May-June 2022: Updates to system based on transit agency feedback
June 2022: Presentation of system to potential commercialization partners
Strategic Description / RD&T
Deployment Plan
Deployment will be part of the project from the very beginning. The data for the initial research will be collected by a commercial video surveillance system (cameras and computer from SafetyVision) mounted on a transit bus (FreedomTransit) running during regular transit operations. As soon as the analysis software is developed, it will be installed on the bus computer and tested during live bus operations. Once the software has matured, we will seek feedback from the transit agency on how the results can improve their work and how they can be made available during their day-to-day operations. The goal is for the applications to be commercialization ready by the end of the project. Possible commercialization partners are SafetyVision and RoadBotics. One option would be for SafetyVision to expand their current safety and surveillance offerings to include the infrastructure and traffic monitoring software developed in this and preceding project. A second option is for a company like RoadBotics to purchase the hardware from SafetyVision and develop their own continuous infrastructure monitoring system.
Expected Outcomes/Impacts
On the lowest level are the detectors which will be able to detect snow, ice, graffiti, etc. The performance metrics will be precision-recall curves and AP50 (average precision at 50% intersection-over-union). We expect a performance of AP50 = 50% to 70% of the fast detector running on the bus computer and AP50 = 80% to 90% for the off-line detector. The limitation for the fast detector is that it has to be able to run on one CPU of the bus computer.
The next level is the system level. It concerns the whole collection and analysis pipeline, from capturing the video to presenting the final result. The metric we want to achieve is that the system detects a new problem within at least one day.
The final level is the judgement of the end-user, in our case the transit agency. Our system should give them timely and actionable information, that will be faster or more cost-effective than their current practices or give them the new abilities, e.g. to make bus stops more accessible.
Expected Outputs
TRID
Individuals Involved
| Email |
Name |
Affiliation |
Role |
Position |
| cmertz@andrew.cmu.edu |
Mertz, Christoph |
SCS |
PI |
Other |
| satya@cs.cmu.edu |
Satyanarayanan, Mahadev |
SCS |
Co-PI |
Faculty - Tenured |
Budget
Amount of UTC Funds Awarded
$97556.00
Total Project Budget (from all funding sources)
$97556.00
Documents
| Type |
Name |
Uploaded |
| Data Management Plan |
Bus_on_the_edge__Applications_DMP_4bCv9mg.pdf |
Dec. 18, 2020, 6:05 a.m. |
| Progress Report |
356_Progress_Report_2021-09-30 |
Sept. 27, 2021, 2:42 p.m. |
| Publication |
Tracking_Ofodike_RISS_2021.pdf |
March 29, 2022, 2:18 p.m. |
| Publication |
Waste_Bins_Rotondo_RISS_2021.pdf |
March 29, 2022, 2:18 p.m. |
| Presentation |
Revolution in transportation - AI and infrastructure |
March 29, 2022, 2:18 p.m. |
| Progress Report |
356_Progress_Report_2022-03-30 |
March 29, 2022, 2:19 p.m. |
| Publication |
Lidar and monocular camera fusion: On-road depth completion for autonomous driving |
April 6, 2022, 5:39 a.m. |
| Publication |
Low-Cost 3D Model Acquisition for Rapid Accident Investigation |
April 6, 2022, 5:39 a.m. |
| Publication |
Improving Rush Hour Traffic Flow by Computer-Vision-Based Parking Detection and Regulations |
April 6, 2022, 5:40 a.m. |
| Publication |
CARLA Simulated Data for Rare Road Object Detection |
April 6, 2022, 5:41 a.m. |
| Final Report |
356_-_Final_Report.pdf |
Sept. 6, 2022, 4:17 a.m. |
| Associated Thesis / Dissertation |
Thesis_Tom_Bu_Towards_HD_Map_Updates_With.pdf |
Sept. 6, 2022, 4:44 a.m. |
| Associated Thesis / Dissertation |
Thesis_Tiffany_Ma_Mining_Spatio-Temporal_Attributes_of.pdf |
Sept. 6, 2022, 4:46 a.m. |
| Poster |
2022-RISS-Inaccessible_Bus_Stops-PANIGRAHI-Indu_poster.pdf |
Sept. 6, 2022, 4:46 a.m. |
| Poster |
2022-RISS-poster-Detecting_trash_cans-STORM-Tim.pdf |
Sept. 6, 2022, 4:47 a.m. |
| Progress Report |
356_Progress_Report_2022-09-30 |
Sept. 28, 2022, 7:31 p.m. |
Match Sources
No match sources!
Partners
| Name |
Type |
| FreedomTransit |
Deployment Partner Deployment Partner |
| RoadBotics |
Deployment Partner Deployment Partner |
| OSU |
Deployment Partner Deployment Partner |
