#335 Real-Time Traffic Analytics at Intersections

Principal Investigator
Srinivasa Narasimhan
Start Date
July 1, 2020
End Date
June 30, 2021
Research Type
Grant Type
Grant Program
FAST Act - Mobility National (2016 - 2022)
Grant Cycle
2020 Mobility21 UTC


Central to a smart transportation system is access to real-time data especially at points of planned conflict such as intersections.  Unfortunately, visual data is too expensive in bandwidth for remote analysis. The proposed research focuses on performing analysis at the edge and only transmitting compact analysis results.  Algorithms will be developed to detect and track vehicles, pedestrians, and bicyclists to summarize counts, travel direction, and notable events such as near collisions. Summary results will be submitted to a website for a real-time look of the road environment.    
More and more city planners are taking into consideration smart mobility solutions to address transportation needs. For example, self-driving cars offer independence for seniors and people with disabilities, greater road safety, cost savings through ride sharing, increased productivity, reduced congestion, and reduced fuel use and carbon emissions. But what type of information and data is needed for city planners to accommodate smart transportation systems? 

Valuable information can be extracted from visual data, but it is expensive in bandwidth and time to transfer to a computer for processing and interpretation. The proposed research is focused on developing algorithms that can analyze visual data at the edge and share the results in real-time. Intersections are of particular interest because they are bottlenecks for efficient travel. As part of this research, we will develop an edge-computing platform, deploy the platform to an intersection, and build a web-based interface for displaying the real time analytics. 

Edge-Computing Platform
The platform will automatically capture, process, and interpret visual data, which will be wirelessly transmitted. It will consist of hardware protected by a weatherproof enclosure and a software architecture for processing visual data and transmitting results. The primate hardware components are a CPU, GPU, ethernet switch, edge router, and remote power switch. This platform permits high-performance computing, ingesting large amounts of data from cameras, and remote control of the system. We expect to install at least 4 cameras for an intersection.

Algorithms and Analytics
We have experience training deep learning models to detect and classify these objects. Crucial for computing critical analytics is tracking objects to observe temporal behavior. This is challenging due to obstructions caused by objects on the road. Capturing data from multiple cameras assists in providing different viewpoints, but corresponding images from all of the cameras needs to be computed. We will develop methods for identifying corresponding objects from multiple, unsynchronized cameras. Tracking objects will permit observing the path that each object traverses throughout the intersection. This will allow us compute statistics on how vehicles travel. Novel methods will be developed to identify and predict anomalous behavior such as near collisions, unexpected entry into the roadway, jaywalking, etc.

Analytics Reporting Platform
Information computed by the edge-computing platform will be displayed in an easy-to-visualize web-based platform. We will build this platform using a privately controlled server on Carnegie Mellon University’s campus. The platform will show live video feeds, as well as all of the analytics over time. Users with permitted access will be able to navigate the temporal data to view analytics within specific time frames. 

The Team
The proposed applied research will be conducted by a team of experts from computer vision, imaging and optics, robotics, machine learning, and systems development with strong experiences in deploying transportation related applications.  

Carnegie Mellon University, and the City of Pittsburgh provide a perfect ecosystem for this research. Key personnel have been part of the University Transportation Center (UTC) for Technologies for Safe and Efficient Transportation (TSET), where they conduct basic and applied research in safety and scene understanding and a National Science Foundation award on Cyberphysical Systems with a Technology Transfer to Practice (TTP) option. Metro21 supports numerous applied research projects at Carnegie Mellon University and will serve as a liason with the City of Pittsburgh and other deployment stakeholders. This proposal fulfills a critical need in bringing together basic and applied research efforts in a principled manner to design, develop, and deploy a system for more efficient transportation systems. 
•	July 1, 2020 – August 31, 2020: Begin process for intersection instrumentation. Design and build enclosure for computing equipment. Stress test equipment. 
•	September 1, 2020 – September 30, 2020: Instrument intersection with camera and computing equipment enclosure.
•	October 1, 2020 – October 31, 2020: Begin initial data collection for algorithm development.
•	November 1, 2020 – January 31, 2021: Continue algorithm development, build front-end and back-end of website. 
•	February 1, 2021 – February 29, 2021: Meet with stakeholders to present preliminary results and obtain feedback.
•	March 1, 2021 – May 31, 2021: Revise methods based on feedback.
•	June 1, 2021 – June 30, 2021: Final Report
Deployment Plan
We have industrial collaborations with Bosch, Intel, and NVidia. We have experience instrumenting intersections based on past deployments through projects with Heinz Endowments, Metro21, and Mobility21. Past work was deployed with the support of The City of Pittsburgh Department of Mobility Infrastructure (DOMI). DOMI will support the deployment of the equipment on City property when needed (see Supplemental Information for email of support).     
Expected Accomplishments and Metrics
By the conclusion of this award, at least one system will be deployed at another intersection in the City of Pittsburgh. The system will serve as a test bed for collecting and processing data for use by researchers. Novel algorithms will be developed and published along with experimental results. Algorithms will be thoroughly tested and analyzed with real data collected at the installation site. 

Datasets will be collected in a variety of weather and lighting conditions. These unique datasets will be published on-line for use by researchers in order to advance the fields of computer vision, artificial intelligence, etc. 

Results will be benchmarked on a concrete list of key stakeholder identified mission-critical tasks. Comprehensive testing will be performed in multiple lighting and weather conditions. The following will be delivered: 1) semi-annual reports, 2) final program report, 3) designs of the physical system, 4) databases of collected anonymized visual data, 5) experimental findings, 6) documentation on construction, installation, and usage of system components, and 7) site visit by interested stakeholders.

Individuals Involved

Email Name Affiliation Role Position
dnarapur@andrew.cmu.edu Narapureddy, Dinesh Carnegie Mellon University Other Student - PhD
srinivas@cs.cmu.edu Narasimhan, Srinivasa Carnegie Mellon University PI Faculty - Tenured
rtamburo@cmu.edu Tamburo, Robert Carnegie Mellon University Co-PI Faculty - Researcher/Post-Doc
tzhi@cs.cmu.edu Zhi, Tiancheng Carnegie Mellon University Other Student - PhD


Amount of UTC Funds Awarded
Total Project Budget (from all funding sources)


Type Name Uploaded
Data Management Plan dmp_intersections2019.pdf Jan. 3, 2020, 1:58 p.m.
Project Brief summary_slides.pptx March 17, 2020, 7:32 a.m.
Publication Traffic4d: single view reconstruction of repititious activity using longitudinal self-supervision March 30, 2021, 10:26 a.m.
Presentation Visual Understanding of Traffic at Intersections: Access to real-time information Oct. 1, 2020, 10:31 a.m.
Progress Report 335_Progress_Report_2020-09-30 Oct. 1, 2020, 10:41 a.m.
Progress Report 335_Progress_Report_2021-03-31 March 30, 2021, 10:52 a.m.
Publication Traffic4D: Single View Reconstruction of Repetitious Activity Using Longitudinal Self-Supervision July 12, 2021, 11:33 a.m.
Final Report Final_Report_-_335.pdf Aug. 1, 2021, 9:53 a.m.
Publication Occlusion-net: 2d/3d occluded keypoint localization using graph networks April 6, 2022, 5:07 a.m.
Publication Deconvolving Diffraction for Fast Imaging of Sparse Scenes April 6, 2022, 5:08 a.m.
Publication Exploiting & Refining Depth Distributions with Triangulation Light Curtains April 6, 2022, 5:08 a.m.

Match Sources

No match sources!


Name Type
City of Pittsburgh Department of Mobility and Infrastructure Deployment Partner Deployment Partner