Abstract
This research focuses on co-designing and testing innovative rider communication tools to enable passengers with disabilities to effectively signal their intent to board public bus transit. People with disabilities often face challenges in visually or physically signaling their needs at bus stops, leading to missed rides, inadequate assistance, and reduced confidence in transit systems. By developing human-machine interfaces (HMIs) that facilitate clear communication between riders and drivers, this project aims to improve access to public transit.
Inspired by advancements in V2X communication, the U.S. Department of Transportation has demonstrated the potential of rider-to-vehicle communication through initiatives like the Integrated Dynamic Transit Operation System (IDTO) (Meng et al. 2018). In this system, riders could use an app to request connection protection, relayed to drivers via dispatchers, reducing wait times at transfer points by an average of seven minutes. Similar systems, such as flex-routing, allow riders to request pickups near fixed routes, addressing access in rural areas (e.g., Skagit Transit). While these approaches demonstrate the value of pre-trip communication, they primarily target specific scenarios and do not fully address the needs of passengers with disabilities, particularly those who cannot visually discern approaching buses or signal drivers effectively (Greer et al. 2014, Steinfeld et al. 2019).
To address these gaps, we will conduct participatory design workshops with transit riders, drivers, and advocacy groups to guide the creation of rider communication systems. These workshops will provide collaborative spaces for sharing experiences, identifying unmet needs, and developing solutions through activities like brainstorming, low-fidelity prototyping, and role-playing real-world scenarios. Insights from these sessions will inform functional prototypes, such as mobile apps or physical interfaces at bus stops. Simulated experiments will evaluate the impact of these tools on driver workload, operational safety, and rider satisfaction. These experiments will also assess how HMIs help drivers assist passengers while minimizing distractions and maintaining safety standards. For example, we will analyze how notifications via mobile apps or visual indicators at bus stops can aid drivers in identifying and assisting riders while avoiding information overload that could compromise situational awareness. Additionally, by accounting for various scenarios—such as varying weather conditions, crowded bus stops, and different disabilities—the tools will aim to address real-world challenges. We will also evaluate the timing and content of communications to optimize their effectiveness for drivers and riders alike.
Our participatory approach seeks to create communication systems that reduce barriers to mobility for passengers with disabilities, ensuring access to public transit. This research contributes to the broader goal of building safer transit systems.
Elizabeth Greer et al. 2014. Accessible Transportation Technologies Research Initiative (ATTRI). https://rosap.ntl.bts.gov/view/dot/3498
Huadong Meng et al. 2018. Development and Field Testing of an Integrated Dynamic Transit Operation System. https://rosap.ntl.bts.gov/view/dot/61940
Aaron Steinfeld et al. 2019. Increasing Access to Transit: Localized Mobile Information. Journal of Urban Technology 26, 3: 45–64. https://doi.org/10.1080/10630732.2019.1614896
Description
Timeline
Strategic Description / RD&T
Section left blank until USDOT’s new priorities and RD&T strategic goals are available in Spring 2026.
Deployment Plan
Our proposed work focuses on conducting participatory design with transit unions and members of transit advocacy groups, with input from transit agencies and manufacturers, to collaboratively generate, prototype, and test rider communication systems to help people with disabilities signal intent to board. This work will be split into two key phases. The first will be centered on a range of participatory design, low-fidelity prototyping, and hazard analysis activities aimed at examining how pre-boarding communication technologies might be leveraged to support transit riders with disabilities through advance notice of assistance needs. The second phase will involve the development and systematic evaluation of prototype rider communication systems to help people with disabilities signal intent to board, as well as impacts on operator workload. Such activities will center on testing whether and how these novel human-machine interfaces impact operational safety and drivers’ workload.
Phase 1 (Q1-2): Participatory Design and Hazard Analysis
Through a multi-step participatory design process, we will include passengers and drivers in designing future human-machine interfaces oriented to their needs and complementing their expertise and lived experiences (Fox et al. 2020) We see this approach as capable of generating novel system ideas based on the needs of drivers and riders and helping to mitigate emergent issues around advanced driver assistance and rider communication. This design process will include (1) in-person participatory workshops with drivers and passengers, including ideation activities and user enactments to deepen the design concepts proposed and (2) remote collaborative prototyping sessions with a cohort of participants.
In the initial stages of the grant period, we will host 2-3 workshops designed to generate novel ideas or design concepts worthy of further development. We will combine strategies that both encourage workshop participants to identify problematic aspects of existing systems, as well as envision alternatives that hold potential for an improved state in the future (Kensing and Madsen 1991). In later stages of each workshop, we will utilize techniques such as user enactments or Wizard of Oz prototyping to help participants elaborate on their designs and evaluate how they might improve current conditions. Such approaches involve creating hypothetical scenarios in which individuals or teams act out how users might interact with a product, system, or service as they envision it (Elsden et al. 2017; Odom et al. 2012). These techniques can be particularly valuable in the early stages of design to gather feedback and refine the user experience.
Following our workshop engagements, over a period of 4 months, we will conduct remote, iterative participatory prototyping sessions with a subset of drivers and passengers, further discussing their needs and prototyping the design ideas they generated (Bødker and Grønbæk 1991; Buchenau and Suri 2000). Though prototypes have long been used as stand-ins to test particular components of a system (Houde and Hill 1997), this application of the technique seeks to pursue iterative development of a design concept centered around multi-stakeholder engagement. Here, we aim to support the imagination of future transit interactions augmented by new technologies, with and for drivers and passengers. To do this, we will use Figma, or another similar web-based collaborative design interface tool, to create prototypes of participants’ design ideas. The resulting prototypes will form a corpus of design ideas, and the subsequent stages of our research will be focused on assessing and giving form to these concepts.
Next, we will conduct a series of collaborative hazard analysis sessions with both drivers and riders to capture and rank the severity of potential hazards as a way to inform our human-machine interface designs for new driver support and passenger communication systems. While hazard analysis is typically an engineering-focused activity, including bus drivers and members of the public will allow our team to leverage their expertise on real-world situations that could cause issues and harms. During these sessions, bus drivers and riders can provide scenarios that could challenge new interfaces or automation. Our team will then consider the technical implications that could cause failures and harms. This thread of work will investigate the benefits and challenges of conducting participatory hazard analysis activities with non-engineer members of the public. Such participatory hazard analysis approaches have been called for and considered in disaster preparedness research (Pearce 2005, Rood 2012, Sarzynski and Cavaliere 2018); however, there is limited research exploring community engagement in hazard analysis for vehicle automation and new public transit technology. Furthermore, while community participation can improve the variety of hazards identified and build community trust, it can be challenging and there are documented needs to develop processes for effectively including various public perspectives in hazard analysis activities (Rossignol et al. 2015; Shmueli et al. 2021), such as those of professional drivers and transit riders in the case of our research.
During our collaborative hazard analysis sessions, we will first introduce hazard analysis thinking through a short round of playing “What could go wrong?,” an informal game developed by Co-PI Martelaro to help people think creatively about the potential harms of new automation technologies (Martelaro and Ju 2020). We will then introduce and collaboratively document potential harms and their severity using a Failure Modes and Effects Analysis (FMEA) spreadsheet. FMEA (Mikulak et al. 2017; US Department of Defense) is a commonly used hazard analysis method and documents possible failures that can occur, the potential impacts of such failures to cause harm to life, property, or systems operations, as well as the severity of such harms and the probability of these harms occurring. We will conduct the hazard analysis sessions with bus drivers, rider advocacy groups, and disability advocates. Based on their respective expertise, we anticipate that bus drivers will identify hazards related to the vehicle and road systems, rider advocates will identify hazards for passengers in the cabin or moving outside the bus when they are pedestrians, and disability advocates will identify hazards related to their safety and stability inside the bus, navigating to the bus, and signaling a driver when at a bus stop. We will compare each group's hazard analysis to see where there is overlap and where different ideas emerge. We will then compile all FMEA spreadsheets into a single spreadsheet that documents the collective set of hazards described across the sessions. These hazards will then be used to support our design interventions for new human-machine interface technologies to support drivers and riders.
Phase 2 (Q 3-4): Prototype Development and Simulation-Based Experiments
Once our team of stakeholders has agreed on the driver HMI and intelligent passenger communication concepts to test, we will develop functional prototypes and test them in simulated driving scenarios. We will use the free simulation environment Strange Land (Goedicke et al. 2022b) based on the widely available Unity game engine and developed by a team at Cornell Tech led by Dr. Wendy Ju. Strange Land is specifically designed for running multi-participant studies, such as the driver and pedestrian interaction studies critical to this research proposal. Strange Land also includes tooling to allow multi-perspective data analysis of interactions between drivers, pedestrians, and the environment (Goedicke et al. 2022a).
Our goal in this phase will be to experimentally test what information and interactions support disabled passengers in communicating their needs in advance without overloading drivers. While the final concepts we develop and test will be determined through our Phase 1 participatory design process with drivers and riders, we will describe our development and experiment plan here with plausible examples.
Functional HMI Prototype Development.
Our team will develop functional prototypes for each HMI concept leveraging teams of student developers led by a Ph.D. student and managed by the Co-PIs. Our PI team has a strong track record of developing functional system prototypes with student teams. A past example includes the development of functional mobile apps to support people with disabilities to independently ride autonomous vehicles led by Co-PIs Martelaro and Fox (Martelaro et al. 2022).
We will develop our internal driver HMI systems using web-based tooling and Javascript as it provides an abundance of tools and user interface libraries for quickly developing functional UI systems that include data displays, user inputs, animation, and audio. Additionally, we will prototype any physical interfaces that may be needed, such as buttons, switches, microphone inputs, and gestural interfaces that a bus driver might interact with using widely available hardware prototyping tools (i.e., Arduino, Raspberry Pi, component electronics). These technologies are easily integrated into the Unity-based Strange Land simulation environment and can be superimposed on digital bus models and in-cabin screens.
Similar to our Internal Driver HMI prototype, we will develop our communication interfaces for disabled passengers with web-based interface technologies, simulated 3D environment technologies, and physical interfaces. Interfaces best suited for display on mobile phones will be tested on real hardware connected via software to the Strange Land simulation. Any interfaces that may be developed for physical infrastructure (i.e., a bus stop) will be developed as 3D models in the simulation environment or with physical electronic prototypes on a mock-up bus stop, again connected to the simulated environment.
We plan to empirically test the proposed driver facing HMI design concepts and rider communication tools for riders with disabilities in a virtual reality environment. While simulation-based studies may not always represent the real-world [134] they are generally valid when compared with on-road studies (Kaptein et al. 1996). Further, simulation studies are critical in testing new concepts before on road use. Our studies will provide the foundational evidence for the impacts of new interfaces on driver workload, operational safety, vulnerable road user safety, and rider communication that will then motivate future HMI development and higher fidelity studies by vehicle manufacturers and retrofitters prior to deployment in the real world. We specifically leverage mid-fidelity simulation technology that can be run on consumer-grade computers and virtual reality headsets so we can conduct our studies with drivers and riders in various locations near where they live or work. Conducting our studies in the locations of our key stakeholders will improve participant recruitment and retention and foster deep participatory design engagement throughout the project.
All HMI systems will be tested across different design variables, allowing us to test hypotheses about which interface designs will improve aspects of safety, while reducing or maintaining appropriate driver workload and rider user experience. As the specific system designs will be collaboratively developed with bus drivers and riders, we provide representative study examples to explain how we will design the simulation experiments.
Study Apparatus:
In each study, we will develop simulated scenarios co-designed with our driver and rider partners. Co-designing scenarios with bus drivers and riders will enhance ecological validity and will ensure that our simulation represents the complexities of real transit environments which can have many distracting factors that influence workload (Salmon et al. 2011). Prior research from the DoT has noted how working directly with drivers exposed a fascinating amount of detail around the complexities and workload inducing stresses of bus driving (Graving et al. 2018). We will create a set of realistic traffic scenarios on top of the scenarios provided with the Strange Land simulator that will be useful across our studies, including light and heavy traffic on straight roads of different widths, crowded interactions, transit hubs with bus docks, surface street bus stops, uncontrolled left and right turns, signaled intersections and signed intersections. We will model these scenarios based on real-world streets in Pittsburgh to enhance the realism for our driver and rider populations. The scenarios will include pre-programmed critical events that drivers, riders, or pedestrians need to manage (e.g., a bus approaching a stop on a rainy day). Participants will experience the simulated environment using a head-mounted, wireless virtual reality display (Meta Quest 3) driven by a consumer-grade high-powered laptop. This will allow the participants to freely look and walk around the world to direct their attention and will allow our setup to be portable so that we can conduct studies with our partners in Pittsburgh. For studies involving bus drivers, we will position a steering wheel, pedals, and driver seat similarly to a transit bus.
Experiment Design: We will generally leverage within-subjects, factorial-based study designs. Such study designs will allow us to understand how different design aspects influence the performance of our interfaces with participants across different tasks. The factorial design will show both the main effects of major design differences as well as the interaction effects between factors. We will use within-subjects experiments to control for personal differences between drivers or riders and because they offer greater efficiency in the number of participants required for each study, which is particularly important with our bus driver participants given their time constraints. We plan to engage between 20-30 participants in each study. This number is determined to provide ample power for statistical analyses given we expect moderate effect sizes when comparing different interface designs. We will conduct repeated measures across varying driving and environmental situations (i.e., weather, traffic conditions, road types) to test generalizability across scenarios. While the final study details will be collaboratively determined with our partners and based on our design concepts, some example studies with details on their study design, variable tested, repeated measures, and participants include:
Accessible Rider Communication Systems - Driver HMI - Study Design: 2 (Identifying riders unaided vs. being alerted to riders on display) x 2 (1 stop in advance vs. 3 stops in advance). Hypothesis: Alerts signaling upcoming passengers will prevent missing riders but increase stress. Increased time of notification will reduce stress. Repeated measures across 5-10 different bus stop scenarios (i.e. clear and sunny, night, rain, fog, heavy traffic, many pedestrians). Participants: 20 commercial driver's license-holding (CDL) union drivers with varying years of experience.
Accessible Rider Communication Systems - Rider HMI - Study Design: 2 (send notification vs. intelligent conversation and information sharing) x 2 (Bus 1 minute away vs. Bus 5 minutes away). Hypothesis: Intelligent conversation will increase rider trust, increased time will increase rider trust, and limited time and intelligent communication may be stressful or challenging. Repeated measures across 5-10 different scenarios (i.e., clear and sunny, night, rain, fog, heavy traffic, many pedestrians). Participants: 20 people recruited from local Pittsburgh population and transit advocacy groups. Participants stratified across sub-groups of able-bodied, low-vision, low-hearing, and mobility-aid-using participants.
Marion Buchenau and Jane Fulton Suri. 2000. Experience prototyping. In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques (DIS ’00), 424–433. https://doi.org/10.1145/347642.347802
Chris Elsden, David Chatting, Abigail C. Durrant, Andrew Garbett, Bettina Nissen, John Vines, and David S. Kirk. 2017. On Speculative Enactments. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17), 5386–5399. https://doi.org/10.1145/3025453.3025503
Sarah E. Fox, Vera Khovanskaya, Clara Crivellaro, Niloufar Salehi, Lynn Dombrowski, Chinmay Kulkarni, Lilly Irani, and Jodi Forlizzi. 2020. Worker-Centered Design: Expanding HCI Methods for Supporting Labor. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing
David Goedicke, Harald Haraldsson, Navit Klein, Lunshi Zhou, Avi Parush, and Wendy Ju. 2022. Rerun: Enabling Multi-Perspective Analysis of Driving Interaction in VR. In Adjunct Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 204–205. https://doi.org/10.1145/3544999.3550155
David Goedicke, Carmel Zolkov, Natalie Friedman, Talia Wise, Avi Parush, and Wendy Ju. 2022. Strangers in a Strange Land: New Experimental System for Understanding Driving Culture Using VR. IEEE Transactions on Vehicular Technology 71, 4: 3399–3413. https://doi.org/10.1109/TVT.2022.3152611
Justin Graving, Paige Bacon-Abdelmoteleb, John L. Campbell, and Battelle Seattle Research Center. 2019. Human Factors for Connected Vehicles Transit Bus Research. https://doi.org/10.21949/1530144
Stephanie Houde and Charles Hill. 1997. What do Prototypes Prototype? In Handbook of Human-Computer Interaction (2nd ed.), Marting G. Helander, Thomas K. Landauer and Prasad V. Prabhu (eds.). North-Holland, Amsterdam, 367–381. https://doi.org/10.1016/B978-044481862-1.50082-0
Nico A. Kaptein, Jan Theeuwes, and Richard van der Horst. 1996. Driving Simulator Validity: Some Considerations. Transportation Research Record 1550, 1: 30–36. https://doi.org/10.1177/0361198196155000105
Finn Kensing and Halskov Madsen. 1991. Generating Visions: Future Workshops and Metaphorical Design. In Design at Work. CRC Press.
Nikolas Martelaro, Patrick Carrington, Sarah Fox, and Jodi Forlizzi. 2022. Designing an Inclusive Mobile App for People with Disabilities to Independently Use Autonomous Vehicles. In Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ’22), 45–55. https://doi.org/10.1145/3543174.3546850
Nikolas Martelaro and Wendy Ju. 2020. What Could Go Wrong? Exploring the Downsides of Autonomous Vehicles. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ’20), 99–101. https://doi.org/10.1145/3409251.3411734
Raymond J. Mikulak, Robin McDermott, and Michael Beauregard. 2017. The Basics of FMEA. Productivity Press, New York. https://doi.org/10.1201/b16656
William Odom, John Zimmerman, Scott Davidoff, Jodi Forlizzi, Anind K. Dey, and Min Kyung Lee. 2012. A fieldwork of the future with user enactments. In Proceedings of the Designing Interactive Systems Conference (DIS ’12), 338–347. https://doi.org/10.1145/2317956.2318008
Laurie Pearce. 2005. The Value Of Public Participation During a Hazard, Impact, Risk And Vulnerability (HIRV) Analysis. Mitigation and Adaptation Strategies for Global Change 10, 3: 411–441. https://doi.org/10.1007/s11027-005-0054-7
Jason Rood. 2012. Public Participation in Emergency Management. Dissertations and Theses. https://doi.org/10.15760/etd.333
Nicolas Rossignol, Pierre Delvenne, and Catrinel Turcanu. 2015. Rethinking Vulnerability Analysis and Governance with Emphasis on a Participatory Approach. Risk Analysis 35, 1: 129–141. https://doi.org/10.1111/risa.12233
Paul M. Salmon, Kristie L. Young, and Michael A. Regan. 2011. Distraction ‘on the buses’: A novel framework of ergonomics methods for identifying sources and effects of bus driver distraction. Applied Ergonomics 42, 4: 602–610. https://doi.org/10.1016/j.apergo.2010.07.007
Andrea Sarzynski and Paolo Cavaliere. 2018. Public Participation in Planning for Community Management of Natural Hazards. In Oxford Research Encyclopedia of Natural Hazard Science. https://doi.org/10.1093/acrefore/9780199389407.013.151
Deborah F. Shmueli, Connie P. Ozawa, and Sanda Kaufman. 2021. Collaborative planning principles for disaster preparedness. International Journal of Disaster Risk Reduction 52: 101981. https://doi.org/10.1016/j.ijdrr.2020.101981
U.S. Department of Defense. 1949. Procedures for Performing a Failure Mode Effect and Critical Analysis.
Expected Outcomes/Impacts
Centering safety, this project aims to develop driver- and rider-driven human-machine interaction that could further enhance operations. This work will produce novel system designs aimed at introducing communication between passengers with disabilities and drivers regarding assistance needs in advance of boarding. We will empirically evaluate these designs with particular focus on the timing of information exchange, to avoid overloading the driver. Finally, this research will produce methodological contributions in the form of participatory, multi-stakeholder hazard analysis, designed to broaden who is involved in the process of defining risks and potential harms of emergent systems. We expect this will lead to more robust and complete hazard analysis to more effectively avoid risks.
Expected Outputs
- Two academic papers based on our design activities and simulation based experiments, prepared during the grant period. These papers will be published in relevant human-computer interaction and transportation research venues.
- A white paper translating our academic findings into recommendations for transit agencies, advocacy organizations and government officials seeking to support public transportation
- Open-source the simulation environments and test procedures. We will circulate research materials, simulation tools, and findings as publicly available materials in an open access toolkit at codesigningtransit-dot-com.
TRID
The research that most closely aligns with our proposed work is an active project from researchers at the Texas Transportation Institute, entitled “Accessible Public Bus and Rail Passenger Information for Riders with Vision Disabilities.” Though still in progress, this work aims to create a guide for transit operators to help passengers with vision impairments navigate transit systems independently, focusing on accessibility challenges across all journey phases. It emphasizes improving communication methods, such as tactile and auditory information and website accessibility, while providing a framework and measurement tools for agencies to assess and enhance information systems’ effectiveness.
In contrast, our proposed project centers on developing and testing communication tools to enable passengers with disabilities to signal their intent to board public transit effectively. This work emphasizes participatory design with stakeholders to co-create human-machine interfaces that address real-world challenges, such as visually or physically signaling needs.
Both projects aim to improve transit operations, with particular focus on passengers with disabilities. If we are awarded funding to conduct this proposed work, we will actively seek opportunities to collaborate with researchers at TTI.
Individuals Involved
| Email |
Name |
Affiliation |
Role |
Position |
| sarahf@andrew.cmu.edu |
Fox, Sarah |
Carnegie Mellon University |
PI |
Faculty - Untenured, Tenure Track |
| nikmart@cmu.edu |
Martelaro, Nikolas |
Carnegie Mellon University |
Co-PI |
Faculty - Untenured, Tenure Track |
Budget
Amount of UTC Funds Awarded
$90000.00
Total Project Budget (from all funding sources)
$200000.00
Documents
Match Sources
No match sources!
Partners
| Name |
Type |
| AFL-CIO |
Deployment Partner_ Deployment Partner_ |
| ATU |
Deployment Partner_ Deployment Partner_ |
| TWU |
Deployment Partner_ Deployment Partner_ |
| Pittsburghers for Public Transit |
None |
