Abstract
The goal of this proposal is to make the GetGoing system which enables seniors to get transportation information over the phone, to also serve individuals with impairments who need more information about how to get to a desired destination such as condition of sidewalks and where the relevant entrance of a building is located. The team will work with PsthVu and UPMC to help individuals access all of the UPMC hospitals.
Description
The goal of this proposal is to enable GetGoing to provide information for users with impairments for travel and access to all of the UPMC hospitals.
The GetGoing team has built an agent that gives local travel information over the phone and is designed to be easy for seniors to use. Most seniors do not have smartphones and are more comfortable making a telephone call. The team has registered a provisional patent on the techniques used to make an agent understandable. We also created a complete working system. Recent work has made the system more robust to palliate speech recognition errors, better understand user intentions, and deliver directions in a more natural way. Present work centers on getting the system to adapt to the caller by using information that it gathers as the conversation proceeds. This means adapting to the user’s choice of words, to the user’s speaking rate and to the user’s preference for more time to write things down. Since GetGoing has callerid, it can remember those preferences from a user’s dialog and activate them next time they call. The team is also working on making the system to adjust speaking patterns in the middle of the call to better guide users who are unfamiliar with system-generated speech.
From discussions with specialists on aging at UPMC, the team has learned that getting the directions to a place is not sufficient for most seniors. By using a Google backend, GetGoing has given directions that include how to get to a bus stop, which bus to take, what time to take it and where to get off the bus. However, once a user gets off the bus, they do not usually know which building a doctor’s office is located in, where the entrance to a building is or which entrance to use. For example, if a user has a doctor’s appointment at Presbyterian Hospital, once they get off the bus, they might need to enter on Fifth Avenue or they might need to go up one of the side streets of the hospital. For users who have impairments like low vision, limited ability to walk, or use a wheelchair, this kind of information is essential. Beyond knowing which door to enter and how to get there, users with impairments need to know where broken sidewalks, ramps and uneven pavement are if they are to remain mobile and independent.
Yet Google, which is used as the backend database for GetGoing, does not provide this information. Fortunately, the PathVu company (http://www.pathvu.com/) has gathered information like this into a database that will very soon be accessible via API. The GetGoing team has had several meetings with Eric Sinagra, the CEO of PathVu, about how our systems can be combined. The result is a vision of a system that can give users with impairments information for travel and access to all of the UPMC hospitals.
The GetGoing team will work with PathVu (see letter from Eric Sinagra) to access its information via API and appropriately transform it into the speech of the GetGoing agent. The resulting agent will retain its great qualities: slowing down the delivery of its speech, getting user attention before giving important information, giving time to write things down and giving information one piece at a time.
The GetGoing team will also need information about each individual hospital. We will get this from each UPMC hospital information desk. In this way, details on how to get to the correct entrance of the hospital as well as additional information as needed about getting inside will be included. For this, we will interview one or more persons from each hospital information desk and transform the information we get into a natural language generation dataset.
As GetGoing currently focuses on transport information in the Western Pennsylvania area, the team also proposes to work on providing more detailed information for each route by using public datasets newly released by the Western Pennsylvania Regional Data Center (WPRDC). This information includes: sidewalk density, bus average on-time performance and whether the bus stop has a shelter. With such additional information, seniors can better plan their trips and predict travel time.
Given that the data from PathVu is primarily targeted at giving visual information about potential obstructions in a users' path, we will developed techniques to present the information in a meaningful and memorable way that is needed when responding to calls by voice alone. Confirmation through dialog; allowing the system to adapt the level of detail, implicitly or explicitly; will be exploited to find the optimal ways to present the information about pedestrian restrictions.
We will also devise experimental techniques to work out the most effective ways of presenting the data independently from the live system. Thus, work on GetGoing will involve off-line experimentation as well as on-line experimentation (with the live system). We will seek IRB approval for any additional human subject experiments that we carry out (we already have IRB approval for the main live system).
We will test the new GetGoing with real users in part by giving out fliers at the information desks of all of the UPMC hospitals.
Timeline
From July to September 2020, the team will work on integration with PathVu. In parallel, we will get access information for UPMC Presbyterian Hospital.
From September to December 2020, the team will integrate information from WPRDC. It will also test the system resulting from the summer modifications.
From December to April 2021, the team will expand coverage to all of the ~30 UPMC hospitals. The team will then begin testing the system offline.
From April to June 2021, the team will test GetGoing with real users, with the phone number given out at UPMC hospital information desks.
Strategic Description / RD&T
Deployment Plan
Deployment will consist of advertising the phone number at all UPMC hospital information desks.
Expected Outcomes/Impacts
After giving out the phone number of the system, we expect to find that users will come back to use the system again (measured via callerid return calls) and that they will recommend the system to others (measured by the increase in number of calls over time). This measures the success of the project. Also, at the end of each call, the system asks the caller a few questions to determine whether they got what they asked for and if they would use the system themselves again and if they would recommend it to others.
Expected Outputs
TRID
Individuals Involved
| Email |
Name |
Affiliation |
Role |
Position |
| max@cs.cmu.edu |
Eskenazi, Maxine |
Carnegie Mellon LTI |
PI |
Faculty - Research/Systems |
| yulanf@andrew.cmu.edu |
Feng, Yulan |
Carnegir Mellon LTI |
Other |
Student - Masters |
Budget
Amount of UTC Funds Awarded
$100000.00
Total Project Budget (from all funding sources)
$100000.00
Documents
| Type |
Name |
Uploaded |
| Data Management Plan |
Mobility21_GetGoing_Data_Management_Plan_kYjp2e5.docx |
Dec. 26, 2019, 9:34 a.m. |
| Presentation |
324_-_Mobility21_GetGoing_slides_for_3-20.pptx |
March 19, 2020, 7:29 a.m. |
| Progress Report |
324_Progress_Report_2020-09-30 |
Sept. 29, 2020, 7:18 a.m. |
| Publication |
Towards Automatic Route Description Unification In Spoken Dialog Systems |
March 29, 2021, 10:02 a.m. |
| Presentation |
GetGoing and Better Mobility for All. |
March 29, 2021, 10:02 a.m. |
| Progress Report |
324_Progress_Report_2021-03-31 |
April 1, 2021, 4:36 a.m. |
| Final Report |
Final_Report_-_324.pdf |
July 28, 2021, 12:32 p.m. |
| Publication |
Incorporating rules into end-to-end dialog systems |
April 6, 2022, 5:22 a.m. |
| Publication |
Beyond turing: Intelligent agents centered on the user |
April 6, 2022, 5:22 a.m. |
| Publication |
Rethinking action spaces for reinforcement learning in end-to-end dialog agents with latent variable models |
April 6, 2022, 5:23 a.m. |
| Publication |
Pretraining methods for dialog context representation learning |
April 6, 2022, 5:24 a.m. |
| Publication |
Personalized Trip Planner for Seniors |
April 6, 2022, 5:24 a.m. |
| Publication |
Personalized Trip Planner for Seniors-GetGoing |
April 6, 2022, 5:25 a.m. |
| Publication |
Cmu getgoing: An understandable and memorable dialog system for seniors |
April 6, 2022, 5:26 a.m. |
| Publication |
A Survey of NLP-Related Crowdsourcing HITs: what works and what does not |
April 6, 2022, 5:26 a.m. |
Match Sources
No match sources!
Partners
