On University at Buffalo Week: Can we use AI to help give those with motor neuron diseases their voices back?
Rohini Srihari, professor of computer science and engineering at the School of Engineering and Applied Science, looks to do so.
Faculty Bios:
Rohini Srihari is a scientist, educator and entrepreneur.
A professor in the Department of Computer Science and Engineering at the University at Buffalo, Srihari’s research and teaching focuses on artificial intelligence for social good, conversational AI, deep learning in natural language processing and other fields.
Her work has been funded by the National Science Foundation, the Defense Advanced Research Projects Agency, the Intelligence Advanced Research Projects Activity and other agencies. She has worked extensively with the U.S. government in developing innovative multilingual text mining solutions.
Srihari’s recent research focuses on advancing the state-of-the-art in socialbots capable of engaging in empathetic, interesting and purposeful conversations. This also involves building trustworthy socialbots for combating disinformation, assisting the disabled and other purposes.
She has published over 100 research papers in computer science journals and conference proceedings. Her Google Scholar h-index is 40. She is also the author of two U.S. patents, one on multilingual text mining.
She has served as Chief Data Scientist at PeaceTech Lab, a non-profit incubated within the United States Institute of Peace in Washington, DC. She has also founded and directed technology start-ups, focusing on “big data” analytics solutions for various markets. Two of these companies were subsequently acquired by large media analytics companies.
Srihari received her B. Math degree from the University of Waterloo (Canada) and her PhD in Computer Science from the University at Buffalo.
Transcript:
People with motor neuron diseases such as ALS and cerebral palsy often lose the ability to speak. When this happens, they use modified keyboards or eyegaze tracking systems to communicate. But to speak, they rely upon augmentative and alternative communication devices. The late physicist Stephen Hawking is an example. Typically, people can generate about five words per minute with these devices due to their reduced motor abilities.
This limits them to simple exchanges such as: “I need a glass of water.” But what these people really want is to engage in meaningful conversation.
We can help with artificial intelligence, specifically an area of AI research known as personalization of large language models.
Large language models are the same technology powering ChatGPT and other popular chatbots. To personalize them, the chatbot needs to know a lot about the user. It needs to know their own experiences. It needs to understand their personality. It possibly needs to recognize the gestures that they use when communicating.
Here’s how it works. The user is talking to someone, the other person says something, the chatbot listens to what they say, and the speech is transcribed into text that’s fed into our chatbot. The chatbot then generates three possible responses for the user. The user either selects one of those responses as, “this is what I want to say,” or they can override that and say, “no, this isn’t what I meant.” So the goal is, with minimal input from the user, to be able to generate much longer responses, but also allowing the user to steer the conversation.
We’re already testing this technology with people whose communication is limited by motor neuron diseases. And we’re running the AI models through powerful supercomputers. We’re hopeful this technology will soon be ready for widespread use, improving the lives of countless people.
