Brent Shiver (CDM MS ’04, PhD ’13), an advisory software engineer for IBM Accessibility, was profiled in the summer issue of DePaul Magazine. When Shiver was a student at DePaul, he helped with an animation project designed to facilitate communication between hearing and deaf populations. Learn more about the project below.

Many people can relate to the frustration of trying to communicate with someone who doesn’t speak the same language. Now imagine facing that problem every day as a person who mainly uses American Sign Language (ASL) to communicate with the hearing world.

A group of DePaul professors and students have undertaken a research project to help bridge this gap. The DePaul ASL Project combines computer technology and linguistics research to create an animated English-to-ASL translator as a way to better connect deaf and hearing populations.

The project began in 1998 when a deaf student approached School of Computing Professor Rosalee Wolfe hoping to develop better accessibility for deaf people, perhaps by hooking up speech recognition technology to video. “ASL is a fully developed linguistic system, and it’s completely different from spoken English. In terms of how the grammar works, it has parts of speech that we’ve never heard of in English,” Wolfe says. “I realized what would work better would be computer animation because it’s much more fluid and you could change its form more easily. Once you shoot video, it’s frozen for all time.”

Wolfe also realized that animating a figure signing ASL would be preferable to animating a figure speaking English and asking viewers to lip read. “So much of English is ambiguous when you look at the face. We have 44-45 phonemes [distinct units of sound that distinguish one word from another], but if you look at animation techniques, animators use between nine and 11 mouth shapes to represent those phonemes—so, for example, mat, bat and pat all look the same,” says Wolfe, who has been learning ASL through Columbia College Chicago’s interpreter training program.

The project has grown over 17 years, and the current online demo shows how the animated signer portrays different parts of the language; for example, the avatar moves her eyes or mouth differently to show “big” or “small” and points certain ways to indicate “he said” versus “she said.”

Associate Professor John McDonald notes that students have worked on a wide range of different aspects of the project, from math and programming to more artistic modeling of the character to studying how to design computer interfaces. “The project is quite amazing in terms of the breadth of different techniques that are required to attack the problem,” says McDonald.

One student who was involved with the project was Brent Shiver (CDM MS ’04, PhD ’13), who now works as an advisory software engineer for IBM Accessibility in Austin. Shiver, who became deaf at age two-and-a-half as a result of a bout of spinal meningitis, had experience with ASL and was willing to share his knowledge with hearing students and professors.

“We have to talk to students about the challenges that the deaf have in communicating and interacting with hearing people on a daily basis, and we also teach them about the importance of ASL as a part of deaf culture,” McDonald explains. “In that respect, Brent was an invaluable resource while he was here. He was incredibly generous with his time in terms of both helping other students learn ASL and helping them understand how to approach parts of it in order to animate it.”

The project doesn’t just aim to create a passable facsimile of sign language—the goal is to create a lifelike animation. “When we started many years ago, the first reaction [from ASL speakers] was, ‘Well, I can understand it, but it’s very robotic,’” McDonald says. “Over the years we’ve been working very hard to incorporate more natural motion into the signer.”

In addition to classroom work, students collaborate with faculty members on research and are involved at all levels in presenting their work. McDonald, Wolfe and PhD students presented at a workshop on sign language avatars in Paris in April and will be presenting in Los Angeles in August. The PhD students are also assisting with a presentation that faculty will be giving at the Theoretical Issues in Sign Language Research conference in Melbourne, Australia, in January 2016.

“We could not have accomplished this work at any other university,” Wolfe says. “DePaul has a unique environment in that there’s such deep-rooted set of values regarding the dignity of the individual and equal access for all, and I believe that’s what has made this work possible.”

Interested in learning more about the ASL Project? Contact Rosalee Wolfe at wolfe@cs.depaul.edu.