Digital voice assistants are nearly ubiquitous in modern homes and offices, but they can have hidden vulnerabilities for hackers. Computer science professor Doug Szajda and his students are shining a light on how to make these digital audio systems more secure.

One of Szajda’s primary areas of focus is researching how these voice-activated systems can be easily manipulated. This artificial intelligence focus, known as adversarial machine learning, has potentially high stakes.

In one example Szajda shared, scammers could theoretically send an audio command to your smart phone, have it dial a 900 number, and suppress the dial.

“You’re standing in an elevator and what sounds like elevator music is telling your phone to dial a number that’s going to charge you gobs of money each minute,” he said.

Szajda pointed out that attempts to trick commercially available smart voice systems are still mainly proof-of-concept attacks. But computer scientists concentrate on this area because the AI systems are so new, and the devices that send and receive audio signals are becoming more prevalent.

Over the past several years, his students learned the mathematics and the theory behind machine learning systems before they begin to run experiments. Szajda said that students in his security research seminar run many of the experiments now.

Their database contains voice samples where a speaker repeats the same phrase 20 times. Speakers of different genders from regions around the country with various dialects provide the samples. From there, students run experiments, generate data, and analyze the output.

André Shannon, a 2022 grad, credited his coursework with Szajda, including a group project over the summer in 2021, with helping him decide to pursue a Ph.D. in computer science.

The summer project involved taking a new method for machine learning systems originally created to use for images and making it work with a voice processing system, which hadn’t been attempted before.

“It was very exciting,” he said. “Working on a specific in-depth project with other students and professors was a lot of fun.”

Senior Liz Smith, a computer science major and math minor, writes code and helps configure environments for experiments.

“Working on this research has really opened my eyes to how little anyone understands the choices very intricate AI make — and not just when it comes to voice processing systems,” she said.

Smith said the machine learning coursework and research made a difference for her academically and professionally. She has a software engineering position with Capital One lined up after graduation.

“The bottom line is, despite the fact that every time you put on a TV, you see ‘the latest in AI,’ ‘the latest machine learning,’” Szajda said, “we still have a long way to go to really understand how these systems work.”