blonde woman's faces

UR-developed software solves visual media mysteries

September 26, 2022


Researchers digging into massive digitized visual collections that lack tags and other metadata frequently face a tedious slog. Who is that in black-and-white news footage? How often does this TV character appear solo onscreen over an entire series? Where are the illuminations in these Medieval manuscripts?

University of Richmond professors Lauren Tilton and Taylor Arnold found a way to cut to the chase. The free open-source software they’re developing in the Distant Viewing Lab applies computational analysis to film, television, photography, artwork, and other media.

“We’re trying to make it more accessible and easier to use computer vision algorithms if you have a research question,” said Tilton, who teaches digital humanities.

Many algorithms are either proprietary or designed for purposes like policing, self-driving cars, and surveillance, she pointed out. And the ones that are available for scholarly research tend to be difficult to use if you don’t have a computer science background.

The Distant Viewing Toolkit, created with a 2018 National Endowment for the Humanities grant, allows users to quickly identify and analyze specific objects, people, and audio. Last spring the software project received a $485,000 grant from the Mellon Foundation to extend the technology.

Tilton called the funding transformative. “Mellon provided us with the opportunity to make the toolkit look more like software,” she said. “And we have a wonderful post-doc named Justin Wigard we hired who’s helping us with that.”

Arnold, an associate professor of statistics, explained that the toolkit can add new dimensions to humanities scholarship. Take the classic sitcoms “Bewitched” and “I Dream of Jeannie,” shows long considered copycats. The toolkit reviewed thousands of hours for factors like which character appears in each shot, leading to analysis that revealed wide differences between the shows.

“‘Bewitched’ is really a show about Samantha. Everything is anchored on her. Something like 90% of the shows start with her as the first character you see,” Arnold said. “Whereas ‘I Dream of Jeannie’ is a show about Capt. Tony Nelson. Jeannie sets off the plot, disappears — literally or figuratively — and then comes back at the end.”

Next, the team plans to make the software into a downloadable computer application.

Digital humanities projects like this are putting the University of Richmond on the map.

Incoming students told Tilton that this interdisciplinary field drew them to Richmond. “UR has earned a national and international reputation as one of the schools for digital humanities,” she said.