Project Revoice uses artificial intelligence to recreate the voices of people who have lost theirs to ALS - LBB's Laura Swinton learns how
Cast your mind back to the summer of 2014. That was the year that the Ice Bucket Challenge engulfed the world. Even if you didn’t find yourself drenched head to foot in ice-cold water (bah humbug), chances are you saw videos of people who did answer the call.
The challenge was initiated in the USA by a group of friends to raise money for research into ALS (also known as motor neurone disease). Two of the friends were themselves diagnosed with the disease and one of them, Pat Quinn, went on to lose his voice. In a tragic turn of fate, Pat, who became a voice for ALS sufferers found himself voiceless.
But thanks to an ingenious project that uses AI to recreate a person’s voice, Pat has regained the ability to speak like himself.
Project Revoice is the brainchild of Australian advertising agency BWM Dentsu, who collaborated with voice cloning technology firm Lyrebird, innovation and production company Finch and sound studio Rumble.
As executive creative director Asheen Naidu explains, the project came about as the creative team were looking to explore the potential of voice technology.
“We’re an ad agency and we’re always looking at the latest creative tech, just because we’re interested in it. Voice has been the new frontier in tech, everything from Alexa to Google Home. But what we didn’t see was voice tech being used for anything beyond commercial purposes. We’re very against tech for tech’s sake, so we started thinking about what we could do with voice that would be ‘for good’ or for a social purpose,” says Ash.
The team started speaking to the ALS Association about how they could use the technology to help sufferers who had lost their voice. Pat, who has been such a prominent figurehead and activist, was the perfect person to work with and help launch the project. The plan was to create a free-to-use and democratic platform on which anyone with ALS could ‘save’ and then generate their voice.
But first they had to figure out how to make it happen. The team reached out to all sorts of tech companies, but in the end it was Canadian voice-cloning startup Lyrebird who proved to be the ideal partners. Although the company has pioneered AI voice cloning, it’s also a company with a strong set of ethics. It’s highly attuned to the potential misuse of the technology and has already demonstrated this by creating voice clones of Barack Obama and Donald Trump. Therefore the social cause behind Project Revoice chimed with their own goals and perspectives.
René Schultz, a self-professed tech connoisseur and one half of the senior creative team who worked on the project (the other half being Oskar Westerdal), says that Lyrebird proved to be a natural fit. “I think that’s the good thing about Lyrebird, their morals are very strong and they want to use it [the tech] for good,” he says. “They seemed like a natural fit and they gave a damn. It was really, really fortuitous that we found Lyrebird because here was this company on the cutting edge of voice tech who just happened to be really nice guys. They just want to do good and make a difference.”
The next step was to find recordings of Pat’s voice that could be fed into the Lyrebird programme. As Pat didn’t have a pre-saved voice bank, the team had to use footage of Pat giving talks and interviews. That’s where the team at Rumble Studios came in. The painstakingly cleaned and levelled out the audio so that it was as clear and near to perfect as possible.
The Lyrebird tech works in a smart way, learning about what makes someone’s voice unique – that’s where it differs from traditional text-to-speech technology, which simply stitches together pre-recorded phrases and syllables and sounds robotic.
Once the recordings were cleaned and fed into the technology, the next step was to create an interface that would allow Pat to use his voice.
“I think the way that Lyrebird phrases it is that it analyses the DNA of your voice and recreates it. Once that’s in we worked with another great company here called Finch. They have an innovation arm called Nakatomi. What they did is they took that and they integrated it with Pat’s eye tracking system, so when he’s using that eye tracking system then his voice would come through,” explains Ash. “We were lucky on this job because we had a lot of partners. Finch filmed the whole content pieces for us and now they are helping us maintain the website and uploading stuff. Everyone who has been involved has been really doing so out of the goodness of their own hearts.”
With the voice cloned and the platform built, some of the BWM team flew to New York to meet Pat. It was the moment of truth as Pat would try out his voice in front of his friends and family – and the team was understandably nervous.
“We went all the way to New York to meet him and before we met him he hadn’t actually heard his new-old voice yet, so we were very anxious to play it for him,” says Rene. “Luckily as soon as we shared the voice he was very moved and I think everyone in the room was also very moved at that moment. I think for Pat that week that we were there has become life-changing for him. For everybody else it was a very meaningful piece of work; to be able to give someone a piece of their humanity back.”
One detail in particular hit Rene hard and resonated with him. “I was in the room when Pat was speaking to his family for the first time and I was observing his Dad and that sense of recognition that washed over his Dad’s face was pretty awesome. He could actually recognise his son’s voice. That moment will probably stay with me forever.”
Now that Project Revoice has launched – and has been picked up by pretty much every news outlet going – the next step is to roll out Project Revoice for everyone at risk of losing their voice. In a few weeks people will be able to record their voice at projectrevoice.org.
And while the project is bound to have a profound impact on anyone at risk of losing their voice – a key part of our identity – the project has also had a profound impact within the agency.
“It has inspired me to think about what’s possible,” says Ash. “When the idea first came up we were like this is amazing – and how the hell you do it? Now we can do this it makes everything feel possible.”