Creating AI-Driven Avatars for Cannes Film Festival

             

Adam> I originally studied interaction design; much closer to traditional user interface design for apps. But I was always interested in the crazier end of the spectrum, what we might do with new technologies as they emerge. So wherever I could in my course, I learned 3D and physical computing and video game technologies. I ended up moving to London to explore emerging technologies for a CG company. Eight years and a few positions later doing more of the same, and here I am!

Adam> We’re a team of a lot of visual creatives constantly thinking about what’s new and fresh and future-facing. Some of the best brands come to us to develop visuals to showcase their offerings. I’m part of FutureResearch, where we’re constantly looking at new technologies and also engaging in a dialogue about the new forms of digital creativity that are emerging, and what we envision as next. 

As a team consisting of myself, Ben Black, Dan Hoopert, Svet Lapcheva, and Curtis Baigent, we developed Spirits as a way to begin a conversation about multi-platform generative avatars. We’ve procedurally created hundreds of unique creatures and brought them into a load of different digital contexts. It started as an internal research piece, which evolved to now encompass multiple projects across game engines, online metaverse platforms, and mixed reality. To bring them to markerless mocap and virtual production was a very natural next step.

Together with Unreal Engine, as well as disguise and Move.ai’s Invisible markerless real-time motion tracking technology, we were able to have Cannes attendees control the Spirits by simply walking around a stage. They didn’t have to wear trackers or a VR headset. All someone had to do was step onto the demo stage to control a digital twin of themselves as a Spirit character. 

Adam> To work with Spirits in a virtual production context meant a higher-fidelity environment than our previous real-time tests. We were able to turn the project around in a bit over a week as a team of four and I think that is the tip of the iceberg on what this means for individual creatives, being able to take their ideas and push them further. I hope that our work at Cannes will empower more creatives to see that they can get their big ideas across with leaner teams. It’s a great thing for any underdog creative or smaller agency to help get their individual voices heard. 

Adam> I once heard someone say something that has really stood out to me: “Anybody who is afraid of AI probably hasn’t used it much.” I think it’s true. AI tools have been around and in production pipelines for much longer than the last two years. At FutureDeluxe, for example, we’ve been using AI for upscaling for much longer. It’s a tool still capable of doing stupid things if you’re not using it right. But we have a lot of room to come up with really good combinations of computer intelligence and human intelligence to make bigger things with it. I’m a big fan of a shared intelligence future.

Adam> Designing for large physical spaces is always difficult. Anything you can’t readily see or experience at full scale demands some extra thinking and guesswork. That being said, what we designed ahead of time matched up very closely with what we expected to see in situ; I can’t say this is the case for all tools of this nature!

Adam> Absolutely. There’s something magical about interacting with something that isn’t totally there in the real world, but becomes real to us. For many people, using video game engine tools to trigger something by moving is a very familiar language. I think a lot of creatives are going to have really good ideas for it. 

We were even talking about how great it would be if we could create a Street Fighter setup, where someone could come onto a stage, push out their fist, and see a computer character make the exact same movement in real-time and have lighting bolts come out and attack the virtual character driven by another person. There’s a ton of room to take this type of experience to whole new levels as this is so interaction-ready.

Adam> For me, the highlight was seeing everything come together so fast, and being able to see something in very close to final quality from early on. It’s a huge help to be able to get a sense of how it’s going to feel early on and make decisions to improve it, rather than working on placeholder renders or assets and hope for the best. I’m really excited about real-time workflows and new tools that make it possible for the creatives to see their vision faster and really fine-tune their creations more than they otherwise would be able to.