senckađ
Group745
Group745
Group745
Group745
Group745
Group745
Trends and Insight in association withSynapse Virtual Production
Group745

Why AI Is Transforming VFX Craft and How We Judge It

06/06/2023
Post Production
Singapore, Singapore
241
Share
After serving on the visual effects jury at D&AD this year, Heckler’s Cody Amos chats to Laura Swinton about the new conversations bubbling up as artificial intelligence transforms craft, and what it might mean for the future of VFX
Beyond the flurry of excitement around the daily developments in artificial intelligence, specifically generative AI, there are artists and image makers trying to figure out exactly how this ever-evolving tech is going to change the way they work practically. Within the advertising landscape, those rolling up their sleeves and getting hands on can be found within the visual effects industry. Before the hype of the last year, they were already applying things like GAN and deepfake to filmmaking, using smart algorithms to simulate organic behaviours and substances, and starting to use AI in the nitty-gritty tasks of clean up and rotoscoping.

However, with this new leap forward, it feels like things are about to change substantially. And for Cody Amos, creative director at Heckler Singapore, big questions about what it really means for the craft of VFX really started to take shape when he judged at this year’s D&AD Awards. For one thing, the show introduced a subcategory ‘Use of AI’. But even in other subcategories like animation, AI work was kicking off debates about the necessity of the human hand in what we consider ‘craft’. 

LBB’s Laura Swinton caught up with Cody to find out how these conversations have made him rethink AI and how he sees it changing the way people work in the VFX industry.



LBB> This year you judged on the Visual Effects Jury at D&AD - how did you find the experience and what was the quality of work like overall?

Cody> The experience overall was great. So fun to see behind the scenes of such a prestigious award and to take part in the process was really interesting. I thought the work overall was amazing. There's so much good VFX work happening around the world that the threshold for what actually wins a pencil has become super high. 


LBB> This year, for the first time, the Visual Effects category included a sub-category called 'Use of AI' - what sort of work did you see in this category?

Cody> It was super interesting to have this category this year. It shows the industry is trying to categorise and understand this new technology. The work was mostly using image generators like Stable Diffusion to create or modify frames, which made some quite dreamlike films. There were also some entries using deepfake equivalents to help create digital face replacements. 


LBB> And what sort of debates or conversations did it inspire in the jury room?

Cody> It inspired a lot of conversation in our jury room, and in others too I believe. AI was definitely the hot topic this year. There were some questions around how much it counts as ‘high craft’ if there aren't humans involved to craft it. What counts as using it well vs what is simply using it as a gimmick? What is it good at right now, and where have artists used it in a great way to achieve something otherwise impossible? Overall I think in the creative community, everyone is at least a little nervous about where things are going. 


LBB> As we know, AI is developing very quickly - how do you think this category will evolve?

Cody> I think the category itself will vanish because it will instead be playing a big part in all of the other categories, CG, Compositing, Colour, everything. 


LBB> Equally, I believe there were a few AI-generated or assisted projects in other categories like animation - what sort of conversations did you have about the importance of the human hand in crafts like animation?

Cody> I heard the animation jury were questioning whether something even counts at all as animation if it's purely AI generated. Which is very valid. But fascinating because we're getting into defining things based on whether or not humans were involved, which has never had to be part of a definition for any craft before. It was always assumed. 


LBB> How do you think AI will change how we think about craft longer term? 

Cody> That's a big one. It looks like right now AI is being integrated into all the VFX tools we currently use. This hasn't quite happened yet, but it's happening. This will make creating complex work infinitely faster and easier. So I guess, the only way that will go is that the value of huge scale VFX will go down. I suppose this just continues a trend that has been happening since film VFX began. Things get easier and therefore more accessible. But this is going to accelerate that in a really exponential way. 


LBB> Within VFX specifically, there are a number of tools that use AI for some less obvious tasks like clean up or rotoscoping - how widely adopted are these tools right now?

Cody> That stuff is getting there. There are some good looking AI roto tools being developed. They are not commonly integrated into production yet. But it won't be too long, I think. The crazy thing is that image generators are helping artists, directors and everyone create style frames to visualise things super quickly. I would have assumed that something like roto would get done by AI before full image generation was possible - but nope! Shows what I know. 


LBB> For now when it comes to AI-generated images and animation there seems to be a very specific AI aesthetic - a bit still and shiny or a warping, weird nightmare. What do you make of that look and will we get to the point where it's undetectable?

Cody> There’s a certain aesthetic that’s been created by AI. I actually think aesthetics being created by technology is a really interesting aspect of where aesthetics in general come from. I always like seeing a new one appear. There were a few entries this year that drew on that aesthetic. Interestingly, they weren’t massively popular. When it works really well though, is when it’s being used to create a painterly effect. Which was done in Coca-Cola’s ‘Masterpiece’ spot, diving in and out of paintings. That dreamlike warping is sort of what you imagine a painting would look like if it was in motion, so it works really well there. 

Will we get to the point where AI is undetectable? Yes. And terrifyingly soon. 


LBB> Is there anything in the VFX pipeline that you think AI will never be able to do?

Cody> So far, it feels like we will always want artists to be driving the overall creative. In the immediate future, I think we will still have 3D artists and compositors etc, but they will be driving software that has been massively superpowered by AI helping with every task. 

I suppose, in theory, you could end up with AI driving those programs alone, and writing and directing films alone. What that would look like, I don’t know. Whether there would be any appetite for that, is a big question. AI can already write poetry. But it’s just mimicking real poets. So, is there any value in AI poetry? Do we want to know that the poet or the screenwriter or the director has really felt the things being portrayed? Can an AI really look at the world and say, “These are the stories that have not yet been told, or not in this way, here is my story?” I don’t know. Maybe not. Which is a roundabout way of saying that I think maybe storytelling is for humans. And not because the AI can’t do it, but because maybe we don’t want it to. 


LBB> And how are teams adopting and adapting pipelines as AI tools rapidly develop?

Cody> Mostly by freaking out and going, “Oh wow, look what this can do now.” Actually, I think most places are adopting things slowly and cautiously. In production, we need control over everything. So a tool that gives you an image instantly is fun for some things, but if I can’t tweak every aspect of it, it’s not that useful yet. That’s why tools like Firefly are interesting, because they are starting to integrate AI into usable software for artists. 


LBB> Outside of the VFX jury, what other sort of presence did the AI conversation have at the festival?

Cody> Adobe had a Firefly stall at the festival, showing off their new tools and where they are heading. A considerable amount of the talks were also about AI, with various views on where it’s going, and how to ride the wave, and why or why not to be concerned. Interestingly most of the talks were very positive and there was definitely a sense that artists will always be in control of the arts. I hope they’re right!


LBB> What were the other big trends that you saw at the festival?

Cody> I think the other big topic of discussion was diversity in representation in the creative industry. There was a great presence of D&AD Shift alumni, which is an awesome initiative running night schools all over the world, targeting people who maybe don’t have as much access to traditional ways into the industry. It was inspiring to see the interest from people wanting to get involved. And I suppose it reminded me that there is such an incredible amount of talent in the world, and people who haven’t yet told their stories, there are so many new ways of seeing things still, there’s just so much potential for creativity in the world still. So here’s hoping that AI will simply break down many of the barriers to creating good work, and we will see a massive flourishing of all sorts of arts in the next few years. 
Credits
More News from Heckler Singapore
Awards and Events
Rooftop Drinks with the Hecklers
28/02/2024
22
0
ALL THEIR NEWS
Work from Heckler Singapore
Singapore Rooftop Party
Heckler
17/03/2024
8
0
Singapore Rooftop Party
Heckler
17/03/2024
1
0
Director's Cut BTS
SK-II and Maison Kitsune
23/02/2024
10
0
ALL THEIR WORK