Wed, 03 Feb 2016 17:05:29 GMT
When The Chemical Brothers’ latest promo landed last week, we were beguiled, entranced… and completely at a loss as to how Outsider directors Dom & Nic and The Mill London had pulled it off. We had to know. At first glance the video appears to be your run-of-the-mill ‘dancing promo’. That is until the dancer’s limbs morph, one-by-one, into what appear to be wireframes. LBB’s Addison Capper chatted with Dom & Nick and The Mill London’s Neil Davies, Dave Fleet, Suraj ‘Sid’ Harrington-Odedra, and Fergal Hendrick to find out how innovation and persistence paid off.
LBB> It’s been seven years since your last work with the Chemical Brothers - how was it working with them again?
Dom&Nic> The Chemical brothers always want to create an original film idea as a music video for their tracks and they are fully supportive through out the process whist giving the filmmakers freedom to create the best piece of work they can. They know if they like an idea early on at the treatment stage then they let us get on with it, it’s fantastic to have that trust in a working creative relationship.
LBB> What inspired the idea of the dancer’s limbs transforming into the webby material? What does it signify?
Dom&Nic> Everyone who watches it can take their own reading of the meaning; that’s the beauty of a music video film like this, it’s open to interpretation. It could be a feeling of losing the sense of your physical human body and losing your mind too. It could be lots of other things, artificial limbs, AI, robots or androids…
LBB> How does the creative idea fit in with the tune?
Dom&Nic> The song has a sense of losing something, it’s mournful but also very conscious of something happening right now, the lyrics feel like they are aware of the loss as it happens.
LBB> I actually found it quite eerie, but it doesn’t lay it on really thick. You have managed to achieve that really subtly. Would you agree with that?
Dom&Nic> Thanks, and yes. There is always an element of darkness in the videos we do for the Chemical brothers and it is here too hopefully. The idea of someone changing into a 3D printed form of themselves is quite an eerie one especially when you see it as a psychological imagined transformation.
LBB> The dancer almost looks like an art installation by the end of the film - did that play a role in the concept at all?
Dom&Nic> That wasn't intended specifically although the lattice structures we used as references have been used to make installation art pieces as well as artificial limbs. Her stillness at the end is part of the choreography, an emotional finish to the film.
LBB> I can’t recall ever really seeing anything like that effect before. Did you ever worry about feasibility?
Dom&Nic> The truth is if any of the people involved in the making of this film had known before we started how complicated, difficult and long the process would be it would never have been made.
LBB> How was the process of working with The Mill? What did they bring to the final production?
Dom&Nic> The Mill brought a huge amount of creative energy, technical support and development to the idea. All departments throughout the post production gave above and beyond all expectations, we owe them a huge debt of gratitude.
LBB> Why did you decide to set the film in quite an industrial warehouse?
Dom&Nic> We wanted it to be a film story about a woman having an inspired unique private moment, a magical experience but it takes place in the real world rather than as a professional dance routine on a stage for example.
LBB> Did you capture it in one take?
Questions answered by Neil Davies, ECD, Dave Fleet, Head of 3D & Co 3D Lead Artist, Suraj ‘Sid’ Harrington-Odedra, Co 3D Lead Artist and Fergal Hendrick, 2D Lead Artist, all of The Mill’s London Studio.
LBB> Tell us about what this opportunity meant to The Mill?
Neil Davies> The opportunity to work with Dom & Nic, The Chemical Brothers, Beck, Sonoya Mizuno and Wayne McGregor was a very exciting proposition. Not only was this a break-through project for The Mill, one of extreme technical complexity and precision, but it’s simply a real piece of art. As the viewer, you are lost in the dancer’s movement as the mechanical and physical blend into one. The final film is simple yet complex, and utterly mesmerising.
We are so proud of the final film and the amazing team at the Mill that has helped realise this unique piece of work and the response it has had already has exceeded all expectations. And the fact so many people have asked how we created it is I guess further testament to its appeal.
LBB> The nature of the wireframe limbs must create an extra challenge because you have to deal with the other side of the 3D object and background, and they react with her body. How did you manage that?
Dave> This was always going to be a big challenge. Not only would we need to paint out whole parts of her body to reveal a clean background, but also create clean parts of her own body for when her limbs occluded parts of herself (for instance when she puts her mesh hands in front of her real face).
Cleaning Sonoya out of the back plates, by hand painting each frame, would have been very time consuming, so we looked into ways of achieving this automatically.
Co Lead 3D Artist Sid Harrington-Odedra developed a custom tool that was able to automatically erase given sections of live action plates, as long as there was an accurate camera track and detailed 3D geometry to project onto. This clean plate pass ended up being a great starting point for the overall cleanup.
Fergal> Whilst Sid’s 3D clean up tool was incredibly useful in sections it still needed refining by the 2D team. The sheer volume of clean-up to do with removing her limbs from the background meant that any help with that was invaluable. However, as with everything on this job, there was no single technique that worked throughout the sequence so it was about using the right tool available for each section.
Any time saved removing her from the background could then be used in the very time consuming and meticulous task of removing her arms when they cross her t-shirt and face.
As she interacts with the t-shirt quite a lot and the shirt is free flowing this required a lot of 2D paint work, tracking, warping and rebuilding of shadows, and again there was no single technique that worked throughout, it was all about using the right tool for the right job.
Sid> There were also the moments where Sonoya’s ‘real’ limbs or t-shirt were masked behind mesh limbs. For these parts of the image, we rendered full CG elements of her arms, legs, head and clothing so they could be used as in areas where traditional 2D painting might be too difficult.
In addition to this, we even had to consider the fact that sunlight could travel through the mesh limb and then hit other areas of her body. In the original live action plate, the shadow of one leg onto another looked incorrect, as there now needed to be holes in said shadow. In order to achieve this, we create a light/shadow pass, which the compositors could use to grade up the live action shadow, to make it appear as if light had travelled through the mesh limb.
LBB> How did you actually achieve the effect? Is it motion capture or something else entirely? I can’t recall ever seeing anything like it before!
Dave> Lots of planning and lots of hard work! We looked into all sorts of methods to ‘acquire’ her performance.
4D scanning was ruled out pretty quickly as it couldn’t provide the accuracy and fidelity that we needed. Even though Sonoya is a great dancer, we obviously couldn't expect every take to be identical, which made a motion capture solution somewhat limited. However, we figured that if we were to use a portable motion-capture suit and get Sonoya to run through the choreography a few times on the day, then at least we’d have some animation to start with.
We enlisted the help of Ivo Slivkov from Timeaware, a company that specialise in remote motion capture. He flew in especially for the shoot to capture the dance. The mo-cap suit allowed Sonoya to perform her dance whilst we captured her movements in real time with sensors.
Even with Ivo’s help, the rest was good old-fashioned 3D animation, and a lot of it. We also paired with Peanut FX, a company who specialise in this kind of work. Peanut helped us mount 11 ‘witness’ GoPro cameras around the set to record Sonoya’s performance from every angle, the idea being to provide as much reference for the animation team as possible.
LBB> The transitions between the dancer’s limbs transforming to the web material are super slick. I love how the camera doesn’t even pan away / cut when it happens. How did you pull that off?
Dave> This is something that Dom & Nic had in mind from the start. They knew exactly when they wanted the transitions to take place and planned for it. For the most part, it comes down to well choreographed dance moves, coupled with great steadicam work. We shot open-gate on the Arri Alexa, instead of 16:9, to give us more pixels top and bottom. This allowed us to re-rack the frame within the crop afterwards.
LBB> I’ve been told by the crew at Outsider that you did incredibly innovative things to bring this to life. What did you learn along the way?
Dave> It’s difficult to explain without going into immense detail, but there’s a long list. We do have experience with many of the techniques we used, but we have never employed so many at the same time. The subtle combination of technologies needed to deliver Dom & Nic’s vision was groundbreaking for us. This job had a habit of testing everything we usually rely on, so a lot of what we learned was about how to extend our existing capabilities.
This project featured the longest shot we have ever tracked, new procedural modelling techniques, human body scanning, motion capture, an extremely complex rig, animated lighting, cloth simulation and a huge amount of clean-up work, which all made for an extremely complex blend of techniques.
LBB> What were the trickiest components and how did you overcome them?
Dave> By far the most labour intensive part of the project was the 3D tracking (matchmoving). The talented guys at Peanut FX and our team of animators at The Mill had to position each limb so that it exactly match Sonoya’s position on all 6,798 frames. This is one of the most complicated things that we do in VFX and is a highly intricate and skilled job. How did we overcome this? Patience, lots of patience!
LBB> How was the experience for you guys, personally?
Dave> This was an enormously exciting project to be involved with and like nothing I have done before. The opportunity to collaborate with The Chemical Brothers, Beck, Sonoya Mizuno and Wayne McGregor doesn't come along too often.
I was really proud to be a part of such a dedicated and creative team. In Dom & Nic’s original treatment they wrote “We see this film as an authentic art project and collaboration”. I couldn't think of a better way to describe it. Dom & Nic and their producer John were very much a part of our team at The Mill, their enthusiasm and excitement for what we were doing really pushed us to keep going.