Matthew Manhire and Canadian production company Feels Like Home originally set out to make this Tesla self-driving car commercial with a driver in a green suit, which would have been removed in post. But then came an opportunity to partner with a virtual production company - The Other End -and capture the entire thing without ever leaving the studio. While along the way, the crew - with members from Feels Like Home, Alter Ego Post, Arri, Unreal Engine and The Other End - executed a number of things that, to their knowledge, had never been done in Canada before.
To delve into the processes involved in making a film in virtual production and the kind of problem solving required along the way, LBB’s Addison Capper chatted with Matthew.
LBB> Tell me about the foundations of this project - how did it come about? And when did virtual production begin to seem like a good approach?
Matthew> The inspiration for this project, funnily enough, occurred while watching segments from the most recent World Economic Forum. I couldn’t help but be intrigued by the developments in AI driving tech. Pairing that with my constant drive to try things nobody has done before, I set out to make a self-driving car spot. The prelim talks were to put a driver in a green suit, have him be the driver and then remove him in post. Then the opportunity to partner with a virtual production company presented itself and the development went rapidly in that direction. We were thrust into a situation where we could now have a self-driving car spot without ever leaving a studio and that allowed us to focus on the back story a bit and play with the redirect of when the viewer discovers the car was driving itself the entire time. The logistics and safety issues involved with having a guy in a green suit driving in an ideal location at sunset would have been insane, so shooting in a controlled environment was the only way we could have pulled this off.
LBB> This is your first shoot in virtual production - talk us through the process! How was it different from a regular live action shoot?
Matthew> Up until this project I had really focused on my work being as authentic and as real as possible. I think that's why working with virtual environments became so attractive to me. How can I still search for tiny moments of humanity in a working situation where there were so many perceived perimeters? I’m at a point in my career where I want to be challenged and I want to work with people that get excited about trying new things that are outside their comfort zone.
In terms of comparing a ‘typical shoot’ to a virtual production, it’s hard to say. Every project is so unique in its challenges and this one provided opportunities for my entire team to step up and walk away as smarter and more confident filmmakers. What really stood out as unique right off the bat was how much I could get a visual sense of how things were going to go. Because we had to pick focal lengths so early on in the process, so that the frustum (the area on the AR wall where the camera is pointed and displaying the visuals at 100% quality and exhibiting the interactive parallax effect) would be calibrated to the Unreal Engine environment and the directions we would be shooting. The shot list was something that we all could work from much earlier on in the process than we would while doing a commercial now.
LBB> You’ve touched on this there, but I imagine the pre-production process is quite gruelling because things need to be decided on earlier in the process. It's almost like doing the post first! What are your thoughts on that? How was the pre-pro?
Matthew> We spent seven months on Zoom calls from ideation to execution, finding the right collaborators along the way and properly on-ramping them as the need for their department became more necessary. Working with Unreal Engine designers and operators is like learning a new language so I wanted to make sure we were all consistently communicating. They are the first department that we sat down with so they could start the architecture of the environment. My main focus was becoming familiar with their terminology and creating as much bond between all the department leads as possible. I knew we would face a lot of ups and downs, seeing as how we were doing something for the first time, that had no instruction manual and also pushing the limits of the technology. You’re totally right; it is like doing post first. So a lot of approvals, many meetings of being ‘location scouted’ through the interactive environments and making big decisions very early on, in terms of locations and shots. We’re working in a circumstance where we literally can move mountains and place the sun where we want it and with that comes an incredible amount of big picture thinking and also leaving lots of mental bandwidth for the details.
This particular aspect is where we as production professionals need to guide our clients, as this tech becomes more and more efficient and viable as an option. Agencies becoming involved a few weeks earlier in the schedule to craft the environment and tune it to the brand's needs [is beneficial]. It’s intensive but it creates a much firmer plan and removes any possible storytelling gaps before we even get to set.
LBB> The actual set you built sounds like a pretty big feat. Tell us about that.
Matthew> There were two main set challenges that we had to focus on. Since we were working on quite a scaled down version of a volume wall, making the car feel like it was turning and moving through the environment meant that we had to get a custom built motorised turntable for the car to be placed on. This made longer moves a lot more doable. The second feat was planning out what aspects of the underground parking garage were going to be real set pieces so that we could properly blend the real world elements with the digital environment.
We ended up having three concrete pillars along with some curbs and railings that we could move and adjust to achieve a proper perspective match. Thankfully it's becoming more common to see AR walls have more permanent setups in studios so knowing that they are set up and ready to go is helpful. The biggest feat of all though is designing and constructing the content that is uploaded to the LED wall itself. That’s where a lot of the mental bandwidth gets used up.
LBB> What did virtual production allow you to do more easily than if you shot something in the old school way?
Matthew> What made shooting in a virtual approach more easy than conventional was that my entire team knew what the world we were shooting in looked like long before we set foot in a studio. In green screen studios there's a lot of… green. Where in a virtual environment we can all see where we are and what’s happening. The actors have something to engage with and we can all see in real time what the shot looks like, rather than a green screen shoot where you're gathering pieces of the puzzle and assembling it in post. On top of that we were able to film on a bridge in California and a city street in Chicago, without even turning the car on. We opted to add in two wide shots with a CGI car inserted to fill out the story but as the tech becomes more reasonably priced and we become more experienced I think we’ll be able to do 100% of our client’s projects on the wall.
LBB> This was the first time you did something like this - what kind of problem solving did you have to do along the way? What were the trickiest components and how did you overcome them?
Matthew> Thankfully for me and my personality type, production is an endless journey of problem solving. A large part of the intention with this project was to push the technology to its limits and educate ourselves in a trial by fire way. The idea of connecting the large turntable to the camera and the screen so that everything moved in unison was an idea that we had no proof of success rate until we tried it. A shiny car turned out to be a very distracting object for the dozens of sensors the wall uses to communicate with the camera.
Thankfully we had some of the most talented people on our team that managed to smooth out technical interruptions by calibrating a record player with a custom sensor attached to it, to the speed of the car turntable. That gave the central hub the information it needed so all the capture components could all communicate clearly, without reflections from the car interrupting the sensors!
The risks of failure are what make stuff like this exciting to me and made for a perfect scenario for learning. We wanted to rub up against failures so we could solve them, that was always part of the pre-production conversations.
LBB> What was the actual shoot like?
Matthew> The actual shoot was the fun part! For me, being on set regardless of how intense it is, is an incredible feeling. That's the part where you materialise the project. There’s the saying that's something like ‘you make a film three times; in pre-production, on set and in post’. Having an entire team of video game designers and learning about the personality types in that industry was entirely new for me so making sure I was expanding my POV and including those roles in the production conversations was where my learning curve really went into overdrive. Thankfully Marni (Luftspring, founder of Feels Like Home) brings such an immense amount of production experience with her wherever she goes and was there to keep a perfect balance of seeing the project through and leaving enough room for all us all to learn from the experience so we can take that into our next virtual endeavour.
LBB> What are your thoughts on virtual production and its potential influence on the industry?
Matthew> I think this is a really exciting time to be a storyteller! Tools like this create opportunities to create work that feels bigger and in turn leaves a larger impression on the viewer. By no means are volume walls a silver bullet that are going to solve every production problem but if you want to create something impossible then this is the tool for the job. I think in the near future I will be walking clients through location scouts as they sit at their desk with Oculus goggles on. I’ll be able to show them time of day options, preliminary colour passes, move undesirable or distracting objects in real time. My vision is to be able to walk clients through my vision as clearly as Leo walks Elliot Page through the city in the movie Inception. Communicating a vision in a crystal clear way that gives the client the security they need to get projects made properly. For me, this is the direction I want to go and I want to help make Toronto a hub for reliable and groundbreaking virtual production.
LBB> Any parting thoughts?
Matthew> If there’s anything I've seen with my own eyes through this experience it's that this technology is ready and waiting for us to come and create. This is a great time to collaborate using these tools and become an early adopter of what will surely be a really disruptive approach to the advertising landscape. I’m really excited by what I’m seeing and experiencing and just want to share it with other talented creatives! It’s wild to think that what’s happening right now is just the early phase of what could be endless advances in streamlining content creation for brands of all sizes.