Your Shot: Salomon Ligthelm's Modern Day Garden of Eden Story for AYIA

Salomon> I initially stumbled on a track by AYIA called 'Ruins' via Spotify's Discover Weekly playlist (which is where I always find new good music by the way), and was immediately impressed by the sound - it felt like Bjork meets Radiohead, as if it could have been part of the Girl With The Dragon Tattoo soundtrack.

I noticed they didn't have a music video for it yet, so I immediately reached out to the band's management/label and asked whether I could do a video for the track. They said that it would be awesome to have something for it, and I started putting my thoughts together for an idea, while we were trying to figure out a schedule. Not much longer after that I ended up having a shoot in Iceland - which is where the band is from - so I decided to reach out to them while I was there to see if we could meet. 

They ended up having an album launch during one of our nights off, and we went to go see them live. They played 'Easy' as part of their set, and I fell instantly in love with it. I spoke to the manager after the show and told him how much I loved that track. I went back to NYC and wrote a treatment for the idea and sent it to the manager asking him whether I could shift focus and write an idea for 'Easy' instead, but he said that there was already another director working on it.

I decided to keep working on Ruins but kept checking in on Easy and how it was shaping up with the other director. A couple of months passed, so I decided to reach out again regarding the status of it and the manager told me the other director couldn't make it work, so I checked whether he'd be happy for me to take it over. He agreed and we pulled the trigger on it.

Salomon> Someone wrote a comment and described it like this (in jest of course) - "A trans-dimensional evil domino remixes a boy-band audition tape beyond recognition in an attempt to frame them as undercover assassins."

I think it's really important for people to be able to dissect a piece like this for themselves, but for me it was about taking the narrative outlines of the Garden of Eden story - the man and the woman being told not to take from the Tree of the Knowledge of Good and Evil, and extending it metaphorically or allegorically into the now. Why were they told not to eat the forbidden fruit - and why was that tempting icon so alluring? Perhaps because it was the promise of connection and transcendence. They could be ‘like God’. This idea of the ‘forbidden fruit’ has a modern manifestation: our devices, our ‘Apples’, are portals to everything, all knowledge... and yes that is alluring but it's also potentially dangerous. The only power that we have over these devices is our ability to exercise self-control.

Salomon> The song was the starting point. The mood of the track is obviously very dark and compelling but the lyrics 'easy does it, the water ripples if you stir it' felt like they were telling a cautionary tale, much like that of the old Biblical one and what we're hearing from social scientists now on the trappings of social media, etc., and its effect on our behaviour and mental health.

Salomon> I think we're all trying to find a way to relate to technology (social media, our devices, the internet) in a more balanced and responsible way. I never want to point a finger because I'm personally dealing with the same issues and struggles. I think the use of allegory and metaphor allow the audience into the narrative and is inviting them to ask questions for themselves, without feeling like the director/author is pointing a finger at them, or guilt tripping them.

Salomon> This thing ran more like a personal project than it did a commissioned piece of work, and so there was a lot of experimentation and exploring happening on set. We didn't have a set storyboard, we just riffed off a shot list that had some very detailed shot ideas, and then some very loose scene outlines. It's very important to me that when I create in that ‘personal project’ realm that I leave a lot of space for finding things - improvisation and experimentation.

I don't have a particular favourite moment but the DP, Shabier Kirchner and I spent a lot of time chatting about the film and its themes on the ride back from NJ to NYC. Those memories of working with friends is priceless.

Salomon> We shot in a manor in New Jersey and then at Fort Lee National Park just across the George Washington Bridge. I wanted to shoot in a place that somehow felt removed from the real world. This was to feel more like a fable or a parable, and so finding a location and a talent that felt more ‘idealized’ in a way was what I was after. It was important to me that the person who had the relationship with this monolith was a product of social media culture. I wasn't going for something real or completely authentic, I was going for someone who felt like they lived out their lives on their devices and so finding someone who had a healthy social media following and who had that idealised ‘social influencer’ aura would help ground the meta-narrative a bit more. Again, all of this isn't supposed to be clear or explicit, but I needed a guiding rule that I could make all decisions by, and this decision was congruent with the whole thematic thrust of it.

Salomon> I worked with Artjail quite early on trying to figure out the right visual style for the post elements. It was always very important to me that this film would feel ‘organic’.

We throw that word around so much these days but I truly wanted everything to feel tactile and textured. I shoot most of my projects on 16mm or 35mm film, and this project was to be no different. So we were always considering how the VFX elements would integrate with a filmic pipeline.

Again, the concept is driven by a monolith - something that is VERY tactile and textured - I wasn't creating a robot or something cold and sterile (and digital). This film was to be conceptually, visually, in post, and in sound design driven by something that was organic, so all my decision-making was driven by making the audience feel that. Kubrick's ‘2001’ is an obvious reference and I love how he was able to ground the futurist ideas with things that felt so tactile - the apes, the manor - it held the future and the past in such beautiful tension. I was also inspired by Gomorrah and La Haine, and so I was trying to hold the sci-fi elements in tension with visual ideas that felt a bit more urban and part of the contemporary social lexicon.

Artjail> Salomon approached us in a very early stage of the project, he was still working on his treatment at that point. Seeing what he wanted to do, we immediately knew that this would be a unique creative job and we wanted to be part of it. 

Artjail> The overall theme of the project from Salomon's treatment: "The film is ultimately an abstract representation exploring our relationship with technology - especially with our screens. The monolith represents a hand-device (phone) in the vertical and desktops or televisions in the horizontal. Similar to communication technology, the being at first, tries to ‘connect’ with us, but ultimately it has the power to destroy us."

Our main creative tasks were to create two main perspectives, and create the monolith for all floating shots:

1. Create a visual language that the monolith uses to communicate with the human world - which is the glowing light that we created and put beneath the subsurface of the monolith itself. 

2. Create a visual language for how the monolith views the world - which is the abstract view of the glitching figures and dancers that you see in the film.

3. Bring our floating CG monolith into the world with an animation style that felt heavy and grounded - we worked with Salomon to create a CG replica of their practical monolith and create scenes, imagery, and subtle animation that brought weight to the object

In our concept design stage we juggled ideas with Salomon to define the creative approach for the monolith communication and the monolith POV. This was not only about the actual look but also about the philosophy behind the idea of technology/AI trying to connect with us. We explored ways the monolith could learn more about the world it landed in, referencing things like autonomous driving, GANs (https://en.wikipedia.org/wiki/Generative_adversarial_network), LiDAR scanning, etc. 

This ultimately helped us find starting points for the look we wanted to create. For example, the monolith communicating with the world through that subsurface light starts in very broad and undefined patterns and during the course of the video it is becoming more elaborate and defined while trying to communicate. This is visible not only in more complex shapes but also in a higher frequency pulse in the light.

Similarly, we wanted the monolith POV to have an abstract digital resemblance. There are a lot of glitches in the object he is recognising and the data is sparse and mostly focused around the important thing - the humans in the scene. There are no real world colours until the very end. Everything is in grey and gold tones, the gold accents meaning to connect the viewer to the gold in the monolith communication light. 

Artjail> Salomon already had a good idea of how he was picturing the monolith, we discussed it's look and feel with him to find the right visual solution. The simplicity of that rectangular shape was a design challenge, finding the right balance of edge and surface detail and material variation was the biggest challenge for us. Since the shape was defined as a rectangle we focused on the look and feel of the material the monolith was made of.

For us it was important that the monolith felt organic, it shouldn't feel like present day technology or a future/alien tech. We explored things like obsidian and crystalline structures as a material for the monolith. The goal was to find a sleek, dark material that would look organic and interesting enough on that simple rectangular shape.

At the end we did go with a surface resembling slate with a slight crystalline addition. This material looked best in the pure form of the monolith and also was helpful to bring out the light from inside when it started to communicate.

Artjail> The monolith was challenging because it is such a simple shape - a simple black rectangle. We did spend a lot of time in lookdev to make sure the monolith would have enough organic detail to make him look interesting even in some of the very overcast light situations we had in the music video.

The animation had to confine a certain amount of gravitas to make the monolith look alien and even a bit foreboding and also express a certain weight when it was flying. It was tricky to find the right speed for it to fly and descend in each of the scenes, it usually took us a few animation iterations to get it right and most often less was more.

While discussing ways to bring Salomon's monolith to life for the music video in very early stages, we decided to create a practical build for the monolith to stand on the ground during production (we would extend its height in all of those shots) to help reduce the amount of CG monoliths we had to create in post. The monolith created by the art department looked fantastic and gave us a lot of good references regarding texture and light interaction.

Artjail> The main creative challenge of the video was finding the visual language for the monolith POV. We were given some general reference that Salomon had in mind, but his vision was something largely unexplored. We experimented with photogrammetry and Point Clouds out of the Nuke camera tracker based on the visual style we wanted to explore and our early discussions with Salomon about the look and feel.

Ultimately we decided to capture volumetric data using a Kinect camera and a software called Depthkit to be able to capture rough point cloud data for live action footage. That allowed us to capture the same action that our actors would perform in front of the camera simultaneously as a volumetric video. While testing the Depthkit approach we realised that we would be able to get some natural glitching while capturing our actors with that method. We even experimented with things like additional glass panels, or plastic foil in front of the sensor to get even more ‘organic’ glitching.

At first we had to go through about  half-an-hour’s worth of point cloud footage and select the best moments to use in the music video. We spent several days experimenting on the look of that world, going back and forth with Salomon. Even though these shots are meant to be a digital representation of the humans as seen from the monolith, it was important to create a tactile feel and texture that would sit well together with the rest of the live action shots. The golden touch on the character was part of that and helped bridge the two worlds.

Half way through production it became obvious that we needed a few additional perspectives for the edit. We ended up creating a setup in Houdini that simulates the way the depth camera captures space and used a typical CG-rigged character to create additional shots needed for the edit.

Another creative challenge was that Salomon wanted both the monolith's communication and its POV to evolve throughout the film as it learns more about the world, so a large part of our exploration came with how to portray this. It's a minimal detail in the final film, but it can be seen in how the shapes and rhythms of the light in the monolith become a bit more complex throughout.

Finding the best way to capture point cloud data on moving objects was a technical challenge. We explored a lot of photogrammetry and point clouds generated out of the nuke camera tracker before we went with the Depthkit approach. With the movement Salomon wanted on the talent Depthkit was the only real solution we found that worked.

Partnering up with the folks at Depthkit was a major win for us as they've been working on software that captures this data and collects it in a way that allows the user to export geometry meshes that can be pulled into Houdini.

Salomon> Yeah. Projects like this are so much fun during the conceptual, pre-production and production phases - you get to dream and play and explore. But they are extremely challenging (and the most rewarding) in the post phase. It is completely unnerving putting together rough cuts of personal work - especially if you were riffing, shooting in a more fluid way, not storyboarding, etc. You feel naked, like you're an imposter and it's really hard to see a way through. But you have to turn it over time and time and time again to arrive at the best version of what it can be. It's a frustrating process at the outset because it shows you just how much you suck - but if you have the willpower to keep going and going and going, it often turns into a rewarding one, because (hopefully) you're able to sculpt and craft it into something unique.

I've really enjoyed working with Artjail in the iterative, exploratory phase of this project. Figuring out the visual language of the monolith's POV was such a fun and collaborative journey - it took a while, and I'm sure we would have liked to bypass some of those steps in order to have expedited things, but you can't fast-forward the process.You have to turn over every proverbial rock to get to ‘just the right one’. 

Artjail> Having a complex job like this running for a year, between other jobs, in a somewhat stop-and-go manner was very interesting. On one hand it was exciting knowing we had a lot of fun and creative shots waiting for us, and on the other it was often not easy to get back in the right mindset when jumping back in as it was such a unique look that we had to work on.

The exploration phase at the beginning of the job was a memorable and enjoyable time. After a week of experimentation we presented a whole range of different options to Salomon to get a first impression and isolate the ones that could work, which we took even further afterwards. He was always open to suggestions, so there was a lot of creative energy going back and forth with him and in the team. You could feel the excitement in the air. 

Steve Mottershead - Owner / Creative Director