Exploring the Intersection Between Art and Technology
Production company ALTERED.LA operates at the cutting edge of creativity. Always on the lookout for new opportunities to break boundaries, the team were thrilled to learn details of their director Juan Pablo Rodríguez Campo’s latest idea for a short film.
One day, when observing the seemingly benign presence of dust particles floating in a beam of sunlight, Juan Pablo - passionate about exploring the future of storytelling through artificial intelligence - began to ponder AI’s ability to interpret the beauty of everyday occurrences such as this. Inspired, he devised the creative brief for a short film which would be created solely using multimodal AI tools, Midjourney and Gen-2 from Runway App.
Through his exploration of the uses of AI within the audiovisual field, he has discovered many unexpected joys of modern technology, and saw his latest idea as an opportunity to finesse his abilities to utilise AI tools for creative output. This resulted in a newfound appreciation for the precision of language.
Leading the ideation, prompting, and editing process, Juan explored the ways in which AI technology interprets language to create detailed, realistic imagery. The resulting film "A Light Moving Across a Room, and Shadows Dancing Accordingly" is an AI-generated marvel that demonstrates remarkable realism.
LBB’s April Summers speaks to the director about how this project expanded his understanding of the creative parameters of filmmaking in the digital age.
LBB> When did this idea come to you? And how closely does the initial idea resemble the finished film?
Juan> The idea came to me while observing dust particles floating in a beam of sunlight one afternoon. I was struck by the beauty and tranquillity of this everyday occurrence, and it sparked my curiosity about whether AI could capture such subtleties. The initial concept was focused on recreating a specific image – dust particles in sunlight. However, as I delved deeper into AI capabilities, it evolved into an exploration of light, texture, and movement within a space. The final film is different from my original vision but is richer for its journey.
LBB> What were you hoping to achieve with this film and do you feel like you have accomplished this?
Juan> My primary goal was to explore the potential of AI technology in creating detailed, realistic imagery that also carries conceptual depth. I wanted to push boundaries and see how far we can go with these new tools at our disposal. Looking at the final product - its intricate textures, nuanced lighting effects, and overall aesthetic appeal - I can say that although not perfect, I'm very much pleased with the results. They have a certain charm that I didn't anticipate, and it's exciting to see how AI can surprise you in that way.
LBB> Viewers could be forgiven for thinking this film features real footage, as each AI-generated scene is stunningly realistic, particularly the shot of the light shining through the cracked window pane! Tell us about how you used technology to bring these images to life?
Juan> I started with Midjourney, which allowed me to generate highly detailed and realistic images based on textual prompts. The original prompt was about a film still from an imaginary arthouse movie, specifically focusing on dust particles floating in sunlight. I then used the "/tune" function to fine-tune this model according to my vision.
Once I had the images, I moved onto Runway's Gen-2 (Image & Text-To Video) tool. This is where the magic happened - I input a new prompt: "A Light Moving across a Room, and Shadows Dancing Accordingly", which essentially guided how these static images were animated into moving scenes.
The process was iterative and required a lot of tweaking along the way, but it was incredibly rewarding to see these AI-generated scenes come alive with such realism. It's almost like painting with words: you guide the AI with your prompts, but there's always an element of surprise in what comes out!
LBB> What would you say are the key differences between writing prompts for generative AI and prompting real human beings on-set?
Juan> Writing prompts for generative AI is a completely different ball game to prompting real human beings on-set. When you're working with humans, there's a shared understanding of the world and its nuances - emotions, colours, textures, and so on. You can say “make it look sad” and they'll bring their own interpretation of 'sadness' into play.
With AI, it's more about precision in language and understanding the capabilities of the tool you're using. You have to be very specific with your prompts because AI doesn't understand context or abstract concepts in the same way we do. It's almost like coding - each word has its weight and can drastically change the output.
That said, there's also an element of unpredictability with AI that makes it exciting. Despite your precise instructions, sometimes what comes out is totally unexpected - but that’s part of the charm! It forces you to adapt and think creatively about how to incorporate these surprises into your final piece.
So while both require creativity and direction, writing prompts for AI leans more towards technical precision (and patience) while prompting humans relies heavily on emotional intelligence and interpersonal skills.
LBB> In what ways did this project help you hone your craft as a filmmaker?
Juan> I don't know if it helped me hone my craft as a traditional filmmaker per se, but it certainly expanded my understanding of what filmmaking could be in the digital age. It forced me to think differently about storytelling, to consider new ways of creating imagery and conveying emotion. It was a fascinating exercise in learning how to guide AI, how to work with it as a creative partner rather than just a tool.
The editing process was particularly enlightening. Despite the advanced technology at my disposal, it was ultimately human creativity that brought everything together into a coherent narrative. This project reinforced my belief in the importance of the editing process - it's where you find the rhythm and pacing of your film, where you shape its narrative arc.
In terms of skills, I'd say this project honed my patience and adaptability more than anything else! Working with AI can be unpredictable - sometimes frustratingly so - but when you embrace that unpredictability, it opens up exciting new avenues for exploration.
LBB> This work is the latest in your exploration of the intersection between art and technology – what would you say is the biggest misconception when it comes to using artificial intelligence as a creative tool?
Juan> I think the biggest misconception is that AI will do everything for you. That it's a shortcut or a cheat code to creating art – that couldn't be further from the truth. AI is simply another tool in our creative toolbox - like a paintbrush or a camera. It requires guidance, understanding, and experimentation to yield results.
The magic of AI doesn't lie in its ability to generate content on its own, but rather in how it allows us to push boundaries and explore new territories of creativity. It's not about replacing human creativity but augmenting it.
LBB> Who or what AI-produced work has inspired you recently?
Juan> There's a lot of inspiring work being done in the AI space, but one person who constantly amazes me is Paul Trillo. He has an uncanny ability to find utility in new tools and incorporate them into his creative process in innovative ways. I'm also constantly learning from the active AI community on Twitter and Discord, where people are always sharing their latest experiments and discoveries.
In terms of specific works, I've been particularly inspired by the advancements being made with AnimateDiff. It's fascinating to see how this tool is being 'tamed' - it gives you a glimpse into how computers perceive us and our world.
Ultimately though, what inspires me most about AI-produced work is its potential for exploration. Every new tool or development opens up a whole new realm of possibilities for storytelling and artistic expression - that's what keeps me excited about this field!
LBB> Do you have any advice for other filmmakers who want to embrace AI as a creative tool?
Juan> My advice would be simple: Start exploring and don't be afraid to experiment. AI technology is advancing rapidly, and it's providing us with the ability to create almost anything we can imagine. The real limit now is not technical skill but rather our imagination and our ability to tell compelling stories..
Also, don't get too caught up in the tools themselves. Storytelling will always be at the heart of filmmaking, so focus on how AI can enhance your narrative rather than overshadow it.
Lastly, stay connected with communities who are also exploring these technologies. There's so much to learn from each other, and being a part of a community will accelerate your growth in this exciting new space.
