An ambitious and innovative collaboration between director Roland Lane , world-leading volumetric capture studio Dimension , and the CGI/VFX specialists team at Prodigious London , has seen the creation of the world’s first 2D rap music video using Microsoft’s Mixed Reality Capture technology for English rapper Tino Kamal.
Filmed entirely using Dimension’s state-of-the-art volumetric capture technology using 106 cameras simultaneously, the resulting 3D data was processed to create an animated and textured photorealistic virtual duplicate of Tino.
In collaboration with director Roland Lane, the CGI and VFX team at Prodigious then crafted a three-minute abstract sequence enhanced using a variety of VFX techniques which would be difficult or impossible to achieve using traditional filming and post-production methods.
Quotes
“Tino was the perfect subject for this project. He obviously has a unique, striking appearance which translates effectively on screen, but he also has a fantastic way of moving whilst delivering lyrics. This expressive element to his performance makes for a constantly interesting presence in the frame, and meant he could be the sole subject present. This enabled us to create visuals based around his appearance continuously altering.”
"These are exciting times. As the ways we experience all forms of content are changing rapidly, so do the requirements in terms of the assets we need to create in order for meaningful engagement. I feel the creative possibilities are endless and we really are only scratching the surface of what can be done.”
Roland Lane, Director ---
“What I really wanted to do was capture an element of self-anarchy within the music. Working alongside Roland Lane and the team he put together allowed me to discover an alternative world and the depths within it.”
Tino Kamal, Featured Artist ---
“It was a fantastic opportunity for us to investigate creatively and technically how far we could push this new technology. Most of the work in volumetric is currently in the area of AR and real-time applications which has its own limitations of what you can currently achieve. By combining this technique with a more traditional VFX-centric approach we could create something that was far visually richer, using all the tools and workflows we use on post-heavy VFX projects."
“While currently this approach has a number of challenges, we’re delighted to show the potential of what can be achieved on this project. The ability to defer many of the usual decisions to post-capture in terms of setting up camera angles was a big advantage to give more creative possibilities for Roland to experiment. This approach will clearly have a massive impact for traditional film-making in the future as the technology matures.”
Andy McNamara , director of CGI and Innovation at Prodigious London ---
“Tino is another great example of the incredible creative possibilities that volumetric capture can help realise, and a continuation of Dimension’s pioneering work on believable virtual humans using Microsoft Mixed Reality Capture. Not only is filming volumetrically uncomplicated, it also has the added benefit of giving unparalleled creative freedom to both the post production team and the director. When filming Tino, our 106 cameras record the performance from every angle generating a mesh and texture. Once processed, a sequenced based OBJ is worked on by the production teams to apply the VFX. This is an important first for volumetric capture and the result is a visually stunning and dynamic music video that is a pleasure to watch."
Simon Windsor , joint-MD for Dimension
---
“Most volumetric projects to date have focused on photo-realism and immersion to push the boundaries of human connection. But this team used volumetric as a foundation to create stunning visuals we’ve never seen before, still rooted firmly in Tino’s performance and presence. It’s an inspiring piece, and we’re eager to see how it will move other talented creators to adopt volumetric in their own new ways.”
Steve Sullivan, Partner & General Manager at Mixed Reality Capture Studios, Microsoft
---
A little information on Stone Dogs Post and the crucial part they played on this project. Initially, they had to prepare the processed volumetric capture filmed at @hello_dimension so the team at @prodigiouslondon were able to apply the complex VFX we wanted to showcase. Stone Dogs had to extract mattes from the unique UV's that were generated, figure out a 2D workflow as opposed to the AR/VR path volumetric footage has most commonly been used for. This was a considerable undertaking for their team. After the final video had been created at Prodigious - Camera Moves, CG, VFX, Grade - the footage then went back to Stone Dogs. I worked closely with their talented team as they added some custom-built lens flares, particle and glow effects and overall enhancement (to the already incredible work). Additional finessing of Tino's edges, added film grain and light bleeds added those finishing touches that always elevate a piece of work. It is such a privilege for me to be able to collaborate with people who care so much about the details, and who have the passion and skill to refine a piece of work like this.
Brian Carbin, Co-Owner & Flame Artist at Stone Dogs Post