Mon, 04 Apr 2022 14:42:45 GMT
Deepfake AI, when done right, has enormous power. Global brand and customer experience agency VMLY&R, used Deepfake for good by drastically de-ageing cricket legend Sachin Tendulkar to create an inspirational film.
Working with Ageas Federal Life Insurance, VMLY&R found that better financial planning leads to more confident parents and a greater willingness to allow their children to follow their passions. When Sachin Tendulkar was just a boy, his choices had already come under enormous pressure and thus, he was the perfect spokesperson for little children’s dreams.
The recreation process that used a mix of custom pre and post processes and innovation took over three months, multiple stages, dozens of technical and artistic experts across India and Spain, to complete.
Training Material Challenges
Deepfake AI video regeneration is a complex face-replacement process that only performs as well as the ‘learning’ material that serves as its input. This was the first hurdle as no photographs or videos were available of Sachin at age 11. The few black and white photographs that were from his later teenage years and news interviews were of extremely poor quality and lacking in detail. The VMLY&R team conducted painstaking archival research with the help of the sport star's team to first visualise how he looked at a young age and assembled a set of photos and rare archival interviews.
The team auditioned over 40 child actors to find one who balanced performing abilities with a morphological resemblance to young Sachin.
They then proceeded to extract as many angles and expressions as possible from the archival material and a preliminary regenerated video was executed on eight different young actors to help with the cast selection.
Precision Shooting and Preliminary AI
VMLY&R planned the shooting extremely carefully to ensure proper conditions for the best results, keeping in mind that it had a limited range of head angles. The team had to shoot the actor to mimic the same light conditions as in the archival photos, to get additional layers of information on each compositing.
Before editing, a second preliminary AI regeneration technique was applied on the best shots to make the take selection easier, to ensure that Sachin’s expressions looked believable. Once the team had all the takes and sources, a final calculation was done to get as many details and expressions as possible.
Integration and 3D
In the third stage, an integration of the AI generated visual layers over the young actor's takes was performed. When comparing the results to the reference pictures, it was found that additional facial shape transformations on the chin were necessary to match a resemblance with the target at this age.
So the team had to innovate again. A 3D model of young Sachin head was built based on the young Sachin photographs. This head was match-moved on every take to provide guidelines for proper shape adjustment.
The VMLY&R team even developed a technique to transfer details of the original actor to complete the missing details on young Sachin. Realistic face details such as the interior of the eyes had to be manually restored to increase the likeness. Next, additional 3D layers provided realistic lighting and texturing resources.
The team included some details that are the gold standard for Deepfake tests and can never be achieved with traditional Deepfake techniques - Such as a ball passing in front of the face, the head hiding behind the arm while he was performing with the bat.
At the end of the third stage, after colour adjustment, an artistic touch was an ‘old videotape look’, that helped melting original and post produced layers together.
And finally, Sachin’s unmistakable voice had to be de-aged. The sport star agreed to spend an entire day recording in a studio. With every take, VMLY&R manually de-aged the voice and had to place it on the Deepfake output to make sure it matched and sounded authentic.
Commenting on the campaign, Mukund Olety, chief creative officer, VMLY&R India said: “It’s amazing what we can do with technology. We can tell stories that could have never been told before. We wanted an inspiring figure to convince parents to overcome their fears with better financial planning from Ageas Federal Life Insurance, and who better than our protagonist - the 11-year-old Sachin. While we knew this was something we could achieve using technologies such as Deepfake AI, unfortunately for us, we didn’t have any pre-existing videos of Sachin as a child. Instead of letting that stop us, we found our way through custom tech innovations.”
The TVC will be supported with integrated digital, print and outdoor pieces.view more - Creative
Categories: Medical Insurance, Healthcare and WellnessVMLY&R Asia, Mon, 04 Apr 2022 14:42:45 GMT