Can We Bridge the Uncanny Valley?

First discovered by robotics professor Masahiro Mori in 1970, the uncanny valley effect is defined as a level of realism in robots in which the human observer has a negative reaction. And, as CGI advances in leaps and bounds, the effect has found its way from the most glamorous of Hollywood movies and into many of our most loved advertising campaigns. The uncanny valley phenomenon is a key deliberation in the lives of concept artists, 3D modellers, animators, production designers and render specialists.

What Exactly is the Uncanny Valley?

A term created to describe the emotional response curve experienced by humans observing lifelike non-human entities. Mori’s original hypothesis plotted emotional response (positive or negative) or familiarity against increasing levels of human likeness and more or less predicted the following:

As the appearance of a photorealistic 3D model is made more human, the observer’s emotional response will become more positive until the likeness reaches a point where slight imperfections become emphasised and the emotional response turns overwhelmingly negative. In other words, the more realistic a computer generated human looks, the easier it is for our brain to spot its imperfections. As the appearance of the entity continues to become less discernible from a human being, the emotional response becomes positive once more and approaches human-to-human empathy levels.

Over the course of the past decade, there have been many attempts at photorealistic human rendering, many of which have failed and some which have overcome the valley of the uncanny. The phenomenon is best conveyed in chart form. As emotional response becomes increasingly positive as human likeness increases, and then drops off sharply into the uncanny valley, as described in Mori’s first prediction. The second part of his hypothesis is shown at the far right of the diagram, where the emotional response curve begins to climb until it peaks when human likeness becomes 100 percent accurate.

LBB Asiapac editor Larissa Meikle speaks to the experts to get their advice on how to best overcome the uncanny valley phenomenon.

How Do You Avoid the Uncanny Valley?

Alt.vfx CG supervisor Matt Hermans usually avoids ‘CG Humans’ at all cost. “I think most companies will avoid it and so-called Digital Doubles are usually used for wider shots where stunt performers could be injured,” he said.

“The work we have done in this area was very involved and required us to, for example, age a young girl’s face as she inhales a cigarette. We made sure to keep her eyes from the main photography, as that is one of the number one cues that the human brain looks for.”

Other than this, and a few other examples, he doesn’t find it common to need to completely reproduce a human (dead or living) believing there is an associated stigma attached with it.

Framestore’s head of 3D, Simon French, said it may seem like stating the obvious but you must either keep your character stylised enough that a comparison with real humans is never made, or if the goal is to create a photo real human, then you must set realistic goals for what is possible and execute it well enough to be indistinguishable from reality.

“This is an easy thing to say but not very easy to achieve and the gulf in-between these two goals is very wide and this is the uncanny valley,” he explained, adding: “There's no simple list of things that you can do to avoid it, as it's so dependent on the subject and the context in which a character is presented.”

Meanwhile, at Glassworks London, Director of 2D Duncan Malcolm finds that an over emphasis on slickness and perfection can contribute to an 'uncanny' oddness that's difficult for viewers to put their fingers on but makes them feel uncomfortable nonetheless. "Eyelids are often a give away, especially where they seem to close fractionally too fast. Or the crease of the mouth can be a little too defined. Sometimes there's an ever-so-slight change in face shape that, although never 'seen' in a real face, causes weirdness when its missing from a fake one. We often are tempted to over stretch a virtual character, allowing it to move slightly faster, react quicker and forget to add human quirks, mistakes and hesitations that make us believable. " 

In fact Resolution is ahead of the game, mapping out a dedicated document to give to clients to ensure humanistic CGI expectations are understood and met. It states that in short, there are two ways the uncanny valley phenomenon can be defeated: with cartoon realism instead of striving for complete photorealism (studios like Pixar, Dreamworks, and Disney have all developed their own unique flavour of cartoon-realism), and innovation of technology, with modern sculpting, texturing, and performance capture technology. 

Curbing Uncertainty with Appearance, Behaviour and Ability

The work Alt.vfx has done includes augmenting live action performances where a real human has made acting decisions to append on; saving the company from a lot of nuanced work which only the largest film houses have been able to touch.

“A great example is The Matrix sequels where several high-speed cameras were used to photograph several actors as they performed; this data was interpreted by custom software to re-build their faces exactly as photographed,” Hermans said.

He explains that a digital head could be put onto digital bodies and lighting and integration work could be done in an arbitrary environment. This work is called Universal Capture and it does not try and synthesize humans, but instead captures every nuance of existing people for later digital use in a realistic context where performance is critically real.

Over at Framstore, French notes that one of the ways it can be managed is to be realistic about what you expect performance wise from a CG double. “A full-on emotive acting performance that is indistinguishable from real life is still something that has not yet been achieved, but more subtle use of CG humans, either in action sequences or in scenarios, where the facial performance is intentionally muted or obscured in some way, can be much more successful,” he said.

When it comes to designing elements to match human realism, Hermans makes mention of Alt.vfx's work on "Iron Man"-style characters where live action characters have had CGI added/replaced parts of their bodies. “This has worked well as the eye can still resolve real human details in the face and as long as integration (shadowing/lighting) is done correctly then it's quite successful and straight forward.”

He also agrees that facial proportions and photo-realistic texture are something to consider. “Personally I found the work done by Weta amazing as they purposefully mix photorealistic human skin detail with exaggerated proportions that cartoon and sci-fi creatures can have. Captain Haddock and Tintin were strangely believable and sat in an area just left of the uncanny valley.”

He said that whilst they moved like humans they were obviously not: “Gollum and the Avatars too were fantastic creatures that had all the texture qualities that human skin has (skin pores, muscle and fuzz/hair) but never approached the uncanny valley.” 

Resolution creative director Tim Dyroff also points out another uncanny valley left-of-field vis-effect and one of Hollywood’s best kept secrets – basically used to keep Nicole Kidman looking like she is still 35. 

For a long time Lola's work remained top secret, both in terms of how it created such realistic work and also the very existence of the Santa Monica based VFX house. Rumour had it that the biggest stars were a little less young and gorgeous than they appeared on screen, and that for an undisclosed amount, you could get a digital tummy tuck, a facelift or a second chin lost. But it was never more than rumour, as Lola never went on the record to discuss its work and no actor would willingly promote that in real life they are older than they appear on screen or that their imperfections were fixed in post.

Dyroff says this VFX tech is especially useful for close ups in scenes where people can study you more carefully and usually in the big stars’ contracts they have clauses that state: “Yes, I will get 35 Lola scenes and I will looks great”, as they are, after all, selling a brand of themselves being youthful and beautiful.

Pushing the Boundaries of CG Humans

Whilst there have been numerous attempts at reproducing deceased celebrities for advertising campaigns including Audrey Hepburn, Elvis, Marilyn Monroe and Gene Kelly, Hermans says there is also a sub genre of work done with CG Humans but they're instantly recognisable as fantasy or sci-fi; an example of this sort of work can be found in most video-game promotional trailers.

“Virtual reality will require more and more of this sort of work in the future and I'm sure the uncanny valley will be slowly dissolved as technology gets better and better at sampling and simulating the factors in human performance and appearance,” he added.

French says Framestore’s highest profile work in this area has been the Galaxy Chauffeur ad starring a CG Audrey Hepburn.

This along with other ads like the Dior J'Adore and the Bruce Lee Johnnie Walker ad, were all commercials that pushed the boundaries of CG humans, but he admits they all have good shots and bad shots, and almost in every case the bad shots are the ones where they try to perform too much.

“The Bruce Lee one may be the most likely to draw uncanny valley comparisons as he is talking and emoting directly to camera, but all of these ads have attracted such comments in some way,” French said.

In retrospect, the one thing Framestore learned from doing the Galaxy job is that making a believable still image of a human is really quite achievable, but making it look exactly like a specific well recognised person is significantly harder.

“Making this character then perform and move exactly like that person is many orders of magnitude harder still,” he explained. “There's no such thing as just getting it right and then having a fully believable digital human to use in any situation – every individual shot is a new challenge.”

Over at Resolution, Dyroff says the CG ad that disturbed him the most was the ever popular Evian Water ‘Roller Babies’, which clocked up close to 80 million hits on YouTube.

“They have strange reactions and their facial expressions are odd – their eyelines are out, and yet it had a great response from viewers,” he said. “There will be a lot of people out there that love Evian Babies, but for me it’s a bit freaky, and funnily enough it doesn't make me feel like the water is pure – I see a very CG commercial and something that is artificial, so it doesn't connect me to the product.”

“Maybe it’s because I’m a specialist as I’m sure lots of people would see it for the pure fun that it is – giving the brand personality.” 

Dyroff admits that the more your eye is trained, the more you see and the more critical you are. “When I was in early days of learning my craft, I was looking at ads and breaking them apart – that’s the curse of the person who is trained,” he quipped.

He believes it will be hard to tell where the future is headed with CG and almost impossible to distinguish between reality and what computers are contriving.

“The eye and the human brain are such beautifully perceptive tools, and it’s not until you work in this industry, and experience it first hand, that you see how complex it is to get something looking absolutely bang on so that people can’t tell that you have put something into a scene that isn’t real.”

He says it can take weeks and weeks of experts adjusting the tiniest of things to get the desired results, admitting humans have always been the last frontier in CGI: "If you can get a humanoid on screen and get people to actually believe it’s real; for me that's the wholly grail of visual effects work.” 

We leave you with this final thought...