Gear Seven/Arc Studios/Shift
I Like Music
Contemplative Reptile
  • International Edition
  • USA Edition
  • UK Edition
  • Australian Edition
  • Canadian Edition
  • Irish Edition
  • German Edition
  • French Edition
  • Singapore Edition
  • Spanish edition
  • Polish edition
  • Indian Edition
  • Middle East edition
  • South African Edition

Mixed Reality and the Future of the Video Call



Addison Capper chats with experts from The Mill, Framestore, Nice Shoes, Preymaker and Taylor James to investigate how we could virtually communicate in the future and how immersive realities will help us get there

Mixed Reality and the Future of the Video Call
“My watch vibrates, reminding me I have a meeting in five minutes, would I like to prepare? No AR preening today, but I’ll head into the waiting room and see who else is on. The meeting pops up on my (barely noticeable) projector - I see a few others - a client, I tap her icon to see more details about her role. The meeting commences, those who are speaking the most gently scale up in the unbound arrangement of portrait bubbles. At the office, a colleague steams milk in the background. AI automatically blocks the high pitch squeal from broadcasting to the group. 

“My turn to share - I broadcast a WIP model of the character animation to the group, using their smartphones to view it in their respective environments - I scale it, rotate it, cycle through versions of the animation, adjust the lighting, play with the textures - on the fly we are able to review, test and finalise. Now we’ve locked down the character, we revert to the screen. I throw our latest character into it’s WIP environment - we go through the same process again in-engine - switching cameras on the fly, adjusting shots, adding a few more mountains, change the lighting from dusk to evening, add some particles to the points of interaction. 

“After the usual review banter, we wrap up the meeting and log off. A transcript of the meeting is sent to my inbox, the updated prototype is already available to the team of artists to continue working.”

Sally Reynolds, associate creative director at The Mill New York, is pondering the not-too-distant future, a future when, hopefully, our working from home lives won’t be impeded by the brain fog - or Zoom fatigue - caused by staring down the barrel of a laptop camera at 2D renditions of yourself, your colleagues and your clients for nine hours a day. 

There is of course some whimsy in Sally’s tale of the future but a fair amount of realism, too. Video conferencing tools have been a saving grace of the pandemic but the uptick in their usage has also highlighted their shortcomings. ‘Zoom fatigue’, for example, is a very real thing. Despite that, our reliance on those platforms shows no signs of slowing down. People working remotely appears to be a trend that will at least be around in some kind of significant way once the pandemic is over

“I think they [video meetings] will become the norm and in person meetings will become the exception,” says Angus Kneale, founder and chief creative officer of preymaker. But he is also certain that the functionality of video conferencing will increase as various forms of mixed reality (XR) advance in functionality and availability to consumers. Take augmented reality, a technology that has been around for decades but is yet to truly make its mark on everyday life. Angus predicts that it’ll be as revolutionary as the smartphone. “AR will enable virtual and remote worlds to be blended with reality, it will allow your workspace to extend beyond your screen and into your immediate environment,” he says. 

Sally pins the evolution of video chat on three key drivers. “Innovation (obviously, bespoke functionality, and physical comfort. I omit perhaps two factors - 1. vanity, and 2. the political state of affairs that may influence progress. I’ll intentionally leave them aside for now, as they’re articles unto themselves. In the near future, there are some simple design changes videoconferencing  platforms could implement into their existing interface and structure. Better design thinking around user behaviour, channelling the various needs of a video conference into tailored formats to best suit their purpose, reducing the emphasis on face-to-screen, the application of collaborative game mechanics, and the list continues.

“A little further down the track,” she adds, “there are functional changes that will affect the way we use VC. Real time audio filters (Google Hangouts), using facial recognition or sophisticated AI to follow a user in their environment (Facebook Portal), utilising user data to provide a personalised experience (which of course opens the door to advertising opportunities, greater funding, and the development of highly commercial VC experiences).”

“We live in a reality where we naturally associate both words, conferencing and meetings, with the word video,” adds Ninaad Kulkarni, a creative director at Nice Shoes. “Although when we attempt to draw the evolution of these forms for engagements in the coming years, the word video seems to most likely be omitted and be replaced with the rise of the ‘R’s, the AR (Augmented Reality), VR (Virtual Reality) & MR (Mixed Reality) sitting under the umbrella of XR (Immersive).”

As such, the evolution of the way we virtually meet each other in the future relies in the hands of not only the video conferencing platforms but also the companies researching extensively into the advancement of various forms of immersive realities. “AR and VR are in prime position to take advantage of this proposed landscape,” says Pete Addington, head of 3D and interactive at Taylor James. “But the real power for shared interaction lies in mixed and cross reality (MR/XR). Put simply, VR can transport us into a different environment. AR allows us to vicariously witness content through our current environment. But it is the potential power and immersion that MR/XR can provide that will allow us to implement a scenario that places the tools and methods necessary to place our working protocols into our lives, wherever this might be.”

Technically, there are already impressive augmented reality devices, namely Microsoft’s HoloLens and the Magic Leap. The biggest barriers to their widespread success is price, however Angus is sure that mass consumer adoption will come as soon as a device launches at a similar price to the iPhone. “AR will become commonplace and the next big consumer electronics boom,” he says, surely. 

What’s more, advancements in this space won’t solely rely on our wearing of some kind of digital goggles. We aren’t all going to wake up one day and enjoy the presence of our colleagues in our living rooms. Augmented reality for the most part right now is being widely used via our mobile devices. “Although limiting, [this] can be an incredible tool for quickly live-sharing prototypes of designs, architectural models, 3D blueprints, interactive flowcharts and even live presentations,” says Ninaad. “All of this content, delivered straight into each colleague’s room, can be live modified using multiplayer interactions from each participant, enhancing the experience of a video meeting.” 

The importance of production can’t be overlooked either, with real-time processing, powered by companies such as Unreal and Unity, eventually playing a pivotal role in the generation of these realities. The ability to capture a user’s facial expressions and body movements would enable more realistic representations of themselves in virtual meetings, which would allow fellow attendees to enjoy a much more natural, visceral experience. “Of course, ‘uncanny valley’ is still a bit of an issue, even with some of the very best real-time human work,” admits Karl Woolley, global real-time director, immersive & IA at Framestore. “But realistic representations of human expressions, even with a non-photorealistic/caricature type avatar can help with engagement.” He adds that Framestore’s pedigree in the animation space has taught the company that the way a character performs and conveys emotion is what makes it believable, just as much as the way it physically looks. “Advancements in capture technology via webcams, TOF or LIDAR sensors when fed through a real-time engine, will help to bring Hollywood level digi-doubles to this space.”

Karl adds that “the evolution has already begun” citing moves from social networks that allow us to apply filters or track objects that surround us or transform ourselves into a cartoon character. Then there are Zoom backgrounds and Microsoft Teams’ ability to approximate depth and simulate depth of field. He also notes an easter egg that you may not be aware of: the ability to combine Snap Camera with any platform that uses your webcam for a video call. Downloading Snap Camera allows you to appear as a potato or dancing banana in Hangouts, Zoom, Skype, Teams, whatever you want. Not quite the potential tools for productivity previously discussed but a step in the right direction and a way to add fun to what are becoming boring and monotonous ways of communicating with colleagues, family, friends and clients. “However, the underpinning technology is what is important,” he stresses. “The ability to track facial features, approximate depth of the user - important data that we need in our traditional work to composite actors and place them in any location and transform them from Mark Ruffalo to Hulk, for example. You then start to build a picture of where this could go.”

Casting our minds forward a decade or three, what will all this mean on a practical level? Will we, as Ninaad predicts, one day have a dedicated room in our homes dedicated to collaborative virtual work, somewhere that could have a common table and chair that people all have in their own homes, a physical prop with which to virtually engage with? “By using wearable devices this room would be capable of transforming into limitless worlds,” he says. “Where all your teammates co-exist in a truly limitless virtual office space. Meetings & conferences aren't bound by limits of occupancy. Navigating to and from departments would be a click away. Passing on files and documents to one another, sharing virtual sketches and drafts are a simple hand gesture away. With devices such as the Oculus Quest already supporting hand tracking, the obtrusiveness of hand controllers is already starting to be eliminated. I think at that point having a meeting in-person might just feel limiting.”

Then there are headset hang ups to consider. Putting it lightly, the augmented reality spectacles currently on the market aren’t exactly the coolest pieces of gear. “[It’s] a real technology battle point,” Sally believes, “as we see companies struggle to develop a wearable that is desirable and functional. Our hearts ever stuck on the vision of Tom Cruise whipping blue screens of HUD graphics with the swoosh of his hands. If it burns your forehead, strains your neck, restricts your vision to a tiny field of view, exhausts your brain… it’s not going to replace a computing interface that is ergonomic. Where there is an amount of physical discomfort, the wearable will continue to struggle.”

But humans are social beasts. Will all of these advancements wrest us from our need for physical, in-person meetings? “The digital landscape is full of technical evangelists and soothsayers,” says Pete from Taylor James. “But we can't predict the future with technology alone. We have to make sure that by replacing aspects of human contact for virtual ones, we are preserving the essence of why this was necessary. Simply being able to technically do something shouldn't replace the reason why we need to.”

As mentioned earlier, we’ve all become closely acquainted with the downsides of virtual meetings, as well as the benefits. This very article was inspired by them. Angus set up Preymaker at the very beginning of the pandemic. Video conferencing platforms have been vital in the growth of his business as he works daily with people from New York (where he is based), Los Angeles and London. It also allows him to tap into talent anywhere across the world and means his employees don’t need to live in an expensive apartment in New York City. “That being said, there is a creative spark that happens when people work together in the same environment,” he says. “I think that it sparks innovation and facilitates more collaboration being in the same space. Nothing beats being face to face with other people.”

Karl agrees. “Much the same as a live gig at Concorde 2 vs, virtual music events, I much prefer the “real life” experience of physical interaction and communication with colleagues and clients as opposed to tele-presence of most forms; especially the real pub versus the ‘quarantini’. The convenience and increasing level of interaction that the platforms offer now and will offer in tomorrow’s world is incredible, but I don’t think anything comes close enough (yet) to the in-real-life experience… time to build The Oasis?”

Big thanks to Sally Reynolds for providing the excellent main image.
view more - Trends and Insight
Sign up to our newsletters and stay up to date with the best work and breaking ad news from around the world.
LBB Editorial, Tue, 29 Sep 2020 16:02:15 GMT