Digital Impressions for Human Interactions?

Humans are social beings, our interactions with the world around us and each other give life to the world we live in. Our interactions are important, enabling emotional transactions to facilitate meaningful communication and social bonds.

Moreover, communication is an employability skill enabling employees and students to perform in the workplace. In the UK 66% [1] of employers report soft skills shortages, disabling collaborative working, and business performance steadily contributing to an increased interest soft skills training.

3D graphical virtual reality (VR) is increasingly emerging in the soft skills training space, most notably in interpersonal skills training and managing challenging conversations. Though avatars are often expressionless, or lack sufficient facial movement to adequately translate real world movement into VR.

In 1967 Albert Mehrabian and colleagues suggested human communication is ‘’7% verbal, 38% vocal and 55% facial’’, leading to the generally accepted notion non-verbal communication has greater importance than the words we speak.

Our face is brought to life by 84 small muscles, 42 on each side of the face driving our facial expressions e.g., blinks, raised brows, smirks enabling humans to communicate emotions e.g., disgust, anger, fear, happiness, sadness, surprise.  Eye contact is also a fundamental aspect of nonverbal communication, helping to build social bonds and strengthen interpersonal interactions.

But is any of this possible in VR?

The answer is unclear, in short it is sort of (partially) possible. Central to the challenge centres on VR hardware i.e. VR head mounted displays with facial tracker/facial capture technology to translate human expression into VR.

Recent headsets and accessories support facial expression capture, tracking between 38-50 facial expressions e.g. HTC Vives’ Facial Tracker and the eagerly awaited DecaGear 1 headset, due in Q4 of 2021.  Yet a deficiency of 34 untracked facial expressions remains lost, increasing the risk of creating of translating pleasant facial expressions, into fuzzy , difficult to interpret facial expressions.

Even if we agree facial recognition hardware and, avatar facial expressions are ‘real’ enough for precise communications, the current headsets required are cost prohibitive for most.

At the opposite spectrum, more affordable headsets often do not contain built in facial trackers, limiting non-verbal communication to larger whole body, or large limb movement e.g., over pronounced gestures to convey communication.  Leaving many users grappling with stiff, inexpressive avatars to communicate with other humans. In short, anything but what a human interaction looks and feels like.

Quite simply, the human in the loop is rather mooted, battling to leave digital impressions of human communication in a VR world. There is a real danger the VR social platforms and communication VR training we may use to train employees and students, may have a reverse impact e.g., lessen understanding and hinder the development of communication skills.

That is not to say VR cannot be used to train communication skills, just current graphical 3D VR avatars need to be approached with caution. Other forms of VR such as 360VR offer users exposure to every communication component outlined by Mehrabian.

360VR branch narratives, enable users to observe human interaction as a 360 film and select their own interactions. Each interaction selection leads the user to the next piece of narrative, allowing them to observe and understand the impact of their communication. Of course, this is not by any means real time communication, nor does it fully represent a real conversation.

But 360VR can provide the user with access to all communication components, including verbal and non-verbal responses to different types of interaction giving the user insight into human communication. Moreover, 360VR can run on the modern-day smartphone via an App and Google cardboard providing an affordable, accessible, and scalable form of VR without breaking the bank.

Bottom line? Before we run excitedly into the world of VR, we need to ensure it is fit for the purpose we intend it for. As VR continues to evolve, contributing to additional VR typologies and, with it a plethora of hardware it is easy to end up with an ill-fitting training solution that may do more harm than good. A common sense approach is needed and one that orchestrates for pragmatism ahead of innovation.