What you see is not always what you get: how virtual reality can manipulate our minds

It is often said that you should not believe everything you see on the internet. But with the advent of immersive technology – like virtual reality (VR) and augmented reality (AR) – this becomes more than doubly true.

The full capabilities of these immersive technologies have yet to be explored, but already we can get a sense of how they can be used to manipulate us.

You may not think you are someone who is easily duped, but what if the techniques used are so subtle that you are not even aware of them? The truth is that once you’re in a VR world, you can be influenced without knowing it.

Unlike video conferencing, where video data is presented exactly as it is recorded, immersive technologies only send select information and not necessarily the actual graphical content.

This has always been the case in multiplayer gaming, where the gaming server simply sends location and other information to your computer. It’s then up to your computer to translate that into a full picture.

Interactive VR is similar. In many cases, very little data is shared between the remote computer and yours, and the actual visual scene is constructed locally.

This means that what you are seeing on your end is not necessarily the same as what is being seen at the other end. If you are engaged in a VR chat, the facial features, expressions, gestures, bodily appearance and many other factors can be altered by software without you knowing it.

In a positive sense VR can be helpful in many fields. For example, research shows that eye contact increases the attentiveness of students, but a teacher lecturing a large class cannot make eye contact with every student.

With VR, though, the software can be programmed to make the teacher appear to be making eye contact with all of the students at the same time. So a physical impossibility becomes virtually possible.

But there will always be some people who will co-opt a tool and use it for something perhaps more nefarious. What if, instead of a teacher, we had a politician or lobbyist, and something more controversial or contentious was being said? What if the eye contact meant that you were more persuaded as a result? And this is only the beginning.

Research has shown that the appearance of ourselves and others in a virtual world can influence us in the real world.

This can also be coupled with techniques that are already used to boost influence. Mimicry is one example. If one person mimics the body language of another in a conversation, then the person being mimicked will become more favourably disposed towards them.

In VR it is easy to do this as the movements of each individual are tracked, so a speaker’s avatar could be made to mimic every person in the audience without them realising it.

More insidious still, all the features of a person’s face can easily be captured by software and turned into an avatar. Several studies from Stanford University have shown that if the features of a political figure could be changed even slightly to resemble each voter in turn, then that could have a significant influence on how people voted.

The experiments took pictures of study participants and real candidates in an mock up of an election campaign. The pictures of each candidate were then morphed to resemble each participant in turn.

They found that if 40% of the participant’s features were incorporated into the candidates face, the participants were entirely unaware the image had been manipulated. Yet the blended picture significantly influenced the intended voting result in favour of the morphed candidate.

What happens in the virtual world does not stay in the virtual world. We must therefore be mindful when we step into this new realm that what we see is not always what we get.