Shifting the field of view: science stories in virtual reality

Since first donning a Virtual Reality (VR) headset only eight months ago, my personal relationship with this technology has progressed at lightning speed, way past the awkward getting-to-know-you phase.

In the broad scheme of things, I probably could be described as an early adopter of the technology and a VR content producer, rather than a mere passive consumer (although, one could argue that ‘passive’ and ‘VR’ shouldn’t be used in the same sentence).

I find being in this position exciting, yet, highly unlikely. I’m not a gamer and I’m sadly a complete movie inexperient. I think I spent most of my youth barefoot in the dirt rather than being immersed by screen-based storytelling. It’s ironic I guess, given that screen based science storytelling is my adult passion and profession.

‘COLOSSE’ was my first VR adventure, a real-time virtual reality storytelling experience, with a stylised, character-focused visual language. I spent a mere two minutes inside the beautiful, simple animated world but it was inexplicably transformative.

I won’t even try to explain how amazing it is inside a well-designed virtual story - there are no words that can describe the power of the immersive experience. VR is something you have to actually try in order to understand. Despite the simplified visual language and restricted colour palate, COLOSSE manages to physically and cognitively transport the viewer into a world that, thanks to VR technology is unbelievably believable.

VR technology has come a long way since the motion-sickness inducing early days. Gaming enthusiasts, developers and film studios have embraced the recent release of the fist consumer-ready VR headgear and the buzz has been huge. VR offers a truly immersive experience and promises to transform how we play, interact and learn but exactly how this will evolve is uncertain.

I am particularly interested in how VR can be utilised for education and awareness in medicine and science, beyond its superficial appeal as a mere gimmick. (Secretly, I believe science and medicine could lean on gimmicks more, serving as an ice breaker and communication platform… but that’s a story for another time).

Mark Zuckerberg / Facebook In previous work, I created animated biomedical stories about cancer and epigenetics, the focus of research carried out at the Garvan Institute of Medical Research. The animations show how biological molecules behave inside our cells, and how things can go wrong in disease. As the biomedical scientist-artist in this context, I present a visual ‘review’ of a science story. In my head I can see the scenes in three dimensions, then the animation software and editing process allows me to convert this for two dimensional viewing on a screen - providing a ‘window’ into the molecular world. One of the most common things people ask is how they can get beyond the field of view. People don’t just want to look through the window, they want to jump through the window. With VR, I can make that happen. The question is, do I want to? what would this achieve? Could VR facilitate embodied cognition, linking our thoughts to our physical experience? Pedagogical approaches that tie experience to thinking have been shown to enhance learning in a number of contexts, dating back to experiential learning theories of John Dewey (1938). Students of anatomy can now progress beyond looking at an illustration within a text book and interact with anatomical structures in 3D. A rendering of DNA. VR and education has also featured in the TED and TEDx speaker series over the past 12 months. Michael Bodekaer, founder of Labster which teaches life sciences through gamified education talks about immersive 3D virtual worlds and laboratories. Alex Faaborg, of Google Cardboard talks about how VR is providing incredible opportunities for the future of art, journalism, and education. Working at the frontiers of interactive technology, Chris Milk stretches virtual reality into a new canvas for storytelling. You will notice that this last example does not have a strong focus on education, but I can’t help but imagine how science storytelling will be transformed by VR and then how those stories will influence education. My challenge is to tell science stories about epigenetics, a topic that is complex, dynamic and extremely abstract to most people. Epigenetic structures and events are essentially invisible, smaller than the wavelength of light. Yet, epigenetic mechanisms are incredibly important. They underpin our health, tie us to our ancestors and commonly go haywire in disease. Communicating this in a visually engaging, informative way can be hard, but biomedical animation has been able to bridge the gap for many through scientifically accurate, beautiful animations that can be awe inspiring. But, imagine being able to go past the field of view, through the window and be physically inside the cell. Some early projects that have explored this include Molecule VR, one of many educational tools developed by Unimersiv that aims to support and integrate classic teaching and learning methods. This 360 degree video demonstration from Nucleus Medical Media looks amazing even on a mobile device. Scientific communication studio Random 42 have also produced an incredible VR experience that uses signal transduction pathway to transport the viewer into the cell (starting at approximately 45s). Random42’s VR 360 showreel is spectacular.This is surely just the start of science visualisation in VR, but will immersive experiences actually translate to better understanding? What science stories are the most suitable for telling in VR? How do we overcome the technical limitations that still exist? As a practitioner and visual communication researcher, while I love viewing science content in VR, it is the research questions and challenges that I find the most exciting.