2016 was a big year for virtual reality — from being stuck in beta mode and kept in research, the technology finally emerged and became available to the masses. Presenting a completely new paradigm of interaction, virtual reality began to move us from touch based interactions to interacting with all senses.
So, what does this mean for hardware and content and how they relate to virtual reality? Where do we think VR will take us? Better yet, how do we think VR will take us into the future?
EVERY UNDERDOG HAS THEIR ORIGIN STORY
Though advanced, the concept of virtual reality isn’t necessarily a new one. Most people know VR as the flash-in-the-pan, bulky yet futuristic entertainment technology from the late 1980’s that fell flat on its face, but in fact, virtual reality got its start over a century and a half before.
In 1838, Charles Wheatstone invented the stereoscope and unknowingly invented the underlying technology now used by VR headsets. The technology proved the possibility of creating the illusion of three-dimensional objects by projecting images optimized for both eyes. And therefore, somewhat abstractly, Charles Wheatstone’s stereoscope provided the first example of visually teleporting a user to new places — virtual ones.
Building on this concept, VR technology evolved over the years, and in the 1960s, Ivan Sutherland created the head mounted display Sword of Damocles.
Physical illustrations and images used in the stereoscope were replaced by screens that displayed computer graphics dynamically based on where the user was looking, creating a stronger sense of being transported to another place. At this point, however, the technology was only available in research labs using dedicated hardware, and the visual output was very primitive compared to today’s standards.
Even though this was a breakthrough for VR, there were still a multitude of technical problems to solve before this technology could be repurposed as a consumer product.
PRE-2015: VIRTUAL REALITY ON THE RISE
Fast forward to today. With the introduction of smartphones, the evolution of mobile computing has taken a gigantic leap. Increased processing power and small, high density screens working in combination with sensors such as gyroscopes have all of a sudden created new opportunities for VR technology that weren’t possible in the past. And, since they weren’t originally built for this purpose, the smartphone essentially turned out to be a VR headset in disguise.
Building on exactly this concept, Google released its Cardboard headset to the public in 2014. Powered by any compatible smartphone, this inexpensive headset had finally made VR accessible to the mass market, and introduced a new era in virtual reality. All of a sudden, this new technology, with roots almost two centuries deep, kickstarted a race for innovation — a race involving the world’s tech giants all sharing the goal of setting the precedent for this new medium and, consequently, new potential business models for those bold enough to take the leap.
The Google Cardboard headset not only made VR accessible to the masses in terms of hardware; it also provided opportunities for anyone to define and shape the relationship between content and the medium. Upon introduction, VR leveraged either linear 360-degree videos or interactive computer graphic-based experiences. And at this same time, across both video and interactive, the focus was to create “snackable” content — 3-4 minute-long VR videos and short, lightly interactive games (such as roller coaster simulators). Mindful of the fact that users had to hold the headsets up with their hands, content creators optimized VR experiences to allow users to get in and out of VR quickly without feeling like they missed something.
At the end of 2015, Samsung released the Gear VR. Created specifically for Samsung’s smartphones and with the intent to push VR technology forward, Gear VR provided users with wide input capabilities as well as built-in sensors for improved head tracking. Compared to the Google Cardboard headset, Samsung’s Gear VR allowed for longer time use. And, thanks to a more ergonomic design, the Gear VR unshackled users from having to use their hands, keeping them in VR for longer periods of time without discomfort.
Consequently, this shifted the evolution of VR in the direction of longer experiences. Cue our very own ustwo Games, who leveraged the new technology to create their virtual experience Land’s End: An interactive game which transports players to another dimension where they’re tasked to solve puzzles and unveil the mysteries of the world. Focussing on a learn-by-exploration style, Land’s End is a textless, calming experience that’s simple enough for anyone, novice or expert, to play and enjoy. It took our team almost sixteen months to perfect the world of Land’s End, with challenges across not only developing for ever-evolving hardware (Samsung’s phone technology had upgraded multiple times in the process), but the added challenge of designing for a completely new user experience in a new medium.
In the past few years, design and development software has also worked to catch up to the exploding VR market. As mobile VR hardware evolved, new tools, platforms, and technology necessary to create VR applications emerged. Game engines such as Unity and Unreal — engines adapted to support VR — as well as 360-degree camera rigs began in to increase in popularity across the industry. And thus, a new market around VR content creation was born.
Seemingly overnight, virtual reality’s user base expanded from thousands to millions, and in accordance, the opportunities for partnerships between brands and VR headset manufacturers began cropping up. The New York Times were one of the first to leverage the technology and try their hand in the virtual world: In 2015, NYT distributed over one million Google Cardboard headsets to their subscribers for free. Accompanied by an application showcasing 360-degree video stories, this move allowed The Times to frictionlessly introduce their subscribers to a new, more immersive way to understand current events.
As Google Cardboard and Samsung Gear VR headsets gained traction worldwide, those of us interested in this new technology began to collect insights and learnings, allow us to better understand user expectations around the new medium and identify areas of opportunity for its future. Simultaneously, Sony, HTC, and Oculus stayed heads down, leveraging these same market insights to define and build what would become the next generation for of virtual reality.
With the announcements of the powerful HTC Vive, Oculus Rift in 2016, virtual reality now attracted the full spotlight of the media, who seemingly abandoned wearables overnight, and instead focussed on touting VR as “the next big thing”. With a growing knowledge base, an understanding of what we could harness virtual technologies for, and what cross-industry role they could play in the near future, it’s become apparent that we’ve officially entered the age of VR, breathing new, lasting life into the concept Charles Wheatstone had invented over a hundred years earlier.
USTWO + VR: AN INSTANT LOVE AFFAIR
Looking back on ustwo as a company, it’s pretty obvious we’ve always had a fierce attraction to emerging technologies. Therefore, it was only natural that the brave new world, (or should we say, virtual world?) of VR experiences would truly be love at first sight for our team. Unexplored, untapped, and untested, designing and developing for all aspects of virtual reality was something we couldn’t wait to dive into.
While the Games Team was heads down on Land’s End, our other studios began to test the waters as well. Around six months after the release of Google Cardboard, our New York team partnered with Google to explore the potential of VR by creating interactive guidelines teaching how to design and develop for VR … in VR. Gathering insights from the existing VR market, we leveraged those to create a set of guidelines for designers and developers to understand what proved to work within VR, what didn’t work, and what made some Cardboard based VR experiences really great.
Of course, crafting something in VR that would teach people how to craft things in VR, all while learning how to craft in VR ourselves, we came across a multitude of hurdles. As it was a first iteration technology, the biggest design challenges with the Cardboard headsets were centered around the constraints of the technology, the inputs of the headset, and the capabilities of using a mobile phone to power the experience. These restrictions enticed us to think creatively about how we could achieve immersion using fairly simple graphics while still deeply immersing the user. This also led us into exploring interactions such as how users could move around in VR without actually moving their physical body, and how 3D audio could ground the user in the experience while creating a suspension of disbelief. The result of our efforts was Cardboard Design Lab, which can be downloaded for Google Cardboard.
Observing users trying Cardboard for the first time and honing in on the way they reacted to their experience helped us understand that there’s incredible power and potential in this medium. Even in this early state of the technology, we could create something that really immerses the user.
Alex Faaborg, friend, former partner of ustwo, and Design Lead for Google Daydream, described it this way: “People often have very strong reactions when first trying Cardboard, with laughter, gasping, exclamations, etc. In general, VR seems to be an emotional amplifier compared to earlier computing platforms."
VIRTUAL FOOD FOR THOUGHT
With almost two years of working in the realm of virtual reality under our belts, we’ve definitely picked up a thing or two. Virtual reality is not only a unique medium that can tap into and enhance the emotions of its users, it’s also a medium that requires those who seek to create for it to constantly re-evaluate their approach and pay close attention to how the technology is evolving.
Arriving at present day, our team has come away with four major insights and questions that you should be asking yourself before venturing down the virtual path:
ASK YOURSELF: WHY SHOULD I MAKE THIS IN VR?
The most fundamental question that anyone who’s thinking about creating an experience in VR should be asking themselves is why they’re choosing VR as their medium in the first place. If you or your company is interested in getting into the VR game, it’s worth it to stop and think about what your goals actually are and if another medium would better help you achieve them, instead of trying to jump on the virtual bandwagon simply to say you’re on it.
Is it for branding purposes? Is it to solve problems in new ways? Is it to explore new possibilities for internal tools? Is it to engage your target group in innovative ways?
Since VR can open up an overwhelming amount of possibilities, thinking about your goals from a medium-agnostic perspective can help you decide if a virtual reality experience is right for achieving them. It’s also worth giving thought to whether or not said potential VR experience would create value for your users beyond their first use: Would they go back and try your VR app a second time? If so, why?
TL,DR: Don’t make things in VR for the sake of making something in VR.
ASK YOURSELF: ARE YOU REINVENTING THE WHEEL?
Generally speaking, simply translating content that already exists on another medium into VR isn’t a successful strategy. Going from a 2D world to a 3D one means considering your user’s newfound freedom, and the various ways they will interact with your experience. Consider the nuances of audio cues, gaze cues, and heightened emotions. From games to linear storytelling, there’s a good chance that if it already works perfectly well on mobile or web, it will be wasted in VR.
DECIDE WHAT TO TRY, NOT WHAT TO DO
There’s a good chance that what you’re hoping to making in VR will be something that hasn’t been done before. While this is certainly a good thing, making sure your concept is fluid and malleable is important. Build real prototypes in low fidelity and make sure your team is not only aligned but in sync through the whole project. This will let you move faster while creating for an unfamiliar medium, which in itself will cause delays.
ALWAYS KEEP IN MIND YOUR USER EXPERIENCE
Virtual reality is still a very new concept and globally, only a very small subset of people have actually given it a try. As VR designers and developers, we have a responsibility to ensure a new user’s first experience is not only exciting and inviting, but will keep them coming back for more.
Design considerations and factors such as motion sickness and phobias have been established and documented, and there are experiences out there that navigate beautifully around these. At the same time, however, we’re still seeing poorly crafted VR experiences released every month. User testing for VR is much more intimate and intense than traditional user testing for functionality or desirability. As we enter this new area, we need to remain aware and be ever vigilant of testing the cognitive qualities of our VR experiences.
2016: THE YEAR VR GREW UP
If 2015 was the year of simple, easy-to-access mobile VR, 2016 would be the year where hardware took its first steps toward maturity, with the industry offering tethered, yet incredibly powerful headsets with more input capabilities.
After massive investments and years of R&D by companies like Oculus, HTC and Sony, this new generation of hardware was initially focused on incorporating gaming into VR in a way that wasn’t possible before. Both the Oculus Rift and HTC Vive are tethered headsets powered by PC’s, so having a focus on gaming made sense, since the early adopters purchasing these headsets would have the existing computing power needed to utilize them properly.
The first of these new players to release was the Oculus Rift: A VR headset that could track your movements in space within a 5 x 11 feet area, and used an Xbox controller as an input. And, as a recent addition, they followed with the introduction of the Oculus Touch controllers and additional sensors, enhancing the experience even further.
Shortly after the Rift, the HTC Vive debuted, which used a headset and controllers that could track your head, hands and body’s movement within a 15 x 15 foot area using sensors. This was a great leap in terms of the amount of freedom a user could have in the experience and this freedom created a greater sense of embodiment than ever before.
Next to come to market was Playstation VR, which was released in October 2016 and offered a cheaper alternative for gaming. Playstation VR was mostly optimized for a seated VR experience, and while it wasn’t directly competing with the Rift and the Vive, it was still innovative in its multiplayer offering: It uniquely allowed local multiplayer functionality by utilizing both the headset and the TV to create a social gaming experience between players in, and outside of, VR.
But one of the most powerful benefits of Playstation VR was its built-in reach: With 40-50 million existing PS4 users, this hardware easily tapped into an already existing, engaged and sizeable ecosystem.
Last on the market, rounding out the year, was Google Daydream. A new headset from Google that was released in November 2016, this handsfree headset was created for powerful smartphones and comes with a controller, enabling completely new ways of interaction, compared to their simpler Google Cardboard headset.
Across this new generation of hardware, we’re seeing an enabling of experiences that are of a much higher fidelity than we’ve ever seen before. Most importantly, the added sensors of each of these new options and the ability to track your body's movement in space has created a higher degree of freedom and immersion; we’ve finally achieved VR experiences where it feels like you’re actually there.
But of course, this newfound freedom comes with a price, both literally and figuratively: One of the biggest problems as we enter 2017 is the cost of hardware, and the fact that it’s dependent on powerful PCs or gaming consoles to run. Also, the need to mount sensors in order to track your position in the room, in addition to the technology still being heavily tethered are things that need to be improved in the future. Much like home automation or wearables, for VR to truly achieve success in the consumer market, its hardware will have to not only be powerful, it also needs to be financially and technologically easy to get into.
"Mainstream VR, regardless of how powerful it is," says Alex Faaborg, Design Lead at Google Daydream, "needs to be incredibly simple to setup and use."
A LOOK AT VIRTUAL CONTENT & EXPERIENCES
Even though all four of these new VR platforms were targeted towards gamers, we’ve actually found that the most interesting and successful experiences haven’t been related to what typical PC or console games usually look like. Here are a few examples:
With Tilt Brush, users can draw and paint in 3D, something truly impossible to do in reality, and an experience that opens up a whole new dimension of creativity.
Job Simulator let the users relive how it was to work in the past, in a future where robots have taken over and people have forgotten how to do ordinary jobs, such as cooking.
Rec Room is essentially a social VR club where people can play games such as paintball, dodgeball and charades with other users from around the world.
All of these experiences are interesting in that, just as much as they don’t resemble traditional games, and they all uniquely enable creativity, collaboration, and offer users exclusive types of freedom to direct their own adventures.
2017 & BEYOND: THE FUTURE OF VIRTUAL REALITY
As we enter a new year, it’s still very clear that the future of VR is wide open, and anyone’s for the taking. At ustwo, we have been (and will continue to) experiment and test the waters with unique virtual content and the latest hardware. In doing so, our team has the following predictions about where the future of virtual reality will take us:
PORTABLE, BUT NOT MOBILE-BASED VR
Over the past year, we’ve seen a lot of improvements in VR hardware. But what’s next? We believe that the next stage of hardware will play a key role in bridging the gap between mobile VR and the powerful tethered headsets.
With prototypes such as the Intel Alloy and the Oculus Santa Cruz, we could soon experience a new generation of robust, portable virtual headsets that can both track your position and understand your surroundings without being tethered to a computer. These would enable a completely new experience that you could move with ease from your living room to accompany you on the go, that could act as entertainment in self driving cars, and that could very well eventually play a key part in how we live our daily lives.
MORE FREEDOM CREATES GREATER IMMERSION
With hardware advancing in leaps and bounds, creating new degrees of freedom in how users can interact with virtual worlds, it’s evident that this will come hand-in-hand with more immersive content than we’ve ever seen. In the near future, we anticipate more technical innovation around tracking finer details in physical movement like eye tracking, facial expressions, and the ability to provide more haptic feedback will propel immersion further.
WE’RE ON THE PRECIPICE OF A VR CONTENT REVOLUTION
As we’ve already noted, 2016 saw a massive leap in virtual reality hardware innovation across a handful of big-name players such as Oculus and HTC. As we enter 2017, we can only assume that this technology is what we as designers and developers will be working with for the next year, and thus the focus now shifts to achieving a breakthrough in content.
A transformation in this area will be crucial for the future of VR. However, we predict this is only possible by taking full advantage of the unique aspects VR has to offer and inventing the new, not translating the old, into virtual reality as a medium. Looking at 2017, the most successful and “sticky” experiences mentioned above all provided users with something that wouldn’t be possible to achieve in other mediums; we firmly believe this is this is the secret recipe to successful VR content, and will be crucial to utilize as we create new experiences.
We also believe that social and multiplayer will play a huge role in the future of VR content. This medium has the opportunity to create a digital presence of other users in your experience that simply hasn’t been possible before; we witnessed hints of this in 2016 when Facebook really stepped into the VR scene with their social VR experience during the Oculus Connect 3 Conference, and we anticipating it taking center stage as time goes on.
From gaming, entertainment, education, and healthcare, to architecture and social experiences, virtual reality truly enables us to create worlds that will affect and enable people on new emotional and physical levels, and with greater focus than ever before.
The VR hardware game is strong. Amazing content is on it’s heels. It’s 2017: Time for us as designers and developers to leave our mark in the virtual realm, and set the precedent for generations to come. Let’s make something amazing.