Creating Experiences in Virtual Reality

Project Brief

  • My Android and Computer Vision Platform Lab was made up of researchers working on next generation mobile services at Samsung Research America, an R&D campus in Mountain View, CA 
  • As a Virtual Reality Publishers/Developers Engagement intern, I became intrigued by the accessibility and content creation capabilities of WebVR (virtual reality accessed through the web). I thought about how this technology could impact Samsung and intersect with the lab's goals. 

Problem Space

  • Samsung Electronics is a global leader in technology amassing an array of products from TVs, smartphones, wearables, cameras, and more. Since their mission is to transform our interactions with these technologies through innovation and discovery, can WebVR be a fitting step to explore a new generation of interaction? 
  • By incorporating WebVR in Samsung's online presence such as through social media, marketing, web platforms, and also as supplementary content to products, new interaction avenues can be explored to draw in interested users. 

Goals

My goals for this project were: 

  • To design virtual reality interactions making use of mechanics such as gaze (sometimes when you look at something, something happens!), reactive storytelling (why'd you have to look at that?!), and role-play (now, I'm going to have to kill you, prepare yourself) to achieve immersion within two VR experiences
  • To utilize WebVR in expressing web content that transcends 2-D design through a series of WebVR experiments

Presentation

  • At the end of the summer, I presented my design's both at our lab showcase and then a public showcase for all employees.
  • My manager, Paul, has seen a lot to do with the VR field and I appreciated his feedback throughout the summer on my project and in daily work. 

"I had the pleasure of having Philip as a summer intern, and I can say Philip is one of the most creative people I've met. He was a reliable member of the team, supporting other team members as well as putting his energy and passion towards a summer project involving creative VR storytelling on WebVR/AFrame, as well as interactive 360 videos. I look forward to Philip's successful studies and career."

What I Learned

  • WebVR Design: A great deal about WebVR design and curated a collection of current work
  • VR Tools: Discovery of some awesome new tools such as A-Frame 
  • The VR Landscape: Exposure to numerous amounts of varying VR content
  • VR Content Trends: Strategic analysis of VR content trends using metrics such as downloads, ratings, price, genre, and current events 
  • Developer Relations: Augmented understanding of the process behind developing relations with large and small companies for products that will be used by millions 
  • The Universe in You: A fundamental shift in mindset from 2D design to designing for an immersive VR experience 

WebVR? 

  • WebVR is an emerging technology that aims to present virtual reality content in traditional web browsing interfaces. The experience is delivered through a JavaScript API that provides support for virtual reality devices. A demo of WebVR can be found on the A-Frame website (even without a headset!)
  • Not only is the long term ambition of WebVR to democratise virtual reality content, but it also provides a great way to create quick VR prototypes and experiments before investing in full production using tools like Unreal Engine and Unity. 

 

VR Portfolio Experiment

  • I'll start off my designs with this WebVR portfolio, built in A-Frame, which showcases the selected projects on this page.
  • It provides a look into interaction methods for WebVR that I am experimenting with.
  • In this scene, I dealt mainly with gaze-based text interactions (look at something and text appears), video playback (play 2D vidoes in VR), and scene atmosphere (like the start of a good movie)

Prototype Experiments 

These are some interactions in WebVR I was exploring from interfaces to animations to gaze to augmented reality.

Interface experiment: See here: philipbegel.com/360images/

  • An interaction for a 360 degree image gallery viewer I was experimenting with.
  • The image gallery selection appears at the bottom of your vision in what I'm calling "link spheres."
  • Gazing over a link sphere will have a name of the image hover over it and enlarge the sphere for selection feedback.
  • Clicking/tapping the link sphere will transport you to the selected 360 image and also animate the sphere by moving away from the others to indicate you have been to this link sphere already. 

Augmented reality experiment: See here: https://zircon-makeup.glitch.me/ (you'll need the Hiro pattern marker opened on your phone or printed out)

A Rick & Morty augmented reality experiment. Text reads: Oh my god, Morty, we're in augmented reality, Morty!

Do you watch Stranger Things? Do you enjoy the Stranger Things theme? What about both in a virtual realityyy trailer, maaan. 

See here: http://acoustic-whip.glitch.me/

Playing with 3D-model animations on a Kuriboh. 

 

See here: https://ripe-spring.glitch.me/

The following are concepts for two virtual reality experiences trying to achieve immersion (feeling like you're there) and shown with some prototypes and sketches

Immersion Techniques

  • I'd like to define some terms on achieving immersion that I will refer back to. I turned to the work of Kent Bye and his Elemental Theory of Presence for describing the qualitative elements of a virtual reality experience.
  • The theory uses the four elements of natural philosophy as a framework for describing the different levels of plausibility in virtual reality (the idea of asking yourself: "Is this really happening?" which contributes to your sense of presence in the experience).
  • I utilized this framework in order to incorporate these aspects into the gameplay of my two VR concepts: To The Moon and IntroVRsion

Concept art for the introduction scene. “You can really feel the heat of the flames as your headsets memory burns up.” ~ Actual user tester E. Musk.

Do you ever wonder to yourself, huh, the world’s most common first name combined with the world’s most common last make a pretty unique name: Muhammed Wang.

If you’ve had these thoughts or similar, then you’ve daydreamed before. IntroVRsion is a virtual reality game taking place at a party where your mind drifts off thinking about a girl such as saving her from a dragon and fighting off ninjas.

Concept art for talking to a party guest…but who wants to talk about video games at a party? Psh, look at her laughing at you! She really wanted to talk about dinosaurs.

  • As you interact with the conversation interface, at times you hear unsettling sounds such as war cries & grass rustling from behind you.
  • You may ignore this if you wish and continue playing.
  • However, if you look around, the scene transitions, surreally, noticing that the sounds get louder as you turn to the source and hear… animal noises? A fading in of a soundtrack? A bamboo forest landscape!? As you turn completely around, you enter a daydream: fighting ninjas.

Grab the sword. Swing it, thrust it, throw it. It’s virtual realityyy maaaan. It will feel like you are a level 90 Warrior taking on level 1 murlocs. Pure pwnage. 

Interactive WebVR concept of the party scene and transition to the daydream (use your mouse to look around on desktop or move your device around on mobile!)

  • Active presence utilized here with ability to use the controller or hands to fight ninjas
  • Embodied presence utilized here with visible body and godly-power use of hands

Your gaze activates a smooth transition of scenes from reality to daydream.

  • To use a metaphor: your eyes are a paintbrush and a new scene is painted with each stroke of a continuous gaze
  • Indicated through use of spatial audio that comprise of sound cues to promote looking around 

To The Moon is a story-driven virtual reality experience about two doctors traversing backwards through a dying man’s memories to artificially fulfill his last wish.

 

 

Disclaimer: This a fan-made version of the original To The Moon by Kan Gao who owns all copyright and intellectual property of the title. Go buy it! I don't aim to make this project commercialized, just a huge fan of the game. 

A full synopsis, script, and storyboard can be found here. The storyboard that complements the script allows you to see how the scene is meant to be portrayed in a 360 environment.

360 splash screen showing a birds eye view of the where the majority of the game takes place: The Abandoned Lighthouse and the House on a Cliff.

  • The camera icon in each panel represents your position as the player in the scene and your resulting perspective for viewing the 360 world.
  • The interaction events (IX), pink icons, represent interactable objects that the player can gaze at and select for further information. 

 

Why make this?

  • The purpose of developing a VR version of To The Moon is to harness the virtual reality medium and the nature of the narrative to achieve a transformative level of immersion that shows off the power of VR storytelling.
  • Achieving full immersion in this experience will be a marriage between the reactive storytelling and your suspension of disbelief by role-playing with two characters throughout the narrative arc: Dr. Neil Watts and Eva Rosalene

Reactive storytelling can be seen with the other protagonist, Eva. With no other characters to directly interact with, she is our link to social presence as we get natural social reactions from her in certain scenes by:

  • staring at her
  • looking away from her at times when she’s speaking to you
  • being idle (not interacting with anything for awhile) while she’s around
  • comments she makes on things that you are interacting with

Many scenes involve situations of the scripted events not starting until after 30s/1min passes in order to let the player absorb certain atmospheric places. For example, scene 1 starts with you getting situated and beginning to immerse yourself with music playing on the radio and banter with Eva before the story begins officially with a sudden deer crash. 

Interactive WebVR concept art for the first scene (use your mouse to look around on desktop or move your device around on mobile!)

  • Emotional presence is achieved by the plot progression and the soundtrack/SFX playing a huge role in setting atmosphere of scenes such as the piano tracks and ambient sounds such as ocean waves.
  • Embodied presence is achieved with your body being shown and scripted to react based on dialogue (see: I Am You) during non-interactive cutscenes.
  • For example, the hands will animate in response to your character’s dialogue/situations (eg facepalm in scene 70)
  • During interactive scenes, the hands will be idle since you are essentially non-existent in the story’s world except with the mementos/memory links