For demos that I work on, there are three simple things that I always do to create a better user experience:
make sure the Oculus health and safety warning (HSW) has been dismissed before any other information is displayed
give the user the option to re-center their avatar after they’ve put the headset on and settled in a comfortable position
encourage the user to create a profile
If you are a regular reader of this blog, you know this isn't the first time I've covered thesetopics. However, now that Unity includes native VR support I wanted to revisit these tips and show how to do them with Unity 5.2.1, the 0.7 runtime, and the Oculus Utilities for Unity 0.1-beta package.
Knowing when the HSW has been dismissed
The HSW is a big rectangle centered in front of the user. While it is semi-transparent, it blocks the user’s view significantly. For that reason, I like to make sure that is has been dismissed before displaying anything the user needs to interact with.
The Oculus Utilities for Unity 0.1-beta package provides a way to check to see if the HSW is still displayed:
OVRManager.isHSWDisplayed
This returns true when the HSW is displayed and false when it is not. Note that if you are running the application in the Editor, you won't see the HSW in the Editor window, however, the HSW will appear in the Rift view.
Re-centering the avatar
It is helpful to give users the option to re-center their virtual selves and this is one of the functions available as part of the Unity native VR implementation.
To re-center, add the VR name space and use:
VR.InputTracking.Recenter()
Encouraging the user to create a profile
If the user has not created a profile, the user may experience discomfort because the default IPD or height is different from their own. In previous versions, you could get the name of the profile in use, and if the name returned was "default," you would know that the user had not created a profile. Unfortunately, I’m not seeing a way to do that with the Oculus Utilities for Unity 0.1-beta package. I tried using OVRManager.profile.username, but itreturns "Oculus User" and not the name from the user profile. Looking at OVRProfile, it appears that only ipd, eyeHeight, and eyeDepth are retrieved from the current user profile (Note: you can get the user's eye height and IPD using OVRManager.profile.eyeHeight and OVRManager.profile.ipd.)
For now, I’ll just stress the importance of creating a profile in the documentation. If anyone knows how to get the current user name, please comment!
A common interaction in real life is to raise your hand to get someone’s attention. We do it when we are meeting someone in a crowd to help them find us, we do it when we are at school to get the teacher’s attention, and we do it as parents to get our child’s attention so they know that we are there and watching. We also do it when we want to hail a cab or make a bid at an auction. It is a simple enough interaction that babies do it almost instinctively. As simple as raising your hand is, using it as a mechanic in a VR environment brings up some interesting questions. How high should the user raise their hand to trigger the interaction? How long does the user need to have their hand raised? And, what should happen if the application loses hand tracking?
To experiment with this interaction, I created a demo consisting of a single character idling, minding his own business. When the user raises their hand, the character waves back and a speech bubble appears saying “Hello, there!”
Let’s take a look at the demo setup and then look at how testing the user experience went.
For the character animation, I used the Idle and Wave animations from the Raw Mocap data package for Macanim by Unity Technologies (free animations created from motion capture data) and I created an animation controller for the character to control when he is idling and when he waves back at you. The animation controller has two animation states, Idle and Wave. It also has two triggers that can be used to trigger the transition between each state:
The animation controller for the waving character has two states and two triggers.
And, of course, I wrote a script (wavinghello.cs) to detect when the user has raised their hand. The interesting bit of this script is how you know where the user’s hands are and how you know when a hand has been raised high enough so that you can trigger the appropriate animation. Let's take a look at the script's Update() function:
To get the all of the hands in the scene, the script uses GetAllPhysicsHands()
from HandController.cs:
HandModel[] userHands = handController.GetAllPhysicsHands(); GetAllPhysicsHands() returns an array of all Leap physics HandModels for the specified HandController. To get each hand's position, the script uses GetPalmPosition() which returns the Vector3 position of the HandModel relative to the HandController. The HandController is located at 0, 0, 0 relative to its parent object, the CenterEyeAnchor.
The HandController is a child of the CenterEyeAnchor.
The HandController is located at 0, 0, 0 relative to its parent the CenterEyeAnchor.
The CenterEyeAnchor object is used by the Oculus Rift integration scripts to maintain a position directly between the two eye cameras. As the cameras are the user’s eyes, if the Y value of a HandModel object's position is greater than the Y value of the centerEyeAnchor, we know the user's hand has been raised above eye level.
The user experience
When testing this demo I was looking at how high the user should raise their hand to trigger the interaction, how long the user should have their hand raised, and, what the application should do when it loses hand tracking. Initially, I went with what seemed comfortable for me. I required the users to raise their hand (measured from the center of their palm) to above eye level and I did not require the user's hand to be raised for any specific amount of time. If the Leap lost hand tracking, the application treated it as though all hands were below eye level.
I then grabbed three people to do some testing. The only instruction I gave them was to “raise your hand to get the guy’s attention.” For my first user, the demo worked quite well. He raised his hand and the character waved back as expected. Great so far. My second user was resistant to raising his hand any higher than his nose. He quickly got frustrated as he could not get the guy’s attention. My third user raised his hand and then waved it wildly around so much so that the speech bubble flickered and was unreadable. Quite a range of results for only three users.
For my next iteration, I set the threshold for raising one’s hand a few centimeters below eye level.
models.GetPalmPosition().y >= centerEyeAnchor.transform.position.y - 0.03f
This worked for my second user as it was low enough that he would trigger the interaction, but not so low that he would accidentally trigger it.
I haven’t done anything to address the third user yet, but whatever I do, waving my hands like a maniac is now part of my my own testing checklist.
I’d love to hear if anyone else is using this type of mechanic and what their experiences are.
Immersion is definitely affected by how closely your avatar's hand looks like your own. In the demo I am working on I want the user to be able to select the hands they have in the game before entering the game. A prototype in-game UI for hand selection is seen in the video below.
To create this UI, I created a world space canvas and added buttons for each of the available hands. To each button, I added a box collider as a child object. A script attached to the box collider detects when a hand has collided with it. To detect a hand, I used the Leap libraries and then checked to see if the collision object is a Leap HandModel.
In this prototype UI, I am using large buttons for two reasons. First, reading small text in the Rift can be difficult, and second, while using the Leap allows me to see my hands, in my experience, it does not track finger motion well enough for detailed interactions to be effective. In several of the tests I ran, the user's hand was generally in the right place but the fingers more often than not were at different angles than the user's actual hand. The effect was that my users seemed to have the fine motor skills of a toddler - they could reach out and touch everything but they didn't have a lot of control. On the positive side, when the user has hands in the game, it appears to be very natural for users to try to touch items with their hands. Even when users don't have visible hands in the scene, you'll often see them reaching out to try to touch things. While I have the start button say "Touch to Start," once users know to use their hand to affect the scene they get it right away and don't need prompting or other instruction.
Leap Motion has just released a "Best Practices Guide" and I'll be looking at incorporating many of the ideas documented there in future prototypes.
In a previous post, I talked about creating GUIs for VR using world space canvases. In that example, the GUI only displayed text - it didn't have any input components (buttons, sliders, etc). I wanted to add a button above each thought bubble the user could click to hear the text read aloud.
As I had used a look-based interaction to toggle the visibility of the GUI, this brought up the obvious question of how do I use a similar interaction for GUI input? And, importantly, how do I do it in a way that takes advantage of Unity's GUI EventSystem?
Turns out, what's needed is a custom input module that detects where the user is looking. There is an excellent tutorial posted on the Oculus forums by css that is a great place to start. That tutorial includes the code for a sample input module and walks you through the process of setting up the GUI event camera. (You need to assign an event camera to each canvas and one twist is that the OVR cameras don’t seem to work with the GUI.) By following that tutorial, I was able to get look-based input working very quickly.
Note that while look-based interactions are immersive and fairly intuitive to use, it is worth keeping in mind that look-based input won’t work in all situations. For example, if you have attached the GUI to CenterEyeCamera to ensure that the user always sees the GUI, the GUI will follow the user’s view meaning the user won’t be able to look at any one specific option.
Let’s say you wanted to have a TV screen that plays a short welcome video on start up in your scene, such as in this demo I'm working on:
Displaying video on a screen in a scene in Unity Pro is typically done using a Movie Texture. Movie Textures do not play automatically - you need to use a script to tell the video when to play. The Rift, however, presents some challenges that you wouldn’t face when working with a more traditional monitor that make knowing when to start the video a bit tricky.
You can’t assume that the user has the headset on when the application starts. This means you can’t assume that the user can see anything that you are displaying.
On start-up all Rift applications display a Health and Safety Warning (HSW). The HSW is big rectangle pinned to the user’s perspective that largely obscures the user’s view of everything else in the scene.
You aren’t in control of the where the user looks (or rather, you shouldn’t be - moving the camera for the user can be a major motion sickness trigger), so you can’t be sure the user is even looking at the part of the scene where the video will be displayed.
In my demo, I addressed the first two issues by making sure the HSW had been dismissed before I started the video. If the user has dismissed the HSW, it will no longer be in the way of their view and it is a good bet that if they dismissed the HSW, they have the headset on and are ready to start the demo. The third issue I addressed by making sure the video is in the user’s field of view before it starts playing.
Making sure the Health and Safety Warning (HSW) has been dismissed
The HSW says “Press any key to dismiss.” My first thought was to use the key press as the trigger for starting the video. Unfortunately this doesn’t quite work. The HSW must be displayed for a minimum amount of time before it can actually be dismissed - 15 seconds the first time it is displayed for a given profile and 6 seconds for subsequent times. The result was that often the key was pressed and the welcome video would start but the HSW had no yet gone away. I also wanted the video to replay if the user reloaded the scene. When the scene is reloaded, the HSW is not displayed, the user does not need to press a key and therefore the video would not start.
Fortunately, Oculus Unity Integration package provides a way to know if the HSW is still being displayed or not.
OVRDevice.HMD.GetHSWDisplayState().Displayed
The above will return true if the HSW is still on screen.
Making sure the video is in the player’s field of view
How you get the user to look at the video will depend a lot on what kind of scene you are using. You can, for example, put a TV in every corner of the room so that no matter which direction the user is looking, a screen is in view. Or, if you have only a single TV screen, you can use audio cues to get the get the user’s attention. (I haven't decided yet how I will get the user's attention in my demo.)
No matter how you get the player to look at where the video is playing, you can check that the video is within the user’s field of view by checking the video’s render state before playing the video using:
Everyone who knows I own a Rift wants to try it out. In fact, I can’t count how many demos I’ve given since getting it last October. Aside from being really fun to see that "wow" moment when someone first moves their head, it has also given me some good ideas for VR usability. Unfortunately, with all the demos I did, I wasn't writing anything down. Time to stop screwing around and do some science.
About
These notes include reactions from 26 first time Rift users (14 women, 12 men), ranging in age from 11 to 84.
The demos tried were Tuscany, Don’t Let Go, Meld Media GigantosaurusVR, Chicken Walk, Dino Town, Proton Pulse, TNG Engineering, Enterprise and Fortune Teller (my own work-in-progress). The PS3 controller was used with Tuscany, Chicken Walk, and Enterprise. Keyboard with Don’t Let Go, Tuscany, and TNG Engineering. And, nothing is required for Meld Media GigantosaurusVR and Proton Pulse. Time spent using the Rift was between 5 and 10 minutes per person.
Of the 26:
6 would consider themselves to be gamers.
8 said they get motion sickness easily.
Observations
6 did not look around during the demo until prompted.
14 felt symptoms of motion sickness.
7 of the 8 who had said they get motion sickness easily reported symptoms while using the Rift.
Everyone who used the keyboard during a demo needed help finding the keys more than once.
2 gamepad users had trouble using the gamepad. All others had no issues.
12 commented on having an avatar or lack of avatar (depending on the demo used).
User comments
On motion sickness
“Felt motion sickness when walking along the curved wall [of the Enterprise Bridge] but not elsewhere.”
“Down the stairs was nauseating.”
Turning around was very nauseating”
“Felt motion sick all of a sudden. I thought my body should have hit something going through the door but it didn’t.” [Note: this was a very large/wide man.]
“Looking in one direction while moving in another direction was the most nauseating.”
On having an avatar/ lack of avatar
“I felt comfortable being an orb in space but it was like I was viewing video from a hovercraft. In Don’t Let Go the body was very convincing.”
“I have a shoulder!” Said by a smallish woman. This was after the knives in Don’t Let Go.
“Where are my legs. This is weird!”
"Found my shirt!"
“Very visceral” “That is some messed up stuff”[said regards to the spider]
“I have no body! This is so funny!"
“Cool! I can see myself!
On what they loved
“I love the butterflies!” (3)
“Oh, are those birds? How cool!”
“I felt totally immersed.”
Other
“I didn’t think to look around when looking at a screen.”
“I wish I could look at things closely.”
"Felt like I wanted to reach out and touch things”
“The only way to play is in a swivel chair.”
Note:This post was updated on June 23, 2014 to include the reactions of 6 additional test users. This will be the last of the test results for the DK 1.
Brad's post on on cut scenes and borrowing from cinematography for VR got me thinking about VR usability. When designing a VR experience, many of the standard usability checklist questions still apply:
Are users able to access the information they need?
Are users able to make progress?
Do the navigation mechanics feel natural?
Does accessing information or making progress take the user out of the experience? That is, are users able to concentrate on their goal and not on how to use the software?
Does the experience create an appropriate emotional response?
Along with the more standard questions, for VR, you also need to add the literal question:
Does the experience make the user want to vomit?
The techniques and conventions used to achieve the desired usability result is uncharted territory. So, without a map or other guidelines, how do you start to address these issues? One way to start is by borrowing from other mediums. Film and cinematography is one source. However, as Brad explained in another post, mapping film conventions 1:1 to VR does not work as some conventions, such as cut scenes and zoom, are simply too disorienting in an immersive environment. So while borrowing conventions is a good start, what we really need to do to build a map is experiment, iterate, and share what we know.
With that in mind, here is a roundup of some current resources:
