👁️HUD UI Framework

HUD (from Heads Up Display) is a smart contextual layer that is drawn before the eyes of the user.

HUD can be toggled on and off, and provides additional information and features that leverage the ‘technological’ nature of virtual reality.

HUD is the ‘augmented reality’ for Morpheus' virtual reality.

Why Do We Need HUD

While we steer close to ‘real reality’, we could make it smarter: more convenient, more helpful.

Each object has its own ‘information underlay’ — also in reality — which is not visible in the physical world, but with help of devices it becomes visible as ‘augmented reality’. In VR, we can bring this ‘information layer' right in front of the user — making it an ‘overlay’ instead of ‘underlay’.

HUD is something that only the user sees. Other persons in the same virtual world don’t see the user’s HUD. It is convenient to think of HUD as of Google Glasses of VR.

As Google Glasses, our HUD should be optional. Regular participants should see virtual reality ‘as it is’ most of the time, toggling HUD on only to get a hint about someone’s name. Advanced users like facilitators will make more use of HUD.

We should not overload user’s frame of attention. Otherwise it will be an ‘advertising hell’ so well pictured in many movies.

How Morpheus HUD Works in VR

• By default, HUD if off. It can be toggled on/off with a hardware button.

• Goals of the HUD:

  1. Enhance user’s experience with contextual information.

     • E.g., help the user with the names of other users.

  2. Provide optional convenience shortcuts for actions that are cumbersome in real life.

     • E.g., allow to turn the light on/off from certain distance without getting close to the light switch.

  3. Provide access to advanced contextual controls.

     • E.g., spawning of artifacts, moving and resizing of decorations and furniture.

• Elements of HUD are always:

  • Visible over ‘real’ objects. HUD is between the user and the world. No objects of the world can be in front of a HUD element. Even of a name tag.

  • At comfortable distance. The elements of HUD are never shown too close or too far.

  • Facing the user. Even if an object it looking sideways, the corresponding HUD element is facing the user thus making it easy to target and use.

  • Readable. They can change scale depending on distance, but at certain distance, when the text of the control isn’t readable anymore, there is no value in showing the control at all. E.g., nametags should not be visible at the distance more than 20–30 meters.

Different HUD Elements

There are two kinds of HUD elements:

• Passive notifications and contextual information.

• Active controls with which the user can interact by clicking trigger.

Passive HUD Elements

Passive elements are always visible when the HUD is on. And status bar is visible even if the HUD is off.

	Name Tags	Laser Hands	Status Bar
Purpose	Show names of other users as well as additional contextual information (photos, etc.)	Allow to interact with objects on larger distances. E.g., pick objects up, toggle and select them.	Show the status of the user herself. E.g., is the user muted.
Looks	Semitransparent badges floating over heads of other users.	Thin rays of light projected from hands. The rays become active when pointed to an object.	Small rounded bar.
Position	Above avatars' heads.	Projected from user’s hands.	At the eyebrow level of the user’s view.
Following gaze	Following other avatars.	Following controllers.	Strictly following the movement of the headset.
Visibility	Always when HUD is on.	Always when HUD is on.	Always visible regardless HUD on/off.

Active HUD Elements

Active elements expect that the user will interact with them.

They are usually rendered as set of buttons or menus that are of a fixed size relative to the user’s view.

At any given moment of time:

• Only one ‘Element Control’ can be visible.

• Only one ‘Toolbar Control’ can be visible.

• Only one ‘Modal Dialog’ is visible.

• NB: When a modal dialog is visible, other controls are not active (and should be hidden). Nametags could also be hidden if it improves performance.

	Element Controls	Toolbar Controls	Modal Dialogs
Purpose	Provide access to advanced contextual functionality of a given object. E.g., flipping slides on a slide screen.	Provide access to advanced functionality of the object that is held or worn. E.g., spawning and moving of furniture and decorations.	Force user to choose something or enter text. E.g., select folder to connect with a slide screen.
Looks	Large block of controls that usually comprises a meaningfully arranged set of buttons. E.g., slide controls.	Set of large buttons that reflect different modes of the given tool. E.g., spawn control (we have only one for now).	Large rectangular window or a 360° semitransparent layer that prohibits any interaction with the world. E.g., ‘spawn’ and ‘connect folder’ dialogs.
Size	Enough to make texts readable and clickable elements easy to click; but not larger. Fixed relative to the user’s view. Doesn’t change if user moves closer/far away.	Large enough buttons (larger than element controls). Fixed relative to the user’s view. Doesn’t change if user moves closer/far away.	Usually cover a sizeable portion of the user’s view. Fixed relative to the user’s view. Doesn’t change if user moves closer/far away.
Position	A bit below or above the selected object. Appear to be ‘almost close’ to user. E.g. for Slide Screens it is below. E.g. for Loudspeakers it could be above.	About 60° downwards from the center view. Appear to be ‘close’ to user.	Appear to be ‘close’ to user.
Following mode	Following the selected object.	Slowly following user’s view when the user moves a lot.	Slowly following user’s view when the user moves a lot.
Visibility	Only when an object is selected.	Always when HUD is on.	Only when entered the dialog.

Deeper Dive: Laser Hands

We use the UI pattern which is common to VR:

• thin beams are projected from the user’s controllers,

• those beams allow the user to interact with objects and controls without being right next to them.

We call those beams ‘laser hands’.

Laser hands are visible only when HUD is on; they can be in one of the three states: Passive, Active, and Scroll.

	Passive (default)	Active	Scroll
Conditions	HUD is on No modal dialog, nothing is pointed at OR modal dialog, pointing outwards.	HUD is on Pointing to an active HUD element. or Pointing to an object that has its own Element Controls.	HUD is on Active modal Pointing inside of the modal but not at one of the active elements
Look and feel	Thin white semi-transparent, gradually losing alpha by distance for the hand.	Thick brightly coloured non-transparent. If pointed to an active HUD element, the HUD element itself is highlighted.	Thin white semi-transparent, not losing alpha by distance from the hand.
Action on trigger	Hides any Element Controls if they are currently visible. Closes the modal dialog if in it.	HUD element is activated (e.g., buttons pressed). Environment objects are selected and their Element Controls are shown.	Scrolls the contents of the modal dialog.
Action on Grab	—	Pulls close and grabs the selected object When in Spawn Modal, spawn-grabs the object.	—