Skip to content

Virtual Reality Application implemented as part of my Master's degree thesis. Using: Oculus Rift DK2, Leap Motion, Unity 3D, Nvidia 3D Vision glasses, hand tracking, computer-human interaction Hands are an indispensable way for humans to interact with the environment in their daily lives. To incorporate the possibility that they can interact wit…

License

Notifications You must be signed in to change notification settings

GiovannyJTT/vr-app-handtracking

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

vr-app-handtracking

Video Link: VR Application

vr app hand tracking

  • Virtual Reality Application implemented as part of my Master's degree thesis (2016).

    • Using: Oculus Rift DK2, Leap Motion, Unity 3D, Nvidia 3D Vision glasses, hand tracking, computer-human interaction

    • Link: Master Degree Thesis "Comparison of 3D visualization: Oculus Dk2 vs Nvidia 3D Glasses"

    • Hands are an indispensable way for humans to interact with the environment in their daily lives. To incorporate the possibility that they can interact with their hands within a virtual world, it raises the degree of immersion and provides a form of natural interaction in virtual reality applications. In this final master work, we undertake the development of a virtual reality application in which stereoscopic visualization and hand interaction are used. At same time, we compare two modes of stereoscopic visualization to determine if participants cope with any of them better. We will observe if the results indicate that regardless of gender, age, or profession, participants prefer the mode with a virtual helmet

  • Requirements

Index

  • Introduction
  • Used tools
  • Application developed
  • Validations
  • Conclusions

vr app hand tracking

Introduction

Virtual Reality

  • “It is the product of the simulation of a physical world created from programs computers, computer systems, graphics engines, which gives the possibility interaction with humans through different sensors” (Sherman & Craig, 2003)
  • Virtual Reality Device (helmet)
  • Stereoscopy with Active Glasses

vr app hand tracking

Hand Interaction

  • Use the hands as a natural interface to interact with the virtual objects increases the degree of immersion
  • Users try to touch virtual objects
  • Keyboard or mouse: handling difficulty, reduces the level of immersion

vr app hand tracking

State of the Art

  • Interaction with hands and screen:
    • Many ways to complete the game (Christian et al. 2011)
    • Avoid arms stretched out for a long time (Schlattmann et al., 2011)
  • Interaction with hands and virtual reality headset:
    • Grasping pieces requires a complex algorithm (Lee et al., 2015)
    • Higher level of immersion (Lee et al., 2015)

vr app hand tracking

Project Goals

  1. Study the potential of Leap Motion for hand tracking
  2. Develop an app that combines visualization stereoscopic and hands-on interaction
  3. Validate the application with participants

vr app hand tracking

Tools

Description

  • The developed application contains 3 games using hands-on interaction and 3D visualization

vr app hand tracking

Unity 3D

  • Cross-platform, abundant information, free vs personal
  • Canvas (buttons, sliders, etc.)
  • Preview of scene seen from VR helmet
  • Import 3D models

vr app hand tracking

Leap Motion

  • 3 infrared LEDs, 2 cameras
  • Leap Motion Core Assets:
    • Graphic part and physical part
    • Parent-child hierarchy
    • Avoid collisions between bones
    • Frame: data structure containing information geometry on the hands

vr app hand tracking

Oculus Rift

  • OVRCameraRig

    • LeftEyeAnchor
    • RightEyeAnchor
    • Center Anchor
      • HandController
  • HandController

    • System of coordinates adjusted for your position on the head

vr app hand tracking

Nvidia 3D vision

  • Supported graphics card DirectX 11
  • Dual link DVI cable
  • Main camera:
    • Static (no tracking positional)
    • Field of vision according to the distance of objects
    • Stereo separation
    • Convergence distance

vr app hand tracking

Blender, Gimp, Reaper

vr app hand tracking

Blender

  • Union, intersection, difference
  • Chamfered on the edges
  • System of FBX coordinates are adjust automatically in unity

Gimp

  • Histograms by channels
  • Smart scissors
  • Brushes, buckets of painting
  • Brightness and Contrast

Reaper

  • cut fragments of Audio
  • Record instructions
  • Remove background noise
  • Equalizer (boost the voice)
  • Compressor (avoids saturation in the recordings)

Application Developed

Description

  • Database Manager
  • Application Manager complete
  • Manager of the Graphic Interface of User
  • Museum Game Manager
  • Soccer Game Manager
  • Tower Game Manager

vr app hand tracking

Data Base Manager

  • Dynamic lists to avoid temporary delays to update the db while play

vr app hand tracking

App Manager

  • Information about the session current game:

    • Current person
    • 3D type
    • Date
    • Etc.
  • Features common to several modules:

    • Set up:
      • Cameras depending on the type of 3D
      • Speed of the hands in the demonstrations
      • Appearance of instructions in text

vr app hand tracking

Graphical User Interface

vr app hand tracking

Game Museum

vr app hand tracking

Game VolleyBall

vr app hand tracking

Game Tower

vr app hand tracking

Validations

Description

vr app hand tracking

Time used

  • There are statistically significant differences
    • Student's t: p-value < 0.05
    • C_Oculus: median of 80 (and range interquartile of 30)
    • C_Glasses: median of 115 (and range interquartile of 26)

vr app hand tracking

Interaction, Ergonomics and 3D quality, 1st time

  • Oculus gets a superior rating (close to 7 – “Strongly agree”)

vr app hand tracking

Comparison - Interaction

  • Group A values both devices similarly

vr app hand tracking

Comparison – 3D Quality

  • There are no statistically significant differences
  • The order of use of the devices has not influenced the evaluation of the quality of the 3D

vr app hand tracking

Comparison - Preferences

  • The difference in comfort is not as great as in the other questions

vr app hand tracking

Comparison – Preferences (gender, profession, age)

  • Neither gender, nor profession, nor age, influence the final preference of the participants

vr app hand tracking

Conclusions

  • We have developed a VR application that allows hands-on interaction and includes 2 types of 3D
  • We have carried out a study in which we have determined that one of the devices favors a better interaction
  • Feedback to the user through instructions, effects sounds, color changes of objects, etc., has prevented this from being look lost

vr app hand tracking

Future work

  • Validate the application with a larger number of participants
  • Possibility of moving the camera depending on the movement of the hands
  • Possibility of incorporating machine learning to detect gestures complex like drawing letters in the air

vr app hand tracking

About

Virtual Reality Application implemented as part of my Master's degree thesis. Using: Oculus Rift DK2, Leap Motion, Unity 3D, Nvidia 3D Vision glasses, hand tracking, computer-human interaction Hands are an indispensable way for humans to interact with the environment in their daily lives. To incorporate the possibility that they can interact wit…

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published