Realistic Assessment of Performance In Devices (RAPID) Lab



 

Purpose  -  Methodology  - Head-Mounted Displays -  Audience  -  Publications



Purpose

The purpose of the RAPID Lab is to evaluate commercial and government off-the-shelf Virtual Reality (VR) and Augmented Reality (AR) systems. Current evaluation methods include in-house technology evaluations and usability assessments, all of which provide empirically-driven results to inform device selection for military, industry, academic, and general consumer use. The lab continues to maintain horizon awareness of relevant virtual technologies.

 

Methodology: The 30-60-90+ Approach

RAPID Lab utilizes a three-phased approach to conducting technology evaluations for Head-Mounted Displays (HMDs). This 30-60-90+ approach (see Table 1) is defined by the approximate number of days spent conducting an evaluation, and the differing levels of effort necessary to meet desired goals. Each phase helps to inform a rating score.

Table 1. Three-Phase Technical Evaluation Layout

 

30-day Evaluation

60-day Evaluation

90-day+ Evaluation

Basic fact checking

General Task Testing

Specific Task Testing

Out-of-the-box testing

Sample Size of 1-5

Sample Size of more than 20

Informs Rating Score

Informs Rating Score

Informs Rating Score

 

The 30-day Evaluation

The goal of the 30-day evaluation is to attain hardware capabilities, particularly from a technical specification perspective. This includes verification of listed technical specifications (i.e., mechanical and software capabilities), including (but not limited to) a device’s visual field-of-view, latency, refresh rate, and resolution.

The 60-day Evaluation

The goal of the 60-day evaluation is to provide an increased level of detail of hardware capabilities, by spotlighting how hardware factors may affect human interaction and performance. Rapid pilot testing is used to determine baseline human interaction results, and thus provide a truer picture of functionality from both a technical and user standpoint. General task testing helps inform field applications. 

The 90-day+ Evaluation

The goal of the 90-day+ evaluation encompasses different approaches related to clarifying potential uses for HMDs and their related systems. For this effort level, the revised protocol from the 60-day evaluation accounts for an increased level of human participation and interaction with various technologies: a usability study provides observable and quantifiable metrics based on recommendations from the International Organization for Standardization (ISO). The usability results represent the utility of the system to stakeholders.

Head-Mounted Displays

FOVE     HoloLens     HPmixedreality

FOVE Headset                                      Microsoft HoloLens Headset           HP Mixed Reality Headset

HTCVive     META2     OculusRift

HTC Vive Headset                                Meta 2 Headset                                  Oculus Rift Headset

SamsungGear     SamsungOdissey     XTAL

Samsung Gear VR Headset               Samsung Odyssey Headset               XTAL Headset

ZEDmini     

ZED Mini (Add-on Stereo Camera)

Audience

· Psychotherapists

· Educators

· Entertainers

· Surgeons

· First responders

· Product designers

· Software developers

· Human factors specialists

· Usability experts

· Research scientists


Publications

Janney, M., Bekheit, V., Schaub, R., & Maraj, C. (2019, April). Assessing simulator sickness in the HTC Vive & Oculus Rift using a virtual walking task. Poster session presented at the University of Central Florida Showcase of Undergraduate Research Excellence (SURE), Orlando, FL. [View Poster]

Maraj, C., Hurter, J., Horde, L., & Ferrante, S. (in press). Oculus Rift versus HTC Vive: Usability assessment from a teleportation task. In Proceedings of the 21st International Conference on Human-Computer Interaction.

Maraj, C., & Hurter, J. (in press). A usability and workload investigation: Using video games to compare two virtual reality systems. In Proceedings of the 10th International Conference of Applied Human Factors and Ergonomics.

Murphy, S., & Maraj, C. (2018).The Realistic Assessment of Performance in Devices (RAPID) Laboratory: 30-day evaluation technical report. Orlando, Florida: Army Research Laboratory.

Murphy, S., & Maraj, C. (2018). ZED Mini positional and orientation accuracy summary. Orlando, Florida: Army Research Laboratory.

Murphy, S., & Maraj, C. (2019). The Realistic Assessment of Performance in Devices (RAPID) Laboratory: 30-day evaluation of the XTAL virtual reality head-mounted display. Orlando, Florida: Army Research Laboratory.

Murphy, S., Maraj, C., & Hurter, J. (2018). Field of view: Evaluation of Oculus Rift and HTC Vive [In-House Poster].

  


Team

Research Assistant Professor
Crystal Maraj, Ph.D.
407-384-5286
cmaraj@ist.ucf.edu

Assistant in Research Software Development

Sean Murphy, M.S.

407-882-3628

smurphy@ist.ucf.edu


Research Assistant & Technical Writer
Jonathan Hurter, M.S.
407-882-2116
jhurter@ist.ucf.edu

Undergraduate Research Assistants
Amber Blatchford
Melissa Marques
Joey Pruitt
Hannah Singer