Realistic Assessment of Performance In Devices (RAPID) Lab




Purpose  -  Methodology  - Head-Mounted Displays -  Audience  -  Publications


Purpose

The purpose of the RAPID Lab is to evaluate commercial and government off-the-shelf Virtual Reality (VR) and Augmented Reality (AR) systems. Current evaluation methods include in-house technology evaluations and usability assessments, all of which provide empirically-driven results to inform device selection for military, industry, academic, and general consumer use. The lab continues to maintain horizon awareness of relevant virtual technologies.

Methodology: The 30-60-90+ Approach

RAPID Lab utilizes a three-phased approach to conducting technology evaluations for Head-Mounted Displays (HMDs). This 30-60-90+ approach (see Table 1) is defined by the approximate number of days spent conducting an evaluation, and the differing levels of effort necessary to meet desired goals. Each phase helps to inform a rating score.

Table 1. Three-Phase Technical Evaluation Layout

30-day Evaluation

60-day Evaluation

90-day+ Evaluation

Basic fact checking

General Task Testing

Specific Task Testing

Out-of-the-box testing

Sample Size of 1-5

Sample Size of more than 20

Informs Rating Score

Informs Rating Score

Informs Rating Score


The 30-day Evaluation

The goal of the 30-day evaluation is to attain hardware capabilities, particularly from a technical specification perspective. This includes verification of listed technical specifications (i.e., mechanical and software capabilities), including (but not limited to) a device’s visual field-of-view, latency, refresh rate, and resolution.

The 60-day Evaluation

The goal of the 60-day evaluation is to provide an increased level of detail of hardware capabilities, by spotlighting how hardware factors may affect human interaction and performance. Rapid pilot testing is used to determine baseline human interaction results, and thus provide a truer picture of functionality from both a technical and user standpoint. General task testing helps inform field applications. 

The 90-day+ Evaluation

The goal of the 90-day+ evaluation encompasses different approaches related to clarifying potential uses for HMDs and their related systems. For this effort level, the revised protocol from the 60-day evaluation accounts for an increased level of human participation and interaction with various technologies: a usability study provides observable and quantifiable metrics based on recommendations from the International Organization for Standardization (ISO). The usability results represent the utility of the system to stakeholders.

Head-Mounted Displays

          

FOVE Headset                                  Microsoft HoloLens Headset           HP Mixed Reality Headset

          

HTC Vive Headset                         Meta 2 Headset                           Oculus Rift Headset

Audience

· Psychotherapists

· Educators

· Entertainers

· Surgeons

· First responders

· Product designers

· Software developers

· Human factors specialists

· Usability experts

· Research scientists


Publications

Maraj, C., Hurter, J., Horde, L, & Ferrante, S. (2018). Oculus Rift versus HTC Vive: Usability assessment from a teleportation task. Manuscript in preparation.

Murphy, S. & Maraj. C. (2018). The Realistic Assessment of Performance in Devices (RAPID) Laboratory: 30-Day evaluation technical report. Orlando, Florida: Army Research Laboratory.

Murphy, S., Maraj, C., Hurter, J.  (2018). Field of view: Evaluation of Oculus Rift and HTC Vive [In-House Poster]. 

  

 


 


 


Team

Research Assistant Professor
Crystal Maraj, Ph.D.
407-384-5286
cmaraj@ist.ucf.edu

Assistant in Research Software Development

Sean Murphy, M.S.

407-882-3628

smurphy@ist.ucf.edu


Research Assistant & Technical Writer
Jonathan Hurter, M.S.
407-882-2116
jhurter@ist.ucf.edu

Graduate Research Assistant
Alfred Schaub

Undergraduate Research Assistants
Veronica Bekheit
Jasmine Carter
Schuyler Ferrante
Lauren Horde
Morgan Janney