Purpose - Methodology - Head-Mounted Displays - Audience - Publications
Purpose
The purpose of the RAPID Lab is to evaluate commercial and government off-the-shelf Virtual Reality (VR) and Augmented Reality (AR) systems. Current evaluation methods include in-house technology evaluations and usability assessments, all of which provide empirically-driven results to inform device selection for military, industry, academic, and general consumer use. The lab continues to maintain horizon awareness of relevant virtual technologies.
Methodology: The 30-60-90+ Approach
RAPID Lab utilizes a three-phased approach to conducting technology evaluations for Head-Mounted Displays (HMDs). This 30-60-90+ approach (see Table 1) is defined by the approximate number of days spent conducting an evaluation, and the differing levels of effort necessary to meet desired goals. Each phase helps to inform a rating score.
Table 1. Three-Phase Technical Evaluation Layout
| 30-day Evaluation | 60-day Evaluation | 90-day+ Evaluation |
| Basic fact checking | General Task Testing | Specific Task Testing |
| Out-of-the-box testing | Sample Size of 1-5 | Sample Size of more than 20 |
| Informs Rating Score | Informs Rating Score | Informs Rating Score |
The 30-day Evaluation
The goal of the 30-day evaluation is to attain hardware capabilities, particularly from a technical specification perspective. This includes verification of listed technical specifications (i.e., mechanical and software capabilities), including (but not limited to) a device’s visual field-of-view, latency, refresh rate, and resolution.
The 60-day Evaluation
The goal of the 60-day evaluation is to provide an increased level of detail of hardware capabilities, by spotlighting how hardware factors may affect human interaction and performance. Rapid pilot testing is used to determine baseline human interaction results, and thus provide a truer picture of functionality from both a technical and user standpoint. General task testing helps inform field applications.
The 90-day+ Evaluation
The goal of the 90-day+ evaluation encompasses different approaches related to clarifying potential uses for HMDs and their related systems. For this effort level, the revised protocol from the 60-day evaluation accounts for an increased level of human participation and interaction with various technologies: a usability study provides observable and quantifiable metrics based on recommendations from the International Organization for Standardization (ISO). The usability results represent the utility of the system to stakeholders.
Head-Mounted Displays
FOVE Headset Microsoft HoloLens Headset HP Mixed Reality Headset
HTC Vive Headset Meta 2 Headset Oculus Rift Headset
Samsung Gear VR Headset Samsung Odyssey Headset XTAL Headset
ZED Mini (Add-on Stereo Camera)
Audience
| · Psychotherapists · Educators · Entertainers · Surgeons · First responders | · Product designers · Software developers · Human factors specialists · Usability experts · Research scientists |
Publications
Janney, M., Bekheit, V., Schaub, R., & Maraj, C. (2019, April). Assessing simulator sickness in the HTC Vive & Oculus Rift using a virtual walking task. Poster session presented at the University of Central Florida Showcase of Undergraduate Research Excellence (SURE), Orlando, FL. [View Poster]
Maraj, C., Hurter, J., Horde, L., & Ferrante, S. (in press). Oculus Rift versus HTC Vive: Usability assessment from a teleportation task. In Proceedings of the 21st International Conference on Human-Computer Interaction.
Maraj, C., & Hurter, J. (in press). A usability and workload investigation: Using video games to compare two virtual reality systems. In Proceedings of the 10th International Conference of Applied Human Factors and Ergonomics.
Murphy, S., & Maraj, C. (2018).The Realistic Assessment of Performance in Devices (RAPID) Laboratory: 30-day evaluation technical report. Orlando, Florida: Army Research Laboratory.
Murphy, S., & Maraj, C. (2018). ZED Mini positional and orientation accuracy summary. Orlando, Florida: Army Research Laboratory.
Murphy, S., & Maraj, C. (2019). The Realistic Assessment of Performance in Devices (RAPID) Laboratory: 30-day evaluation of the XTAL virtual reality head-mounted display. Orlando, Florida: Army Research Laboratory.
Murphy, S., Maraj, C., & Hurter, J. (2018). Field of view: Evaluation of Oculus Rift and HTC Vive [In-House Poster].