Q. When you look back at MS&T in the 1980s, what was your expectation/hope then for what the technology or field might achieve?

A. Hank Okraski: In the 1980s, the MS&T community identified areas of training research that were based on known requirements or a technology showing promise for the future. Research initiatives within these areas were funded and underway by the services and through industry’s independent research and development funding. Critical areas of research in the 1980s included training effectiveness, instructional systems design and development, visual systems, embedded training, voice recognition, deployable trainers, weapons training, improved computer architecture and aircraft Cockpit Resource Management. Artificial Intelligence (AI) was in its infancy. Areas such as visual systems, voice recognition and computer architecture received a huge boost from the commercial world for high-demand consumer products. Consequently, for those areas identified and pursued, the direction of technology transfer was reversed. Commercial products developed for entertainment or other applications were adopted by the military in their research initiatives, saving time and costs.

Q. What were IST’s greatest breakthrough achievements during the 1980s, and why were they considered important for the time?

A. Charlie Hughes: In the late 80s and early 90s, researchers at UCF contributed to some of the earliest work on the use of virtual environments to support military training. The impetus for this was the placement of two SIMNET nodes (M1 Abrams Tank Simulators) at IST in 1988. Users of the SIMNET simulator were immersed in a physical replica of the compartments of the Abrams M1 tank where operators and others were situated. While in the simulators, you had the ability to view activities in a field of combat, in which all tanks were represented by virtual replicas. All tanks were controlled by people in SIMNET stations. They see your virtual tank and you see theirs, both on a shared virtual landscape. This joint experience works even if participants are physically located on other continents. This was an early, large-scale distributed virtual reality (VR) experience.

The presence of SIMNET at IST resulted in the creation of the first IST VR lab, the Visual System Lab (VSL), led by Mike Moshell. This lab gave rise to many subsequent VR labs, including MCL, IRL, SREAL, and E2i. Basic research included work on network protocols, dynamic terrain (ground and even water that changed based on interactions with the simulation, e.g., virtual tank movement leaving visible tracks on the ground) and user studies of effectiveness. VSL was also an incubator, housing the “Toy Scouts of America,” where high school students came every Friday night to learn to program, develop visual assets and create virtual experiences using professional tools. Mike, the “scoutmaster,” Dan Mapes, and our resident artist, Jacki Morie, helped these students create engaging experiences, some of which were presented at high impact conferences. Many of the toy scouts later became VR researchers.

In 1993, Mike Moshell and Charlie Hughes, funded by the Department of Defense Dependency Schools (now called DODEA), created a shared virtual environment, ExploreNet, that allowed middle schoolers in Florida to mentor third-graders in Germany within virtual worlds. ExploreNet was also used as a vehicle to introduce young people to stories authored by Zora Neale Hurston.
Access to SIMNET and the opportunity to contribute to its development helped IST and UCF to become a leader in basic VR research and its applications, especially in education and training.

Q. During this decade, what were the emerging trends or work in its infancy at IST or elsewhere that would later significantly affect the industry?

A. Hank Okraski: The SIMNET Program was an enabling initiative for future simulator networking and the Live-Virtual-Constructive paradigms. The Multiple Integrated Laser Engagement System (MILES) was the technology that led to improved marksmanship and weapons team engagement simulation. Cockpit Resource Management and Tactical Decision Making Under Stress were foundational programs for team training in air, surface, sub-surface and land warfare. It also had influence on medical team performance. Research in head-mounted displays in the virtual world led to improved and more realistic simulations to include augmented and mixed reality. The institutionalizing of the Instructional System Design process moved the focus on simulator procurements from hardware to human performance. AI technology was rather primitive in those days, but the concepts were being integrated into training system design in a very basic sense. For example, in land warfare simulations, adversary platform tactics were dictated by the “own ship” behaviors. For the Navy, initiatives were underway for bringing training and mission rehearsal to the Fleet in the form of deployable trainers.

Q. Locally, who stands out during this decade as a pioneer(s) — someone who advanced MS&T work — and what did he/she do?

A. Charlie Hughes: Mike Moshell ignited the spark that led us to where we are in VR at IST/SMST. His creation of VSL and his ability to effortlessly move between basic and applied research were essential to the evolution of many of us. His dedication to developing learning environments also set a path that has been pursued by many who followed him at UCF.
There are others who were pioneers in areas like human factors and team training.

Q. When you look ahead 40 years from now, what does your best version of MS&T look like? How has it changed society?

A. Charlie Hughes: I feel MS&T will be embedded in nearly all human activities, including our desire to be a multi-planet species. Simulation was key to planning for the Lunar Excursion Module in the 1960s. VR has and continues to be used in training astronauts (beginning in the 80s), in planning missions (used by scientists to make daily decisions as to what rovers on other planets should do next) and in investigating different strategies (even how vehicles should most effectively navigate on other planets). MS&T and the visualization of simulations is key to humanity’s future, including predicting and controlling pandemics.

A. Hank Okraski: The next 40 years will be exciting with MS&T moving at break-neck speed. M&S technology will permeate every aspect of our lives. For example, in defense, cyber security will be an emphasis-area brought about by current and future threats. The global arms race has added a new dimension: the cyber world. Simulation will be used to determine vulnerability and in risk assessment. AI and machine learning will be incorporated in most military training systems. More tasks in the operational world will be assumed by robots and AI, redefining what the human needs to do. Aircrews will practice Air Combat Maneuvering in flight using augmented reality (AR). Brain waves of aircrew members will be used to determine levels of realism in simulators appropriate for circumstances. In other words, brain wave in-flight data will be compared to simulator data when undergoing the same scenario. Simulator realism will be adjusted to fit the individual as necessary.

K-12 education will include more simulated experiences that will greatly enhance learning. Commonplace will be students actually participating in historical events (signing of the Declaration of Independence) in the virtual world using AR, with students asking questions and interacting with the historical figures.

In medicine, simulation will be combined with data visualization in genetic research and engineering. Data collected in cardio and other tests, including calcium mapping, will be entered into a life-time simulation of the person to determine effects of diet and lifestyle patterns. VR will be used to treat PTSD and other disorders.

In entertainment, new thrills will be generated using AR in the home and at location-based sites, such as theme parks. More research will be done in simulator sickness to assist thrill designers.

For proposed environmental legislation, simulation may be required by Congress and other governing bodies to visualize the pros and cons of a proposed project. For example, when the Pebble Copper Mine in Alaska was being considered, the possible effects of failure of contaminated holding ponds could have been examined through simulation by determining the adverse effects on the Alaskan salmon industry and other environmental effects when compared to economic benefits.

Q. What role(s) did you serve during the 80s?

A. Charlie Hughes: Professor, UCF Computer Science, Affiliated Faculty, IST; Researcher at Visual System Lab

A. Hank Okraski: Director of Engineering, Director of Research and Technology, Director of Research and Engineering, Chief Scientist, Deputy Technical Director and Acting Technical Director at NAWCTSD

Dr. Hughes is a Pegasus professor of computer science, co-lead of the Learning Sciences Cluster, co-director of the Synthetic Reality Laboratory (SREAL) and co-director of the Center for Research in Education Simulation Technology (CREST). He has secondary appointments in Electrical and Computer Engineering, the School of Modeling, Simulation & Training, the College of Community Innovation & Education, and the Department of Games & Interactive Media at UCF. Dr. Hughes was recently inducted into the National Center for Simulation Hall of Fame. His educational background includes a B.A. in mathematics from Northeastern University, a master’s degree and Ph.D. in computer science from Penn State University, and a National Research Council postdoctoral fellowship at the National Institute of Science and Technology. He has more than 60 years of experience in simulation development. His virtual environment research spans over three decades, with the last 15 years focused on virtual learning environments, and he is co-holder of seven patents in this area. Dr. Hughes has authored or co-authored seven books, 62 journal articles, 38 book chapters, and 121 refereed conference papers. He has advised 28 students to completion of their Ph.Ds. in computer science and modeling & simulation. He is a Life Senior Member of both the IEEE and the ACM, and he is an ACM SIGGRAPH Pioneer. He has been PI or co-PI on $29 million in external grants since 2000, of which $6 million are currently active.