We caught this interesting article which reflects an issue we’ve all been thinking about, given the interest in VR in simulation we see today: where do Virtual and Actual Reality intercept? If you are in VR how do you deal with things that need to be done in AR?
In an article published this summer, Wired notes: “The downside is that in today’s systems, beside the joystick, rudder pedals, and maybe a throttle lever, all the controls are digital renderings. You “activate” the switches and dials by poking and jabbing into thin air. That amplifies the challenge of VR-based training, where the nuances of touch and movement are essential to programming the pilot’s brain.” In short, there is less transfer of knowledge and especially no “muscle memory” build up when pinching things in the air.
The solution? Haptic feedback, of course. Haptic technology or kinesthetic communication recreates the sense of touch by applying forces, vibrations, or motions to the user. This mechanical stimulation can be used to assist in the creation of virtual objects in a computer simulation, to control such virtual objects, and to enhance the remote control of machines and devices. We experience haptic every day, when our cellphone vibrates to signal a call or the successful digitization of a code. In a simulator we already know of force feedback, which utilizes servo motors to make controls as responsive as if they were in a real aircraft as the external pressures increase on control surfaces and make control feel “live”.
The industry is moving. Basic and rather expensive gloves are already available to capture gesture and touch. As for force feedback, the military industry is already there, with companies such as VirtualMotionLabs providing offerings such as the VMG 35 Plus™ system to an impressive number of civilian and military clients. This glove has 30 sensors plus 5 vibro-tactile actuators – one on each finger – letting the user “feel” their virtual environment.
Wired: “In Go Touch VR’s new system, derived from its engineers’ expertise in haptic feedback, the user wears three sensors on each hand, which look like the blood-pressure sensors doctors place on your fingertips. By applying pressure to your fingertips, the actuators can replicate object stiffness, coarse textures, and the sensation of holding physical objects in your hands.” “The trick is fine-tuning the subtle mechanical interactions—what Vezzoli calls the “cutaneous force feedback through skin indentation”—so they feel natural. “The technology reproduces the exact skin stimulation that you perceive when you are interacting with real objects,” he says. “We are concentrating in the area that you use to interact most, the fingertips. When we couple it with a visual rendering in virtual or augmented reality, you reach out your hand toward an object, activating the skin pressure, the brain ‘clicks’ and lets you perceive the virtual object in front of your eyes as real, because it is feeling a sensation that it is expecting.” Another company, HTC Vive , offers a precision movement tracking device on their goggles plus hand devices to manipulate controls presented virtually.
Haptic feedback could mean that in the not so distant future we may be able to use several parts of our body to natively interact with an hybrid, mixed simulated environment in which some controls will actually be real and provide force feedback (think rudder pedals on an aircraft) and some will be entirely simulated and sensed only by haptic devices such as gloves. VR Goggles already seem to be performing well in directing our head-eye movement to display the correct view, and even adjust on a second degree of freedom to compensate for movements of the actual aircraft.
The flight simulator industry is also catching up. After Laminar Research announced full VR support on X-Plane 11, other software vendors are catching up. Flyinside (see our post here) is a brand new flight simulator built for virtual reality! It ships with ten aircraft, and scenery for the continental USA. FlyInside is still under development. It has been released as a (paid) beta to get add-on developers on-board, and of course help fund development.
And VR in real flying…
And to conclude, things are moving even faster, with VR being used out of the simulator and into the real aircraft. Follow our ongoing story of this development starting with our post on Thrust Vector, a Texas-based company that is trying to revolutionize instrument flying through the use of VR goggles and available geo-referenced and mapping systems.