As original equipment manufacturers (OEMs) build artificial intelligence into aircraft cockpits with the use of augmented and virtual reality technology, pilots gain situational awareness, spend less time managing instruments and communicate faster with crewmembers, including ground control operators.
To fully maximize the benefits of AI in the cockpit, pilots need to be trained on the newfangled gear.
The U.S. Air Force uses AI-run simulators to train pilots by replaying real missions for them to better understand and correct mistakes. The simulators also allow pilots to fly more missions in less time.
Maj. Scott Van De Water, deputy director of the Air Force’s experimental Pilot Training Next program, said the digital space allows instructors to make quick changes to the physical environment that pilots are visualizing. Problem-solving is accelerated by being able to “fail fast and light,” he said.
AI instructors understand how individual student pilots learn and process by examining their eye movements, indicators of cognitive processes, he added.
The avionics of different aircraft types are programmed into the Air Force simulators, which can be linked to teach formation flying.
Aero Glass, the developer of a headset pilots wear to view control information, and recipient of an research grant for the European Union’s Horizon 2020 initiative, said that crewmembers being able to share the same augmented visuals is pivotal for mission coordination and to train pilots how to fly in formation and refuel while airborne.
Not having to superimpose data and variables on reality saves pilots time and minimizes the potential for human error, said Akos Maroy, founder of Aero Glass. AI is particularly useful in flying helicopters as it provides terrain and elevation data that can be visualized as a 3D terrain contour.
While hovering or trying to land in a dense area, pilots gain full visual range with augmented reality, even seeing through the airframe. Engine and instrument data is also displayed on the terrain screen, eliminating the need to look at the instrument panel and gauges and interpret top-down charts or waypoints and mentally super-impose spatial information on reality. A flight procedure or a flight path appears in the sky with markers and visualizing terrain in low visibility situations, such as brown- and white-outs, is enabled.
In the pursuit of modernizing cockpits Maroy’s ultimate goal is to replace traditional instruments because alternating between AI systems — often embedded in goggles or a headset — and traditional controls can distract and confuse pilots.
“By taking less mental effort to understand their own position and situation, we're helping pilots in a very vital way,” he said. When precise timing determines a mission’s outcome, “we are making it natural to understand one's difficult situation, as opposed to interpreting complex instruments like an horizontal situation indicator (HSI). We're giving the pilot more time to save the day.”
In search-and-rescue missions, scouts identify the person to be rescued and use visual markers for the rescue location in the augmented-reality overlay for all crewmembers. For logistics and air drop missions, the system allows the scout aircraft pilot or crew to draw an ideal drop position in virtual reality for pilots of other aircraft to see.
Global flight school CAE is investing $1 billion in digital innovation of pilot training systems. Its new CAE Rise, a cloud-based training system that assesses pilot skills with live data during training sessions, has an analytics platform used to rate pilot progress.
“We saw an opportunity to leverage our simulation and technological expertise to put a new source of actionable real-time data at instructors’ fingertips,” according to CAE’s 2018 Annual Activity report. “We also wanted to mitigate the subjectivity and variances between how different instructors grade pilots to strengthen the operators’ ability to deliver standardized training.”
CAE expects that by automating more training, pilots will be taught more against standards and less against an instructor’s subjectivity. Instructors can dig into graphical scorecards that track milestones. “Statistical insights allow the instructor to identify and close pilot proficiency gaps,” the report said. “Instructors may even be able to use the system to reduce remedial training time.”
Although Bell has promoted its futuristic FCX-001 concept helicopter, which would be flown by both human and AI pilots, and expects cockpits to be fully autonomous in the future, its helicopter training school is not currently relying on AI to train pilots.
Bell is, however, using a 3D cross-platform developed by Unity Technologies in the classroom portion of its training program.
Chad Oakley, a flight training manager, said that since Bell has incorporated 3D gaming into its training program, it is evident that pilots are “more interactive, stay more focused and involved and ask more questions.”