Smart system makes humans and machines better flight partners

Smart system makes humans and machines better flight partners

RMIT research is helping machines work better with pilots and air traffic controllers.

Researcher in the flight simulator wearing electroencephalogram (EEG) device. Researcher in the flight simulator wearing electroencephalogram (EEG) device.

The computers helping with flight control and collision avoidance can now consider how pilots are thinking and feeling as part of their decision-making, said Intelligent Transport and Mission Systemsresearch group leader, Professor Roberto Sabatini.

“If computers are helping us fly planes they need to consider more than just weather conditions and other aircraft movements, they also need to understand the condition of the pilot they’re working alongside and change the balance of human-computer control accordingly,” Sabatini said.

The new Cognitive Human-Machine Interface and Interaction (CHMI²) system analyses data on the pilot’s eye-movement, facial expression, brain activity, body temperature, voice patterns, posture and exertion as well as emotional stress indicators like those used in lie-detector tests.

It uses this information to determine the pilot’s state of mind – for example how tired, stressed or distracted they are – and decide how much computer support is required to fly the plane.

“In the most obvious example, the system can tell if a pilot is severely incapacitated or unconscious then take full control of the aircraft, but there are many more subtle instances of where they can work together and maximise the combined human-machine team performance,” he said.

The system can also be used to support air traffic controllers and may have applications for single person control of multiple unmanned aircraft.

Algorithms and neural networks modelled on the human brain allow the system to learn and improve over time so the more a person interacts with it, the better it performs with them, like with the development of a real human partnership.

“The emerging concept of human-machine teaming has massive potential to extend our abilities and performance in aerospace, defence, healthcare and a whole range of industries,” Sabatini said. “This is a significant step forward in enabling true, two-way collaboration between humans and machines.”

The research at RMIT’s Aerospace Automation and Autonomous Systems Laboratory in the School of Engineering is being conducted with industry partners Northrop Grumman Corporation, Thales Australia and the Australian Defence Science and Technology (DST) group.

At the latest IEEE International Workshop on Metrology for Aerospace, the team’s paper on “Cognitive Human-Machine Interfaces and Interactions for Avionics and Air Traffic Management Systems” was awarded the Best Paper prize on the grounds of quality, technical merit, originality and potential impact in the field.

“While humans are creative, flexible and good at strategic decision-making or adapting to unexpected situations, machines provide tactical precision and are better at information acquisition and synthesis,” Sabatini said.

“Both need to work together to achieve optimum performance and that’s where we’re taking this exciting technology – towards a whole range of real-life applications.”

Researcher in the air traffic control simulator wearing eye-tracking glasses. Researcher in the air traffic control simulator wearing eye-tracking glasses.

Story: Michael Quin

22 August 2018


22 August 2018


  • Research
  • Science and technology

Related News

Subscribe to RMIT NewsSubscribe
aboriginal flag
torres strait flag

Acknowledgement of Country

RMIT University acknowledges the people of the Woi wurrung and Boon wurrung language groups of the eastern Kulin Nation on whose unceded lands we conduct the business of the University. RMIT University respectfully acknowledges their Ancestors and Elders, past and present. RMIT also acknowledges the Traditional Custodians and their Ancestors of the lands and waters across Australia where we conduct our business.