The computers helping with flight control and collision avoidance can now consider how pilots are thinking and feeling as part of their decision-making, said Intelligent Transport and Mission Systemsresearch group leader, Professor Roberto Sabatini.
“If computers are helping us fly planes they need to consider more than just weather conditions and other aircraft movements, they also need to understand the condition of the pilot they’re working alongside and change the balance of human-computer control accordingly,” Sabatini said.
The new Cognitive Human-Machine Interface and Interaction (CHMI²) system analyses data on the pilot’s eye-movement, facial expression, brain activity, body temperature, voice patterns, posture and exertion as well as emotional stress indicators like those used in lie-detector tests.
It uses this information to determine the pilot’s state of mind – for example how tired, stressed or distracted they are – and decide how much computer support is required to fly the plane.
“In the most obvious example, the system can tell if a pilot is severely incapacitated or unconscious then take full control of the aircraft, but there are many more subtle instances of where they can work together and maximise the combined human-machine team performance,” he said.
The system can also be used to support air traffic controllers and may have applications for single person control of multiple unmanned aircraft.
Algorithms and neural networks modelled on the human brain allow the system to learn and improve over time so the more a person interacts with it, the better it performs with them, like with the development of a real human partnership.
“The emerging concept of human-machine teaming has massive potential to extend our abilities and performance in aerospace, defence, healthcare and a whole range of industries,” Sabatini said. “This is a significant step forward in enabling true, two-way collaboration between humans and machines.”
The research at RMIT’s Aerospace Automation and Autonomous Systems Laboratory in the School of Engineering is being conducted with industry partners Northrop Grumman Corporation, Thales Australia and the Australian Defence Science and Technology (DST) group.
At the latest IEEE International Workshop on Metrology for Aerospace, the team’s paper on “Cognitive Human-Machine Interfaces and Interactions for Avionics and Air Traffic Management Systems” was awarded the Best Paper prize on the grounds of quality, technical merit, originality and potential impact in the field.
“While humans are creative, flexible and good at strategic decision-making or adapting to unexpected situations, machines provide tactical precision and are better at information acquisition and synthesis,” Sabatini said.
“Both need to work together to achieve optimum performance and that’s where we’re taking this exciting technology – towards a whole range of real-life applications.”
