Using the latest in artificial intelligence (AI) and computer vision technologies, this European-funded project will integrate— for the first time— remote eye-gaze tracking and gesture recognition for pilots in a single framework. Its purpose is to enhance human-machine interaction in the complex flight operations of today’s cockpits through pilot monitoring, for applications aiming to improve crew efficiency and pilot training, towards the development of new generation cockpits.
“We need to recognize when limitations such as confusion or drowsiness impact attention, mental workload, and decision-making on the flight deck,” explains Andrea Dunbar, Head of Embedded Vision Systems at CSEM. “Additionally, the new HMI we are developing will eventually enable a more intuitive and natural interaction so they can make quick, informed decisions across any situation, even when stressed.”
Optimizing human–machine interactions
To design a powerful and tailored solution, the Neuchâtel-based CSEM and three partners will provide the essential complementary skills, ensuring the project’s success. “Our company is pleased to support the consortium partners with the expertise of a professional airline”, says Christoph Ammann, Vice-President and Head of Crew Training at Swiss International Air Lines. “Mutual exchange with research and industry partners enables us to reflect on our training standards and on potential future applications”. Along with Clean Sky 2, another partner is SERMA Ingénierie, who will be responsible for integrating the PEGGASUS output into a cockpit prototype for testing.
