Through Air-Guardian, a specialized software application can monitor the focal point of a human pilot by utilizing eye-tracking technology. This enables the software to make informed decisions that align with the pilot’s actions or intentions. Source: Alex Shipps/MIT CSAIL via Midjourney.
Created with the objective of enhancing air safety, Air-Guardian combines human instinct with machine exactness, fostering a more harmonious interaction between the pilot and the airplane.
Visualize being aboard an aircraft guided by dual pilots—one human and the other a computer. Both operate the controls, but each has different areas of focus. When both focus on identical tasks, control is given to the human pilot. However, if the human pilot becomes inattentive or overlooks something, the computer immediately assumes control.
Introducing Air-Guardian, a system engineered by the scholars at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). Modern pilots must navigate a plethora of data from various displays, particularly during crucial phases. In these instances, Air-Guardian serves as a preemptive copilot, founded on the principles of mutual attention between the human and the machine.
Table of Contents
Deciphering Attention Through Air-Guardian
But how is attention quantified? For human operators, eye-tracking is employed, while for the neural system, “saliency maps” are used to determine the focus of attention. These maps function as visual outlines that emphasize significant areas within a visual frame, facilitating the interpretation of complex algorithms. Air-Guardian can identify early indicators of possible hazards through these markers of attention, unlike traditional autopilots that intervene only during explicit safety violations.
The implications of this technology extend beyond the aviation sector. Analogous collaborative control systems could eventually be applied in automobiles, drones, and a broader range of robotics.
“The distinguishing feature of our technique is its differentiability,” says Lianhao Yin, an MIT CSAIL postdoctoral researcher and lead author of a recent paper on Air-Guardian. “Our collaborative layer and the complete end-to-end mechanism can be trained. We opted for the causal continuous-depth neural network model specifically for its dynamic capabilities in mapping attention. Another noteworthy aspect is its flexibility; the Air-Guardian system can be tailored according to situational requirements, maintaining an equitable collaboration between human and machine.”
Empirical Tests and Outcomes
During field experiments, both the human pilot and the AI system based their decisions on identical raw imagery while navigating toward a pre-determined waypoint. Air-Guardian’s efficacy was measured by cumulative rewards obtained during flight and a shorter route to the waypoint. The system lessened flight risks and amplified the success rate in reaching target locations.
“This technology epitomizes the cutting-edge approach to human-focused AI in aviation,” adds Ramin Hasani, MIT CSAIL research affiliate and pioneer of liquid neural networks. “Our implementation of liquid neural networks offers a dynamic, adaptive strategy, ensuring that AI not only substitutes but also enhances human discernment, thus contributing to increased air safety.”
Technological Core and Prospective Developments
The underlying strength of Air-Guardian lies in its foundational technology. It employs an optimization-based cooperative layer that uses visual cues from both humans and machines, in conjunction with liquid closed-form continuous-time neural networks (CfC), renowned for their adeptness in interpreting causal relationships. This is complemented by the VisualBackProp algorithm, which discerns the system’s areas of focus within an image, thereby clarifying its attention maps.
For future widespread adoption, there is a need to fine-tune the interface between humans and machines. Preliminary feedback suggests that an indicator, such as a bar, might be a more intuitive way to signify when the guardian system assumes control.
Air-Guardian signals the advent of a new era in aviation safety, providing a dependable backup for moments when human attention may falter.
“The Air-Guardian initiative accentuates the synergistic relationship between human skills and machine intelligence, advancing the aim of employing machine learning to bolster pilots in challenging circumstances and minimize operational errors,” remarks Daniela Rus, the Andrew (1956) and Erna Viterbi Professor of Electrical Engineering and Computer Science at MIT, director of CSAIL, and senior author of the paper.
“Among the most intriguing results of employing a visual attention metric in this work is the prospect for earlier interventions and improved understandability by human pilots,” notes Stephanie Gil, Assistant Professor of Computer Science at Harvard University, who was not involved in the research. “This represents an exemplary instance of how AI can collaborate with humans, lowering the trust barrier through natural communication mechanisms between the human and the AI system.”
Reference: “Towards Cooperative Flight Control Using Visual-Attention” by Lianhao Yin, Makram Chahine, Tsun-Hsuan Wang, Tim Seyde, Chao Liu, Mathias Lechner, Ramin Hasani, and Daniela Rus, published on 20 September 2023 in the field of Computer Science > Robotics.
arXiv:2212.11084
This research received partial financial support from the U.S. Air Force (USAF) Research Laboratory, the USAF Artificial Intelligence Accelerator, Boeing Co., and the Office of Naval Research. The findings do not necessarily represent the opinions of the U.S. government or the USAF.
Frequently Asked Questions (FAQs) about Air-Guardian
What is the primary objective of MIT’s Air-Guardian system?
The primary objective of Air-Guardian is to enhance aviation safety. The system is designed to work in harmony with human pilots, using eye-tracking technology to understand where the pilot is focusing. This allows the AI system to make decisions that align with the pilot’s actions or intentions.
How does Air-Guardian determine the focus or attention of a pilot?
Air-Guardian employs eye-tracking technology to monitor where the human pilot is looking. For the machine component, the system uses “saliency maps” to pinpoint where attention is directed. These maps highlight key regions within a visual frame, assisting in the interpretation of complex algorithms.
What sets Air-Guardian apart from traditional autopilot systems?
Unlike traditional autopilots that intervene only during explicit safety breaches, Air-Guardian identifies early indicators of potential risks through markers of attention. This proactive approach allows for timely interventions that can prevent accidents.
Can the technology used in Air-Guardian be applied to other fields?
Yes, the collaborative control mechanisms used in Air-Guardian have broader implications beyond aviation. Similar systems could eventually be employed in automobiles, drones, and a wider range of robotics.
What are the technological foundations of Air-Guardian?
Air-Guardian utilizes an optimization-based cooperative layer that receives visual attention cues from both humans and machines. It also uses liquid closed-form continuous-time neural networks (CfC) for interpreting cause-and-effect relationships, complemented by the VisualBackProp algorithm for attention mapping.
How was the effectiveness of Air-Guardian measured in field tests?
The system’s effectiveness was gauged based on the cumulative rewards earned during flights and the shorter path taken to reach target waypoints. During these tests, Air-Guardian successfully reduced the risk level of flights and increased the rate of success in reaching target points.
Who funded the research for Air-Guardian?
The research was partially funded by the U.S. Air Force (USAF) Research Laboratory, the USAF Artificial Intelligence Accelerator, Boeing Co., and the Office of Naval Research.
Does the system require any modifications for mass adoption?
For broader adoption, there is a need to fine-tune the human-machine interface. Initial feedback suggests that an intuitive indicator, like a bar, might be beneficial to signify when the guardian system takes control.
What experts were involved in the development and assessment of Air-Guardian?
The system was developed by researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). Notable contributors include Lianhao Yin, a postdoctoral researcher at MIT CSAIL, and Ramin Hasani, an MIT CSAIL research affiliate. External commentary was provided by Daniela Rus, director of CSAIL, and Stephanie Gil, Assistant Professor of Computer Science at Harvard University.
More about Air-Guardian
-
MIT CSAIL’s Air-Guardian – MIT CSAIL’s official page on Air-Guardian, providing insights into the system’s development and objectives.
-
Research Paper: “Towards Cooperative Flight Control Using Visual-Attention” – The academic paper detailing the technology
