GlowTrack: Advancing the Use of Artificial Intelligence for High-Resolution Human and Animal Movement Analysis

Image Caption: Human hand featuring fluorescent markers from GlowTrack. Source: Salk Institute

Researchers from the Salk Institute have developed GlowTrack, a technology designed to offer enhanced resolution and greater flexibility in tracking both human and animal behavior.

Understanding movement serves as a crucial conduit for studying brain functionality and its control over the physique. The evolution in tracking techniques has progressed from rudimentary manual observations to sophisticated methods leveraging artificial intelligence. Despite these advancements, existing approaches using AI are labor-intensive and are constrained by the requirement for scientists to manually label each body segment on multiple occasions.

Introducing GlowTrack

Associate Professor Eiman Azim and his research team have introduced GlowTrack, an innovative, non-intrusive methodology for movement tracking that employs fluorescent markers as training data for machine learning algorithms. GlowTrack distinguishes itself by being efficient, high-definition, and robust, capable of monitoring intricate movements ranging from a single finger on a mouse’s paw to multiple landmarks on a human hand.

Published in Nature Communications on September 26, 2023, the methodology has a wide array of applications, extending from biological sciences and robotics to medical fields and beyond.

Image Caption: Daniel Butler and Eiman Azim. Source: Salk Institute

“In recent years, a paradigm shift has occurred in behavior tracking, thanks to the introduction of potent artificial intelligence tools into research labs,” remarks Azim, the principal investigator and holder of the William Scandling Developmental Chair. “Our method enhances the versatility of these tools, refining how we record multifaceted movements in controlled settings. Improved measurement of movement augments our understanding of brain-behavior interactions and may facilitate research into movement-related disorders like ALS and Parkinson’s disease.”

Addressing Existing Shortcomings

Contemporary techniques for capturing movement frequently demand that scientists manually tag body parts on a digital display— a procedure that is not only tedious but is also prone to human error and time limitations. This method typically restricts the application to specialized test conditions since machine learning models are trained on limited datasets. Changes in variables like lighting, orientation of the subject, or camera angles can throw off the model’s ability to recognize the labeled parts.

To overcome these challenges, the team used fluorescent dyes to mark regions of interest on the body. These “invisible” markers provide a large and visually varied dataset that can be readily incorporated into machine learning models, eliminating the need for human annotation. This rich data allows for tracking across diverse conditions and provides a level of detail that is challenging to achieve through manual tagging.

According to Azim, the ability to compare and reproduce results across different research settings is vital for scientific exploration.

Daniel Butler, Salk’s bioinformatics analyst and the study’s first author, states, “The use of fluorescent markers proved to be the optimal solution. Much like the invisible ink on a banknote, these markers can be activated or deactivated instantaneously, enabling the rapid generation of a vast training dataset.”

Future Prospects

Looking forward, the team is keen on expanding the applications of GlowTrack. They plan to integrate it with other tracking systems that can reconstruct 3D movements and analytical methods capable of scrutinizing large datasets for patterns.

Azim concludes, “Our methodology has the potential to significantly benefit a wide range of disciplines that require more precise, reliable, and holistic tools for capturing and analyzing movement. I am enthusiastic about the prospects of other scientists, and even non-scientists, adopting these techniques and discovering new, unanticipated applications.”

References

The research paper, entitled “Large-scale capture of hidden fluorescent labels for training generalizable markerless motion capture models,” was published on 26 September 2023 in Nature Communications. DOI: 10.1038/s41467-023-41565-3

Contributors to the study also include Alexander Keim and Shantanu Ray of the Salk Institute.

Financial support for this project was provided by various institutions, including the UC San Diego CMG Training Program, Jesse and Caryl Philips Foundation Award, National Institutes of Health (multiple grant numbers), Searle Scholars Program, Pew Charitable Trusts, and the McKnight Foundation.

Frequently Asked Questions (FAQs) about GlowTrack

More about GlowTrack

• Nature Communications Journal
• Salk Institute for Biological Studies
• Overview of Artificial Intelligence in Healthcare
• National Institutes of Health (NIH) Grants
• Introduction to Bioinformatics
• Understanding Movement Disorders
• UC San Diego Training Programs
• Pew Charitable Trusts Research Programs
• Searle Scholars Program