GlowTrack: Harnessing AI to Monitor Human and Animal Behavior

Scientists at the Salk Institute have developed GlowTrack, a revolutionary method to track human and animal behavior with improved precision and flexibility. The ability to track movement provides valuable insights into brain functioning and motor control. From traditional observation methods using pen and paper to the latest artificial intelligence (AI) techniques, tracking movement has evolved significantly. While modern AI-based methods automate the tracking process, they still require extensive training and manual labeling of body parts.

Introducing GlowTrack, Associate Professor Eiman Azim and his team have introduced a non-invasive movement tracking method that uses fluorescent dye markers to train AI. This technique offers numerous advantages, including robustness, time efficiency, and high definition. It can track a single digit on a mouse’s paw or hundreds of landmarks on a human hand. The findings of this research, published in Nature Communications on September 26, 2023, have wide-ranging applications in biology, robotics, medicine, and beyond.

Over the past few years, there has been a revolution in behavior tracking as advanced AI tools have made their way into laboratories, says Azim, the senior author of the study. The GlowTrack approach enhances the versatility of these tools, enabling the capture of diverse movements in the lab. By quantifying movement more accurately, researchers can gain deeper insights into how the brain controls behavior, potentially aiding the study of movement disorders such as amyotrophic lateral sclerosis (ALS) and Parkinson’s disease.

GlowTrack addresses the limitations of current methods that require researchers to manually mark body parts on a computer screen repeatedly. This process is not only time-consuming but also prone to human error. Moreover, human annotation limits the applicability of these methods to specific testing environments. Artificial intelligence models trained on such limited data struggle to recognize tracked body parts if changes occur in the lighting, orientation, camera angle, or other factors.

To overcome these limitations, the Salk Institute researchers utilized fluorescent dye to label body parts in animals and humans. The use of these “invisible” markers eliminates the need for human annotation and allows for the creation of visually diverse data. The robust data can then be fed into artificial intelligence models, enabling them to track movements in a variety of environments and at a much higher resolution than achieved through manual labeling.

This breakthrough enables easier comparison of movement data across different studies, as various laboratories can use the same models to track body movements in diverse situations. Dr. Azim emphasizes the importance of comparing and reproducing experiments to drive scientific discovery forward. The inclusion of fluorescent dye markers has proven to be the ideal solution, according to Daniel Butler, the first author of the study and a bioinformatics analyst at Salk. Similar to invisible ink on a dollar bill that becomes visible when desired, the dye markers can be turned on and off instantaneously, allowing for the generation of a vast amount of training data.

Looking ahead, the team is eager to explore additional applications of GlowTrack by integrating it with other tracking tools that reconstruct three-dimensional movements. They also plan to employ analytical approaches to uncover patterns in these extensive movement datasets.

Dr. Azim believes that GlowTrack can benefit various fields that require sensitive, reliable, and comprehensive movement-tracking tools. He eagerly anticipates the adoption of these methods by scientists and non-scientists alike, as well as the emergence of unique and unforeseen applications.