Exploring how primates create meaning from visual scenes

How do humans and monkeys find meaning in visual scenes? New in JNeurosci, researchers led by Vinay Raghavan, from the City College of New York, advanced understanding by assessing eye movements together with brain activity as humans and macaques watched full-length movies or short video clips. 

The researchers discovered that new semantic—or conceptual—information during full-length movies influenced neural activity in attention and scene-processing brain areas. Deeper brain recordings taken from a separate set of human participants and two female macaques showed early, rapid brain activity in attention and scene-processing brain areas when short videos presented new semantic information, but only in between eye fixations while taking in a scene. What might that mean? Raghavan explains that the involvement of brain areas beyond visual regions suggests there is a higher order—or more abstract—mental system that may be involved in predicting what is about to happen between glimpses to quickly create meaning. 

According to the researchers, this work suggests that visual processing is a continuous process that involves comparing semantic information from separate glimpses of visuals to understand scenes as they unfold. The research team emphasizes the relevance of this work to artificial intelligence (AI) models. Says Raghavan, “It’s possible the learning strategy in this study trained neural networks and the primate brain to produce a ‘semantic novelty signal.’ The fact that this semantic novelty signal is encoded in the brain suggests that [we can implement this learning strategy] to train new vision models in AI.” 

### 

Please contact media@sfn.org for full-text PDF. 

About JNeurosci 

JNeurosci was launched in 1981 as a means to communicate the findings of the highest quality neuroscience research to the growing field. Today, the journal remains committed to publishing cutting-edge neuroscience that will have an immediate and lasting scientific impact, while responding to authors' changing publishing needs, representing breadth of the field and diversity in authorship. 

About The Society for Neuroscience 

The Society for Neuroscience is the world's largest organization of scientists and physicians devoted to understanding the brain and nervous system. The nonprofit organization, founded in 1969, now has nearly 35,000 members in more than 95 countries.