Distinct macaque neurons process reward prediction error for self and others

Humans and other primates learn cause-and-effect relationships in their environments by predicting the likelihood of an event, such as a reward. Importantly, we can learn this through direct experience or by observing the experiences of others.

Macaques, a primate species, can predict the likelihood of receiving a reward and also detect errors in those predictions, when expectations don’t match reality. Both self reward prediction errors (S-RPE) and reward prediction errors of others (O-RPE) are thought to help macaques better understand the physical world and social environments.

The areas of the macaque brain that are thought to contribute to S-RPE and O-RPE are primarily found in the frontal cortical areas of the brain, which are associated with planning and higher cognitive functions. Researchers, however, were unsure whether or not macaques (and presumably other primates) could predict both O-RPE and S-RPE simultaneously when two macaques are learning different stimulus-outcome associations. Critically, this ability would improve the macaque’s learning efficiency and reward prediction accuracy.

Additionally, questions also remained regarding how O-RPE is encoded at the single-neuron level in the macaque brain and which specific areas in the frontal cortex are involved. In order to better understand the neural mechanisms behind O-RPE and S-RPE, scientists from the National Institutes of Natural Sciences (NINS) designed a social Pavlovian conditioning procedure for two macaques that received rewards after being presented the same visual stimuli. The team published their research on March 25 in Cell Reports.

“Our working hypothesis is that neurons encoding the RPE for the self and others play a central role in learning the causal structure of rewards in the environment from one’s own and others’ perspective, respectively. This brain function is intimately associated with mentalizing, the ability to infer each agent’s mental states, such as belief and motivation,” said Masaki Isoda, professor in the Department of System Neuroscience at NINS in Okazaki, Japan and senior author of the research paper.

Specifically, macaques were seated facing one another and a visual stimulus was shown to each monkey at the same time. After one second, a water reward was given, with a certain probability, to the “other” monkey, whose neural activity was not being recorded. One second later, a water reward was given, with a different probability, to the “self” monkey, whose neural activity was being recorded. This design was chosen to simulate resource competition in nature. The macaques never received a reward simultaneously; either the “self” or the “other” macaque or neither received water.

The research team was able to record the firing of individual neurons in the dorsal part of the medial prefrontal cortex (DMPFC) for the “self” macaque. In this experiment, both S-RPE and O-RPE occurred after reward feedback to “other,” but only S-RPE occurred after reward feedback to “self.” By altering the probability of reward to the “other” macaque in one trial block and holding the probability of “other” reward the same in a separate trial block, the research team could deduce which neurons were responsible for O-RPE.

The research team was able to identify the DMPFC neurons responsible for encoding positive and negative O-RPE, signifying the “other” macaque received more or less of a reward than predicted, respectively. Importantly, separate neurons are responsible for positive and negative O-RPE.

The scientists also established that 35 neurons in the macaque DMPFC encoded the “other” reward outcome based on neuronal activity when the reward was given versus omitted. And by correlating the activity of neurons during trials when the “self” reward probability was constant or varied, the research team was able to establish the neurons responsible for positive and negative S-RPE.

“We found that the reward prediction error for oneself and others is encoded by separate groups of neurons in the medial prefrontal cortex. Remarkably, these neurons fire at exactly the same time, demonstrating that the brain is capable of learning and updating the causal structure of reward simultaneously from the perspective of oneself and others,” said Atsushi Noritake, assistant professor in the Department of System Neuroscience at NINS in Okazaki, Japan and first author of the research paper.

Overall, the number of neurons encoding O-RPE is greater than those that encode S-RPE, which is consistent with previous studies. The authors also suggest that positive RPE and negative RPE can be associated with positive and negative emotions, respectively, and that the S-RPE and O-RPE DMPFC neurons identified in the study may play a role in processing social emotions.

“Our ultimate goal is to understand the brain mechanisms underlying mentalizing. To this end, we will take advantage of artificial intelligence in our systems neuroscience research. Specifically, we will compare and contrast neural information representation and computational algorithms in biological brains with those in artificial neural networks during the performance of mentalizing tasks,” said Isoda.

This work was supported by grants-in-aid for the Japan Society for the Promotion of Science 22H05081 and 22H04931 and by AMED under grant JP23wm0525001.