Identifying what a particular neuromodulator does in the brain—let alone how such molecules interact—has vexed researchers for decades. Dopamine agonists increase reward-seeking, whereas serotonin agonists decrease it, for example, suggesting that the two neuromodulators act in opposition. And yet, neurons in the brain’s limbic regions release both chemicals in response to a reward (and also to a punishment), albeit on different timescales, electrophysiological recordings have revealed, pointing to a complementary relationship.
This dual response suggests that the interplay between dopamine and serotonin may be important for learning. But no tools existed to simultaneously manipulate the neuromodulators and test their respective roles in a particular area of the brain—at least, not until now—says Robert Malenka, professor of psychiatry and behavioral sciences at Stanford University.
As it turns out, serotonin and dopamine join forces in the nucleus accumbens during reinforcement learning, according to a new study Malenka led, yet they act in opposition: dopamine as a gas pedal and serotonin as a brake on signaling that a stimulus is rewarding. The mice he and his colleagues studied learned faster and performed more reliably when the team optogenetically pressed on the animals’ dopamine “gas” as they simultaneously eased off the serotonin “brake.”
“It adds a very rich and beguiling picture of the interaction between dopamine and serotonin,” says Peter Dayan, director of computational neuroscience at the Max Planck Institute for Biological Cybernetics. In 2002, Dayan proposed a different framework for how dopamine and serotonin might work in opposition, but he was not involved in the new study.
The new work “partially recapitulates” that 2002 proposal, Dayan adds, “but also poses many more questions.”
T
he axons of both dopamine and serotonin neurons project to the posterior medial shell of the nucleus accumbens, Malenka and his team found via viral tracing. When a mouse consumes a reward, dopamine axons in this region increase their activity, and dopamine release rises, whereas serotonin axons decrease activity and release less serotonin than usual, the team found using calcium imaging and fluorescent sensors.“That was our first clue that we were onto something that looked closer to opponency than to synergy,” says Daniel Cardozo Pinto, who worked on the study while a graduate student in Malenka’s lab and is now a postdoctoral researcher in Naoshige Uchida’s lab at Harvard University.
Simultaneously blunting dopamine neurons and stimulating the serotonin neurons prevented mice from learning a conditioning task, the team found using two-color optogenetics. Manipulating one cell population at a time, on the other hand, weakened but did not block the animals’ performance.
The simultaneous manipulations also led mice to consume less sugar water, the reward used during conditioning, the team found in a control experiment. That suggests that the neuronal changes affected how pleasurable the animals found the reward, and thus its ability to reinforce learning, Cardozo Pinto says. The work was published in Nature in November.
The rivalry between dopamine and serotonin also extends to the nucleus accumbens core, a region critical for reward learning through its receipt of dopamine signaling, the new study shows: Mice prefer to spend time in the corner of a room in which dopamine release in the nucleus accumbens core was increased and serotonin release was inhibited, rather than in corners that delivered dopamine or inhibited serotonin alone.
“It really nicely pits these signals against each other,” Dayan says. Yet the two signals are still working together, he says. “They’re not directly dependent on each other, but they have these synergistic effects.”
That push and pull may enable animals “to have flexible decision-making and adaptability in decision-making and learning processes,” says Carolina Rezaval, associate professor of biosciences at the University of Birmingham, who was not involved in the study. Flies also show an opposing relationship between dopamine and serotonin to balance reward-seeking and caution during mating behaviors, work from Rezaval’s own lab showed.
E
xactly how the opposing relationship of dopamine and serotonin applies in different contexts for mammals remains unclear, Malenka says. In other behavioral settings, a rise in serotonin release in the nucleus accumbens has pro-social effects, for example.Those effects are likely to be brain-region-specific, too, says Ilana Witten, professor of neuroscience at Princeton University, who was not involved in the study. Serotonin prevents bursting activity in lateral habenula neurons, which should promote, rather than suppress, dopamine activity. “It stands to reason, given the types of data other people have seen, that you’re not going to see this type of opponency everywhere,” Witten says.
Malenka agrees that he and his team would need to perform similar experiments in other brain regions to confirm the generality of the findings. “While I would love to be able to say, ‘We have discovered a general principle about how these neuromodulators work,’” he says, “deep down, I don’t believe that.”
Even so, the fact that the opposing relationship holds up in two different regions of the nucleus accumbens suggests that the findings generalize to some degree, even if those regions tend to function differently, Dayan says. For example, dopamine release is decreased in most of the nucleus accumbens but increased in the areas of the medial shell in response to aversion, other research has shown.
It would be interesting to investigate the relationship between dopamine and serotonin under these conditions, Dayan adds. “That suggests that there’s even more generality that really needs to be looked at.”
