Mouse study nails neurons that trigger repetitive behavior

This article is more than five years old.

Neuroscience—and science in general—is constantly evolving, so older articles may contain information or theories that have been reevaluated since their original publication date.

A “beautiful” new study traces a complex repetitive behavior in a mouse model of autism to a subset of neurons in one brain region¹.

The study, published 3 July in Cell, looked at two sets of mice with mutations in neuroligin-3 (NLGN3), a gene that has repeatedly been linked to autism. The NLGN3 protein helps form and maintain synapses, the junctions at which neurons communicate.

The two sets of mice have different mutations in NLGN3, but they behave similarly on a rotarod test, the mouse equivalent of a log-rolling contest. Control mice fall off the rotarod as it spins faster, but the mutant mice learn to adjust their feet to stay on the rod for about a minute longer.

This may serve as a model for learned repetitive behaviors, reminiscent of people in autism acquiring certain routines, the researchers say.

Other studies have found these sorts of behavioral differences in autism mice. But this study went further: The researchers then traced the behavior to a group of neurons in the nucleus accumbens, a brain region linked to goal-directed behavior.

“That is exactly what we’re going to have to have if we’re going to combat autism and related disorders,” says Ann Graybiel, professor of brain and cognitive sciences at the Massachusetts Institute of Technology in Cambridge, who was not involved in the new study. “It’s a beauty.”

Chasing circuits:

The researchers began their search for the circuits responsible for the repetitive behavior by doing a series of genetic experiments. They knocked out the NLGN3 gene in one set of mice, and introduced a mutation in the other set that lowers the levels of the NLGN3 protein by 90 percent. Both of these mutations are linked to autism^{2, 3}.

However, these mutations affect the entire brain. “[The results] don’t tell you where in the brain the mutations or the loss of NLGN3 is causing this dramatic behavioral phenotype,” says lead investigator Robert Malenka, professor of psychiatry and behavioral sciences at Stanford University in California.

The researchers narrowed their search to specific regions, including the cerebellum, a brain region linked to motor coordination. To their surprise, removing NLGN3 from certain cells in the cerebellum does not produce the repetitive behavior⁴.

Instead, they found that deleting NLGN3 from the nucleus accumbens, an area of the brain thought to be important for goal-directed behavior, induces the repetitive behavior.

“You would not think of the nucleus accumbens as being involved in that kind of [motor] activity because they’re not trying to learn a complicated task to get a reward,” says Jonathan Britt, assistant professor of behavioral neuroscience at McGill University in Montreal, Canada, who was not involved in the study.

A careful analysis of the nucleus accumbens implicated a subset of neurons that express a receptor for the chemical messenger dopamine. These neurons play a key role in organizing sequences of movement.

In control mice, these neurons express higher levels of NLGN3 than do others in the same region, suggesting that the protein is critical to their function.

When the researchers restored NGLN3 to that subset of neurons, the mutant mice behaved normally, falling off the fast rotarod as the controls did.

“That says that the behavioral shift is not due to some developmental difference,” Malenka says. “If it were a developmental difference, we wouldn’t be able to rescue it by putting back the protein. And that has therapeutic implications.”

The researchers then recorded the electrical activity of neurons in the nucleus accumbens. They found that neurons lacking NLGN3 are less efficient at dampening brain activity than are control neurons. This precise knowledge may provide pharmaceutical companies with a specific target, but researchers must first determine whether the findings apply to people with autism.

Mouse behaviors do not always match those in people, cautions Steven Hyman, director of the Stanley Center for Psychiatric Research at the Broad Institute in Cambridge, Massachusetts. Instead, researchers should investigate whether the neural circuits in mice and people are similar, he says.

“We should focus less on whether the behavior reminds us directly of human behavior,” he says.