Mutant mice show inconsistent behaviors in different labs

Researchers from three different labs have found conflicting results on social behavior using similar tests in the same strain of mutant mice. The mice all lack a functional copy of neuroligin-4 (NLGN4), an autism-linked gene.

A 2008 study from Nils Brose’s lab at the Max Planck Institute in Göttingen, Germany, first suggested that mice lacking NLGN4 have a number of autism-like social deficits¹. The lab published similar findings from the mice in a paper in November².

But in September, two independent groups reported that the same strain of mice, bred from animals furnished by the Brose lab, do not show any social deficits³.

The reasons for the differences are not yet clear — they may be compensatory mutations that have evolved in later generations of mice, unknown environmental factors, or some inherent variability in the gene’s function.

In any case, the findings raise questions about the reliability of these models. They also reflect the increasing variability in the phenotype, or outward symptoms, seen in people with the same genetic mutations. Drug developers will need to take this variability into account as they use the mice to screen new therapies.

“The big question is, are there robust behavioral phenotypes in mouse models of autism that can be replicated in different labs?” says Mustafa Sahin, assistant professor of neurology at Boston Children’s Hospital, who was not involved in any of the studies. “This is crucial, because if we are going to use mice as a surrogate to test different treatments, we need to make sure the phenotype is robust.”

Previous research has shown that mutations can have different effects depending on a mouse’s background strain, but the NLGN4 studies involve mice of the same genetic background.

The September study is also unique in that scientists went to great pains to replicate the methods used in the earlier study. “No one has looked at it as carefully as this group,” says Sahin.

Conflicting results:

In the 2008 study, Stéphane Jamain, then a postdoctoral researcher in Brose’s lab, reported that mice lacking NLGN4 showed an array of social deficits.

The mice didn’t spend as much time as controls interacting with other mice. Unlike controls, they also showed no preference for another mouse over an object, or for a new mouse over a familiar one.

After visiting the Brose lab in 2008, Jacqueline Crawley, professor of psychiatry at the University of California, Davis, asked for some of the animals, intending to study more complex behaviors. She and her team used the same battery of tests used in the original study, but to their surprise, the mice appeared to behave normally.

The same thing happened when Thomas Bourgeron’s lab at the Institute Pasteur in Paris tried to replicate the original study. The two groups together published the findings in Genes, Brain and Behavior in September.

“It’s a very well-performed study,” says Sahin. “They go to great pains to explain exactly how they did their analysis.”

Brose’s November study, published in Behavioural Brain Research, replicates his original findings, however. “I am confident this phenotype is real and has been reproduced once by us, but we need to make this really reproducible,” says Brose, director of the Max Planck Institute for Experimental Medicine.

Genetic tests have confirmed that the mice used by Crawley and Bourgeron still carry the NLGN4 mutation, which prevents the full-length protein from being produced. All three teams praise the members of the others as extremely careful scientists, suggesting that it is not a matter of human error.

“We’re talking about the exact same findings in a lab in Bethesda and a lab in Paris,” says Crawley. “We spent a lot of time setting up the methods in our lab, and social behavior was normal even using those approaches.”

The mice that Bourgeron and Crawley studied are from a later generation than those analyzed in the Brose lab in 2008 as well as in the November study. The mice in the latter study were generated from embryos frozen when the strain was first created.

That raises the possibility that spontaneous genetic mutations that compensate for NLGN4 arose somewhere along the line.

“If you look at human patients with NLGN4 mutations, there are some who have serious autism, some who have Asperger-type autism, and some who have intellectual disability,” says Brose. “It looks as if individual genetic modifications can compensate for the loss of NLGN4, or lead to different [symptoms].”

Multiple variables:

So far, however, researchers have not found any obvious compensatory mechanisms, such as a boost in activity of other neuroligins. (Both humans and mice have four neuroligin proteins, which are found at synapses, the connections between neurons.)

Brose says his team is gearing up to look for differences in the expression of other genes. Mutations in other genes would be incredibly interesting, says Brose, because they may point to new drug targets.

The researchers may also look for epigenetic changes, chemical modifications to DNA that alter gene expression without changing the genetic code itself. These may result from environmental differences, such as how the animals were housed or handled early in life.

Altogether, any number of factors can modify the results different labs see, Brose notes.

“I am a biochemist by training and mouse biologist now by trade,” he says. “There are so many sources of variability that scare the shit out of me as a biochemist, even [from] people working at their best.”

Scientists have anecdotally reported conflicting behavior in the same strain of mice, but these findings often aren’t widely known because it can be difficult to publish negative results.

“We thought it was important to publish these negative findings from two labs to encourage other labs to publish their data,” says Crawley. That’s especially true when later generations of a mutant line do not display the same symptoms seen in the original generation, she says.

The findings also emphasize the need to study multiple sets of animals.

“Reproducibility is a basic principle of all scientific research, but particularly for behavioral research,” says Crawley. “Because the environment can have so much influence, you really need to run behavioral tests in two cohorts.”

The implications for drug development are not yet clear.

“Pharma companies need to be able to believe that the behavioral phenotype in a mouse model is replicable from study to study,” says Sahin. “Unfortunately, what this paper raises is that this might be quite difficult, at least for certain genes.”

Brose’s team had started screening treatments for autism in the mice, but put a hold on the experiments after learning of Crawley’s results.

In the new paper, his team proposes a ‘composite severity score,’ a combination of the results of several behavioral tests, calculated separately for male and female mice. He says this approach lowers some of the variability among animals in the same group.

Some scientists argue that it’s more important to look at biological changes in the brain, such as changes to the shape or electrical activity of neurons, than behavior in assessing mouse models of autism.

For example, Pavel Osten, assistant professor of neuroscience at Cold Spring Harbor Laboratory in New York, is studying the brains of NLGN4 mutant mice. “I certainly think that just because a mutation does not lead to social or behavioral deficits doesn’t mean it doesn’t have an effect on the mouse brain,” Osten says.

References:

1: Jamain S. et al. Proc. Natl. Acad. Sci. USA 105, 1710-1715 (2008) PubMed

2: El-Kordi A. et al. Behav. Brain Res. Epub ahead of print (2012) PubMed

3: Ey E. et al. Genes Brain Behav. Epub ahead of print (2012) PubMed