A short, widely used screening survey for autism called the Autism Spectrum Quotient – 10 items (AQ-10) may not be reliable for measuring traits of the condition in the general population, according to a study of more than 6,500 adults1.
The test, a 10-item questionnaire that was adapted from the 50-item Autism Spectrum Quotient (AQ), initially was intended to flag adults who might require diagnostic evaluation for autism. But in recent years, many researchers have used it to assess autism traits in large studies of the general population.
The new paper raises questions about this use. “It’s not quite as good as we think it is, and it might not be reliable,” says lead investigator Punit Shah, assistant professor of psychology at the University of Bath in the United Kingdom.
The results are consistent with those from a smaller study, published last year, that also documented problems with the tool2.
The findings suggest that more research into the AQ-10 is needed, experts say — and that it may ultimately be necessary to develop an alternative.
“Let’s not just blindly use what is the most popular thing,” says Ronnie Jia, associate professor of information systems at Illinois State University in Normal, who was not involved in the new study. “If it’s not empirically reliable, you unfortunately can’t really use it because you don’t really know what you’re measuring.”
Rating reliability:
Shah and his colleagues used several statistical techniques to analyze the AQ-10 responses of adults who had completed the questionnaire for a study by another team. The 6,595 participants, who were from 104 countries, were recruited online.
Because the questions are all designed to measure aspects of autism, the responses should also be related. But in fact the responses don’t correlate, suggesting that the AQ-10 has poor internal reliability or consistency. The results appeared in Experimental Results on 5 March.
Based on the responses, the questions seem to cluster into four groups, the team found. That suggests that the questionnaire may measure four different traits, rather than autism as a single, whole construct, Shah says.
The diagnostic criteria for autism are divided into two categories: problems with social communication and the presence of restricted, repetitive behaviors or interests.
If the questions had similarly clustered into two groups, “that probably maps onto what we know about autism,” he says. “Given that it’s 10 items and it’s measuring four different things, it’s not ideal. It’s not measuring or telling us what we want it to.”
By assessing the AQ-10’s validity, the researchers are asking an important question, says Aja Murray, lecturer in psychology at the University of Edinburgh in Scotland, who was not involved in the research. “Their effort is really welcome, because we don’t have a lot of information about how the AQ-10 performs in various different ways,” she says.
However, she disagrees with some of the statistical techniques that they used, noting that they could have underestimated the instrument’s reliability.
Shah himself encourages more research and says that it would be premature to discard the AQ-10, in part because there is not currently a good replacement for it. “The thing that I’d recommend is at least testing the reliability and validity of the instrument wherever possible in each study where it’s used,” he says.
