On 11 January 2023, as the workday began, Thomas Frazier experienced academic whiplash. First, the editors at Developmental Medicine and Child Neurology let him know that they had published his paper on a new autism trait measure. Then, an hour later, a colleague alerted him that the Daily Mail had covered it, with a splashy headline boasting the tool’s ability to “diagnose autism with up to 95 percent accuracy.”
“I freaked out,” says Frazier, professor of psychology at John Carroll University in University Heights, Ohio. According to Frazier, the data behind the new measure, the Autism Symptom Dimensions Questionnaire (ASDQ), are far too preliminary to justify its clinical use absent the addition of other validated measures.
For the paper, Frazier and his colleagues used the survey platform Prolific to ask the parents of 1,467 children, including 104 who said they had an autistic child, to complete the ASDQ, along with validated measures of social behaviors, behavioral problems, and daily living and executive functioning skills.
ASDQ scores correlated with parent-reported autism diagnoses and, to a lesser extent, scores on the other questionnaires, the researchers found. Using a baseline cutoff score identified 95 percent of the autistic children — the statistic the Mail grabbed onto — but a more stringent cutoff that differentiates autism from other developmental conditions flagged only 56 percent.
Spectrum spoke with Frazier about creating the measure and his hopes for getting it into the right hands.
Spectrum: What concerned you about the Daily Mail article?
Thomas Frazier: Well, the headline touting the 95 percent statistic. That was an exploratory analysis that I was asked to do by a reviewer, that we put into the paper with lots of caveats to say, “Hey, we’re just developing this measure, we’re looking at the psychometrics.” To pick up on that little piece is just very annoying. Focus on the fact that we’re trying to build a good measure. But I know that doesn’t get clicks.
The other thing about it is that they published the measure in the story, which is counterproductive.
S: How so?
TF: Well, it’s open source so that people who know how to use measures and questionnaires can use it. But it’s not necessarily for everybody. Really what we want is for professionals to use it. We’re trying to build an open source measure that researchers can use without cost so that they can do research at scale and not have to pay for commercial measures.
I don’t like the fact that we have to pay so much money for our current measures. Some people will say, “Well, the measure is only $2.” Well, yeah, it’s only $2 for each patient. But if you see 1,000 patients for a research study, now that’s $2,000 of your budget that just went to a parent-report autism trait measure.
S: Can you tell me how this work started?
TF: We first developed a 33-item parent-report questionnaire of autism traits. The Diagnostic and Statistical Manual 5 (DSM-5) had only been out for few years at that point, and there weren’t any DSM-5-specific autism questionnaires that were widely disseminated and had really good psychometrics. So, for example, we specifically put on items for sensory sensitivity and sensory interest that would capture that domain of the DSM criteria.
We thought the 33-item version worked well, but the social domain was not as differentiated as we thought it should be. So we talked to families and parents and clinicians to try to help us write more items to capture more specific social factors. And then we also wrote a few more repetitive behavior items, which got us up to the 39-item version we just published.
S: Why did you go with this online survey approach, with so many neurotypical children in the sample, rather than using a sample of children who have confirmed diagnoses in more structured age groups?
TF: We know through SPARK and the Interactive Autism Network, for example, that when parents say, “My kid has an autism diagnosis,” it turns out they’re right a lot of the time. So we felt comfortable we’d get the sample we needed to see if the 39-item version is promising and useful.
We’ve been really impressed with how we could get a large sample quickly with Prolific and how we can get a fair number of parents saying that their kid has some developmental disability, including autism.
S: How quick?
TF: We got our data within a month. It has limitations, as you pointed out. But we needed a large and very diverse sample to do the kinds of psychometrics that we wanted to do.
S: You mentioned a reviewer on this paper wanted you to go back and do some more on its use as a clinical screener, which is what the Mail picked up on.
TF: Yeah. So we had done some of those analyses originally, and then we decided to kick them out of the paper, partly because of length, but also partly because it was all exploratory. We knew we were early in the process. But then the reviewer asked us to keep that piece. A lot of that journal’s readers are clinicians, and they are going to be very interested in whether this measure is useful for screening. So I know why the reviewers wanted us to keep it. But it was frustrating from our end because it’s the least important part of the paper.
S: You weren’t there yet.
TF: We weren’t trying to say with this paper, “Oh, you should immediately implement this for screening and identification.” That wasn’t our goal. It’s the first development paper. We’re trying to say the measure has a good factor structure. It’s reliable. It seems to be related to other measures of autism traits at a high level. It seems to be less related to measures of other things. We were just trying to show that the measure has good characteristics.
S: What is the status of the measure now, and where are you going from here?
TF: We’re offering it on Open Science Framework, and we’re collecting data from a clinical sample. In one to three months, we’ll have a good sample size there too.
S: How would you see this fitting into the existing library of measures? Is it an add-on, or will it replace certain things?
TF: Let me put it this way: I think it’s going to become a good option for situations in which people are starting data collection and want a good measure that maps on to the DSM-5 and has good psychometric characteristics, meaning a lot of differentiation of trait types. And they need something low-cost or free. To my knowledge, there’s nothing that’s open source and linked to DSM-5 and modern understandings of autism trait dimensions. I think this is going to fill that niche.
