How Dennis Wall became the ‘bad boy’ of autism research

It was a sunny California afternoon in January 2015 when Dennis Wall received an unexpected gift: ‘smart glasses’ made by Google that had failed to live up to their hype in the press.

An employee from the company pulled up to Wall’s lab at Stanford University in a sleek gray Tesla, popped open the sedan’s trunk and unloaded a brown cardboard box with long, dangling cords.

It was a scene straight out of the television comedy “Silicon Valley,” which satirizes the absurdity of the tech world. Wall’s ambition for the Google Glass, however, is dead earnest: He aims to help people with autism interpret others’ emotions.

Many people with autism have trouble understanding social cues and emotions, and this can greatly limit how they fare in the world. Wall developed an algorithm that relies on artificial intelligence. His plan was to incorporate the algorithm into the glasses, so that someone wearing the glasses would see a tiny emoticon that matches the expression on the face of another person.

The algorithm was all set to go, and Wall had been waiting for the glasses to test his idea.

“It was lifesaving for us because we were desperate to get started,” he recalls.

This project represents just one of Wall’s lofty ambitions. Among the others: trimming the number of questions — and, as a result, the amount of time — required to diagnose autism, using machine-learning techniques to treat and diagnose the condition, and crowdsourcing data to map the prevalence of autism in the United States^1,2.

And with his smarts and charisma, Wall seems like just the man to pull all of this off. In his trademark black high-top sneakers, skinny jeans and spiky dark hair, Wall fits right into the milieu in Silicon Valley (albeit less so at scientific conferences). He parlays his proximity to this area into powerful allies — including Thomas Insel, the former National Institute of Mental Health director who co-founded a startup called Mindstrong. (Insel served a stint at the Google spin-off Verily.) Wall also founded Cognoa, a company that aims to diagnose autism via an app on parents’ smartphones. Cognoa has raised more than $20 million in venture capital and is seeking approval from the U.S. Food and Drug Administration.

With these ideas and energy, Wall, at just 43, seems poised for success. But his execution of these ideas leaves something to be desired, his critics say.

“Without reproducibility and transparency, it’s marketing and advertising.” — Matthew Goodwin

Some of the stalwarts in the autism field view Wall’s work as “poor,” “pseudoscience,” and “dangerous.” Wall’s results are tantalizing, given their ambitious premise, but few of them stand up to scrutiny, they say. In 2012, for instance, Wall published findings suggesting he could shorten two gold-standard diagnostic tests for autism. But three years later, another team reported that they were unable to replicate this finding³.

“He just keeps saying these things that are not true,” says Catherine Lord, director of the Center for Autism and the Developing Brain at New York-Presbyterian Hospital, and one of Wall’s most vocal critics. Lord created the two gold-standard tests and was part of the team that questioned Wall’s work.

Wall is “passionate” and “his heart is in the right place,” says Fred Shic, associate professor of pediatrics at the University of Washington in Seattle. But Shic, who also studies the use of artificial intelligence to improve autism screening, says Wall’s early work is flawed.

Wall’s 2012 algorithms boast nearly 100 percent accuracy, so despite their flaws, they make it difficult for others to publish anything with lower numbers — even work that is more rigorous. “It makes things harder for people who are trying to apply appropriate methods to these data-science approaches,” Shic says. He adds, though, that Wall’s recent work is more careful.

Wall, the self-described “bad boy” of autism research, goes quiet when he hears some of this criticism.

“I guess I just don’t worry about it because what we’re doing, for me, is so much more important than politics and my image and how I’m perceived in the field,” he says.

From bayside to bayou:

Wall grew up in a small Massachusetts town on Cape Cod. He loved the outdoors and spent hours wandering the coastal marshes and woods.

His father was in and out of the picture. His single mother was resourceful but overworked, finding jobs as a liquor store clerk, a house cleaner and, eventually, as a nighttime security guard. Her work schedule gave Wall and his brother Erich free rein to “raise each other,” Wall says.

With little support at home, Wall says he threw himself into sports and school to keep himself motivated. When he was around 11, he started running, and was so fast he was invited to join the U.S. Junior Olympics cross-country team. He traveled across the East Coast, and often placed in the top three.

Later, he and his brother took city buses or trains two hours each way to attend Boston College High School in Dorchester, a historic Boston neighborhood. The all-boys college preparatory school has an excellent reputation and a price tag to match; the brothers earned scholarships to help pay their way. Wall excelled in his studies, played lacrosse, swam and continued running. After school, he’d trade in his formal school shoes and collared shirt for sneakers and a skateboard — despite his mother’s disapproval. “[Skateboarding] is really physical, you know; it requires a lot of intensity, energy, balance and focus. For me, it was great,” he says. He still takes time to indulge in this hobby — three skateboards and two large surfboards adorn his office walls — or in other intense physical activity every day.

When it was time to apply for college, he set his sights mainly on small liberal arts colleges near Boston. But a hefty scholarship to Tulane University in New Orleans, Louisiana, brought him south to the bayou. He had always been interested in biology, but dove into his studies at Tulane with a passion. He read Charles Darwin’s “On the Origin of Species” (and revisits it a couple of times each year). He took every microbiology and ecology course that fit into his schedule and, in his third year, helped teach a genetics course. To earn money, he cleaned crawfish tanks in a molecular biology laboratory, pressed and mounted leaves and stems at the school’s herbarium and counted spider sperm in an arachnology lab.

When he decided to pursue a Ph.D., integrative biology seemed like a natural fit for his wide-ranging interests. At the University of California, Berkeley, he sought out moss expert Brent Mishler and was consumed by fundamental scientific questions, such as how evolution affects disease and the diversity of species.

Mishler remembers Wall as precocious and always willing to try new methods. For instance, he says, Wall adapted statistics typically used to map epidemics to plot the timing of the lineages of species.

“He’s got a good mind for taking something that might be useful in one area and then recognizing that oh, it’s actually useful in this area, too,” Mishler says.

Wall’s curiosity took him to islands in southeast Asia and Tahiti, where he did field work chasing down new species of moss. “He would do these dumb things where he would go out overnight in the jungle and he’d get bitten by centipedes,” says Daniel Rubinoff, professor of entomology at the University of Hawaii. “He’s always been a bit crazily committed to his craft.”

Rubinoff, who met Wall in graduate school in the mid-1990s, is among Wall’s biggest supporters. He says the people who criticize Wall don’t know him well — and misunderstand his motives.

“He’s not devious, he’s not sneaky. If he thinks something, he tells you — and he cares deeply,” Rubinoff says.

Solving problems:

Wall also excels in creative approaches to his scientific ideas, Rubinoff says. “He’s succeeded more than anybody I know in — I don’t want to say ‘inventing himself’ because that sounds like a magician — but in following his interests and developing them into legitimate scientific enterprise.”

For example, Wall learned how to extract DNA and sequence genes from the moss samples he collected. Later, as a postdoctoral fellow studying computational genomics at Stanford, he became proficient in applying algorithms to genetic data. “Knowing all that stuff really armed me with the ability to think about solving medically focused problems,” Wall says.

In 2002, when he was a postdoctoral fellow, Wall was courting Abby Keefe, a woman he grew up with who still lived in the Boston area. After exchanging letters for a few months, he moved from the Bay Area to Boston, and they married in 2004.

Wall also spent a lot of time with Keefe’s sister Becky, who has severe autism, in his high school years and on vacations during college and graduate school. “That kept my interest in autism because I knew what it looked like; I hung out with Becky a lot,” he says. He began thinking about how his new expertise in systems biology could be applied to studying autism.

In late 2003, Harvard University hired Wall to lead its computational biology initiative. In this role, Wall collaborated with researchers at Boston Children’s Hospital to analyze gene-expression data from children with autism. His hope was to compare them with data from typical children to arrive at clues about genes that contribute to the condition.

He had made his first foray into autism research and, more importantly, had spotted an area where his computational skills could be uniquely useful.

By Wall’s own account, this first project was a failure. He didn’t spot any differences in the autism group that hinted at a biomarker for the condition. And he never published the findings.

He remembers having the realization that he had a lot to learn about autism, and a good starting place might be to examine the tools used to diagnose the condition. He began looking at the Autism Diagnostic Observation Schedule (ADOS) and the Autism Diagnostic Interview-Revised (ADI-R).

Under pressure:

These two tests, both developed by Lord in the 1980s, comprise standardized questions and criteria to guide clinicians making an autism diagnosis. But to Wall, the tests seemed much too time-consuming and vulnerable to human error.

“That subjectivity, whether conscious or unconscious or whatever, can impact our ability to identify a statistical signal,” he says. “What if it’s a ‘garbage in, garbage out’ problem?”

His skepticism set him on a path to make autism diagnosis more objective and efficient. He set out to use a machine-learning technique to truncate both tests. He cut down the ADOS from 29 behavioral ‘codes’ to just 8, and the ADI-R from 93 questions to just 7.

His team reported that the eight-code version of the ADOS performed with nearly 100 percent accuracy in a sample of 612 children with autism and 15 controls. The shortened ADI-R was also almost 100 percent accurate in a sample of 891 people with autism and 75 controls.

These results quickly came under scrutiny. In 2015, Lord and a team of computer scientists applied Wall’s shortened tests and reported that they could not reproduce either of the findings.

Their paper laid out a laundry list of design flaws in Wall’s ADOS project. For instance, the ADOS categorizes children into three categories: ‘autism,’ ‘autism spectrum’ and ‘non-spectrum.’ But Wall omitted children who score in the intermediate ‘autism spectrum’ range, and included only those easiest to distinguish from each other: those with the most severe features of autism and those without the condition.

The researchers also harshly criticized Wall’s statistical analysis. “It appears that no effort was made to evaluate the reliability or validity of their results aside from peripheral reporting of accuracy on the test data,” they wrote.

Matthew Goodwin, one of the researchers, says this is just one example among many of “one-shot” studies led by Wall. “Without reproducibility and transparency, it’s marketing and advertising,” Goodwin says. “I have not seen any wide-scale reproducibility of any of the results in autism and around diagnosis that Dennis has put out there.”

Still, Wall’s efforts to improve diagnosis proceeded apace. In 2014, he reported that non-experts can diagnose autism by evaluating home videos of children that parents upload to YouTube⁴. Experts disparaged this project, too: “This is a very poorly designed study,” Lord told Spectrum at the time.

Wall says these criticisms of his studies are unfounded. “The original [2012] work we published was not flawed,” he says, and “there is nothing scientifically unsound about our 2014 video study. It was rigorously reviewed, revised and ultimately accepted for publication in a strong and highly regarded journal.”

Perhaps the disagreements with Lord and others are the natural result of merging two disparate fields — computer science and clinical practice — he suggests. “When such different fields come together, I think great things can happen, but the potential for some tension exists.”

“What we’re doing, for me, is so much more important than politics and my image.” — Dennis Wall

Lord also takes issue with Wall’s assertion that lengthy tools such as the ADOS delay children’s access to an autism diagnosis. She says there are indeed waiting lists at top-notch clinics, but that most children can be assessed either before school through early-intervention programs, or in their school districts. In fact, she says, they are entitled by law to such an assessment.

“It just drives me crazy when he goes on and on and on as if he is solving that problem and then blaming it on the ADOS, which has nothing to do with that,” she says. And Wall’s suggestion that a quick diagnosis may be sufficient, she says, ignores the real issues families face: a shortage of well-trained clinicians and insurance reimbursement for the tests.

High stakes:

Wall’s supporters say some of these criticisms are too personal — and short-sighted.

“You can disagree with somebody and be frustrated by what they are doing, but you don’t go that far,” Rubinoff says.

And Wall’s wife, Abby, says the remarks cut deep. “This really hurts him to the core because he’s trying to do a mission that is personal to him but also really important work for the field,” she says. “My only guess is perhaps people who have one way of doing something, which we all find in life, have a hard time with change and perhaps even change in technology.”

Wall’s work may be preliminary, but it shows promise. And he has shown integrity even when he’s wrong, says John Constantino, professor of psychiatry and pediatrics at Washington University in St. Louis.

For example, Wall published a paper in 2014 that included incorrect information about the Social Responsiveness Scale, a widely used autism screen that Constantino developed⁵. Constantino called Wall and says he was pleasantly surprised when Wall apologized and promptly issued a correction.

“What Dennis is doing is the new generation of empirically based assessment,” Constantino says. (Wall and Constantino have never collaborated, but Constantino is on the scientific advisory board for Wall’s company, Cognoa.)

Wall draws strength from these few supporters and is continuing to forge ahead with his goals.

These days, he splits his time between Cognoa, his lab, his family and his skateboarding. He continues to win big private and federal grants for his projects. He is collecting fecal samples from children with autism in hopes of analyzing their microbiomes. He and his collaborators are sequencing the genomes of people with autism and their families.

And he is still plugging away at his emotion-recognition project.

At an autism research conference in San Francisco last year, graduate students from Wall’s lab drew a thick crowd in the massive hall. Attendees lined up to take a turn with the Google Glass, which Wall had integrated with the emotions tool. As the scientists put on the glasses, they saw yellow emoticons on the corner of the lens, mirroring the emotions of the people they chatted with. Then they took the glasses off.