Virtual games teach real-world skills to kids with autism

On 13 May, 2008, Matthew Belmonte received a curious e-mail from Google. For more than a year, Belmonte, an assistant professor at Cornell University, and a team of student computer scientists had been designing a dynamic video game to test the social, sensory and attentional abilities of children with autism.

Belmonte had set up a website explaining the project and the game, called Astropolis, in which children act as pilots of their own spaceships. The site attracted a lot of online traffic. “We figured it would only be a matter of time before we heard from Google,” he says, joking.

Chris Cronin, the business strategist who contacted Belmonte, had a vested interest in autism: He was part of a team working on SketchUp, Google’s three-dimensional drafting software intended for architects and professional designers.

Soon after SketchUp’s release, Cronin and fellow Colorado-based business developer Tom Wyman began hearing from users that children with autism love using the software. “You hear it once, and it’s a heartwarming story. Hear it twice, and it’s a coincidence. Hear it three times, and you think, ‘Gosh, there must be something going on here’,” Wyman says.

That led Cronin and Wyman, in late 2007, to launch Project Spectrum. The project initially sought to teach children with autism in the Boulder school system how to use SketchUp — giving them a creative outlet that they typically don’t have through writing, speech or social play.

Word spread quickly about the software, and now many people with autism, from all over the world, have downloaded it and built floor plans, buildings and landscapes of their own design. Project Spectrum’s ultimate goal is to help children with autism communicate more effectively, as well as build self-esteem and skills for a rewarding career.

Cronin’s email to Belmonte launched a collaboration in which members of Project Spectrum build parts of the Astropolis virtual world. In addition to teaching adolescents how to work cooperatively (they tend to get irritated and obstinate when programmers make even small modifications to their designs), Astropolis allows for the unprecedented testing of children with autism on a variety of cognitive skills, all at once, without the artificial, boring and anxiety-ridden setup of a typical psychology lab.

Gaming technology is more sophisticated, and more accessible, than ever before, and Belmonte is part of a burgeoning wave of researchers using it to test or train people with autism.

The first video and virtual reality games made for children with autism, developed about a decade ago, looked too simple and unrealistic, says Sarah Parsons, one of the leaders of these initial efforts and a senior research fellow in the School of Education at the University of Birmingham, UK. “Now they have the potential to be very realistic in the way that they look, and you control everything that’s seen and heard within them.”

In several studies over the years, Parsons has shown that children with autism usually recognize that virtual environments are meant to simulate reality¹.

Parsons is part of a new £1.65 million project in Europe, called COSPATIAL, which will place multi-player virtual reality programs into public schools, aiming to help high-functioning children with autism interact socially with their teachers and normal peers.

Using this technology is a particularly attractive method for studying autism because people with the disorder tend to have a knack for visual learning; some studies have even reported that they have superior visual acuity.

“They’re not so keen on interpersonal, face-to-face interaction, so there’s something comfortable about using the computer,” Parsons says. “And of course the visual stimuli appeal to a visual way of processing the world, which tends to favor people with autism.”

For one piece of the COSPATIAL project, which launched in February, Parsons is developing virtual games that can be projected on to a piece of hardware that’s already available in most UK classrooms: interactive whiteboards. To win the game, the children will need to work together with another player. “Their natural inclination is not to do that, but we hope that the games will be suitably motivating in order to help them learn social skills,” Parsons says.

The project is still in the development phase, which has included significant input from autism teachers. The team plans to start testing this and similar technologies in early 2010 in mainstream and specialty schools in the UK and Israel.

“We know that there are a lot of schools that are struggling to meet all the needs of young people on the spectrum,” Parsons says. “The impact will be quite substantial if we design this in a way that allows it to be used quite easily in typical school contexts.”

Out of this world:

Typical psychological experiments present two big problems for testing children with autism. First, they’re long and tedious. Children with autism tend to have an increased ability to focus, and can repeat certain tasks over and over without a fuss. Healthy controls, in contrast, get “bored out of their skulls,” Belmonte says.

“What you end up measuring is the difference between boredom and non-boredom, instead of the difference between autistic and non-autistic,” he says.

Second, psychological testing isn’t done in a realistic environment. Children come to an unfamiliar space, surrounded by strangers, and perform tasks that may have no relevance to their daily life. This is particularly challenging for children with autism, who don’t like novelty and are prone to social anxiety.

Belmonte envisioned Astropolis as a solution to both confounds.

For instance, one of the program’s mini-games, called ‘Maritime Defender’, puts a new twist on the classic ‘go/no-go’ test meant to measure inhibition and executive function. In the classic lab paradigm, children are taught to press a button, as quickly as possible, when they see certain cues, but to refrain from pressing the button in response to other cues. In Astropolis, children see a series of spacecraft coming out of a wormhole. Some are enemy ships, which they must shoot as soon as possible; others are ships full of valuable resources.

In a different game, children must steer their ship through a starry black sky. The pattern of stars they see through the windshield shows them when their ship is starting to steer off course — and how well they steer gives Belmonte a measure of their motion perception.

Astropolis even includes a Sally-Ann–type test of theory of mind, in which the player has to figure out where a space pirate believes that a precious resource is located.

“The games are really fun — even my lab has fun playing it,” Belmonte says.

So far, Belmonte has successfully piloted the games on healthy children and on some children with Project Spectrum. He’s recruiting more children with autism for the first set of experiments and plans to loan participants a laptop, so they can practice the game at home — with the machine recording their behavioral data all the while.

Once participants become comfortable with the game, Belmonte plans to bring them back into the lab, and have them play while they’re hooked up to electroencephalography electrodes, which can measure their brain activity.

“When we get these observations, a really interesting further question is, are social and non-social tasks being perturbed by some of the same sorts of neural properties? And if so, then could training in a non-social domain facilitate the development of social skills?” he asks.

One drawback of Astropolis is that it only works for individuals with high-functioning autism spectrum disorders, who can deftly use a mouse and sit in front of a computer for extended periods.

“It’s something that I honestly feel guilty about. None of the experiments I’ve done have addressed people like my brother,” Belmonte says, referring to his older brother with autism, who does not speak and cannot sit still in front of a computer for extended periods.

Down the line, Belmonte hopes to design similar games in which people with autism can experience full-body virtual worlds.

Social simulations:

Though virtual reality may be useful in measuring the perceptual and cognitive skills of people with autism, it has shown mixed results when used as a method of therapy.

Starting in the mid-1990s, computer scientist Dorothy Strickland began creating virtual environments that could help young children with autism learn safety measures, such as how to cross the street, or avoid a fire in their homes. Even with the cruder technology, these efforts worked fairly well².

“Safety skills are pretty easy to teach actions on, because they give a good practice scenario situation,” notes Strickland, a principal researcher at Do2Learn, which develops social learning games for children with disabilities, many of which are free to download on their website. She says the site gets about 10 million views per month.

Attempts to do social-skills training in virtual reality, on the other hand, have proven more difficult because they require subtle actions and vary a lot from one situation to the next.

For example, Strickland has completed a project in which people with autism learn how to enter a restaurant on Second Life, a popular online social world. Although the participants can learn a set of steps in one restaurant setting, they have trouble generalizing those lessons to a slightly different restaurant setting.

“We introduce different parameters. So if someone comes up to them and are rude, then the individual with autism may lose all connection to the right steps because they personally feel threatened,” she says.

Strickland’s team has found that it’s more effective to combine virtual reality scenarios with explicit instructional videos that explain why various skills should be applied to different situations.

Peter Mundy has run up against similar hurdles in his work on children with autism. “The hardest thing is figuring out how to help them generalize what they learn in one context and apply it in a different context in the real world,” says Mundy, director of educational research at the M.I.N.D. Institute at the University of California, Davis.

Mundy has a two-year National Institutes of Health stimulus grant to set up virtual reality situations that teach children with autism to improve key social abilities. Data collection began in December, and Mundy plans to run these experiments on at least 40 high-functioning children with autism.

In one game, for example, kids strap on a headset and are asked to talk about themselves with a group of virtual peers. If they don’t maintain eye contact, their friends will fade away.

“We think we can build virtual social interactive situations that will help kids have a lot of opportunities to practice,” he says.

Virtual training might even supplement more traditional behavioral interventions, he adds. “If we use it in conjunction with social-skills training, or some sort of person-to-person intervention,” he says, “we think the virtual practice will help the kids learn to apply what they are getting out of the intervention to a lot of different social contexts.”

References: