Faulty use of data threatens early intervention for autism

Early-intervention programs can make a huge difference in children’s lives — particularly in the lives of those with autism or related conditions. We worry, though, that current procedures for assessing the performance of these programs could lead to cuts in financial support for the programs. Those assessment procedures need to change.

All U.S. states and territories provide services to infants and toddlers up to 3 years old who either have or are at risk of developmental delays. (The programs operate under Part C of the Individuals with Disabilities Education Act.)

The evaluations of the performance of early-intervention programs are based on reports that states submit each year to the U.S. Office of Special Education Programs. The office then determines whether the states need to take corrective actions. It might withhold funds from states whose programs are persistently rated as performing poorly, or it might require that the program modify its services¹.

However, this process has a big flaw: Children with autism in these programs tend to make slower progress than do those with less severe conditions, yet the performance assessment process does not take these differences into account. As a result, programs that enroll a relatively high proportion of children with autism or other complex conditions may improperly be judged as less effective than those with a smaller proportion of these children.

High costs:

Studies show that children with certain developmental challenges progress more slowly than others, no matter what treatment they receive. For instance, delays are more likely to persist in children who have trouble in multiple areas — such as communication, cognition and motor function, as is common in autism — than in children who struggle in a single domain².

A study published this month showed that children with autism tend to progress more slowly in an early-intervention program than do children with other conditions³. Children with autism are also less likely than children with delays unrelated to autism to have caught up to their typical peers by the time they leave the program.

The system does not consider these factors, however. As a result, states that enroll a high percentage of children with complex conditions, such as autism, are likely to report lower proportions of children making better-than-expected progress. This outcome could disproportionately hurt the programs that serve infants and toddlers with the greatest need.

Although we do not know for sure whether these evaluations have led to funding cuts, the most recent annual survey of state program coordinators, released in 2017, raises concern⁴. Multiple coordinators reported that their states are reconsidering their participation in Part C early-intervention programs. The main reason: anxiety over the high cost of these services and their ability to cover expenses. Most worrisome for children with autism, two states indicated that the “increased costs of serving children with complex needs” might cause their state to discontinue these early-intervention services.

Leveling the field:

To more accurately assess the performance of early-intervention programs, we recommend assigning a weight to results that adjusts for the proportion of children who have severe conditions. This tactic would create a level playing field for the programs and provide useful information for guiding program improvements.

This type of severity adjustment is often used when comparing medical outcomes. For example, outcomes in neonatal intensive care units may be adjusted for factors such as a child’s birth weight⁵. In early-intervention programs, outcome data could be adjusted based on staff evaluations of the severity of children’s conditions when they enroll.

We also recommend that states report results for subgroups of children, such as those with autism. And we suggest detailing the types and amounts of services that children with autism and their families receive.

This information would help to clarify which combinations of services are most useful to these families. Importantly, it would also provide a way to compare programs across a similar set of children.

Batya Elbaum is professor of teaching and learning at the University of Miami. Steven Rosenberg is associate professor of psychiatry at the University of Colorado.