The new work aimed to evaluate the reliability of studies that examine the effectiveness of early interventions. Conventional wisdom on therapy says “the earlier the better, and the more the better,” Sandbank says.
But the evidence supporting that idea is weak, she and her colleagues found. Several sources of bias threaten to undermine the reliability of many early-intervention studies: selection bias, when experimental groups and control groups are not randomly assigned; detection bias, when the same person administers an intervention and judges its effectiveness; and attrition bias, when participants from a control or experimental group disproportionately drop out over time.
The number of studies at high risk for selection or attrition bias decreased from 1989 to 2017, Sandbank and her colleagues found, drawing on data collected for their 2020 study on early interventions. But both types of bias were still common among the studies they reviewed, and 50 to 75 percent were at high risk for detection bias.
Many studies also leaned heavily on reports from parents or caregivers to assess the effectiveness of an intervention, instead of relying on clinical observations, which studies show are more reliable. Between 25 and 50 percent of the studies from 2011 to 2017 relied on parent reports, down from around 90 percent in 2000.
The team’s 2020 meta-analysis concluded that many early interventions had positive effects. But once they excluded studies that relied on caregiver reports or had a high risk of one or more types of bias, there were often too few studies left to assess an intervention’s effects.
