Drug Court Efficacy vs. Effectiveness

More research has been published on the effects of drug courts than on virtually all other interventions for drug-abusing offenders combined. How, then, can the field continue to be in serious dispute about whether drug courts "work"? How is it possible for some reputable scholars to conclude that the success of drug courts has been definitively established (e.g., Meyer & Ritter, 2002), whereas others insist that drug courts are little more than a sham perpetuated by irrational believers (e.g., Anderson, 2001; Hoffman, 2002)?

The answer is at least three-fold. First, the more extensive the literature on an intervention, the greater the likelihood that it will contain conflicting findings that can lead researchers to different conclusions. To preserve unanimity, one should conduct a single study, declare victory, and then spread the word -- which happens all too frequently in the substance-abuse and criminal-justice fields. Like the old adage, "no good deed goes unpunished," if a field takes seriously its responsibility to carefully study its operations and impacts, it will almost certainly turn up some damning evidence.

Second, the more studies that are conducted on an intervention, the greater is the probability that some of the studies will have been poorly implemented, the data poorly analyzed, or the implications overstated. This leaves proponents open to the charge that they are relying on "junk science." Even if some well-designed studies do support the utility of the intervention, those studies may become unfairly tainted in the minds of critics, by association with poorer studies that reached the same conclusion or were mentioned in the same review papers.

Third, there are different standards of proof for establishing the efficacy of an intervention as opposed to its effectiveness. Efficacy refers to whether the intervention can be successful when it is properly implemented under controlled conditions, whereas effectiveness refers to whether the intervention typically is successful in actual clinical practice (e.g., Howard et al., 1996). Efficacy is a necessary, but not sufficient, condition for effectiveness, and is ideally established through randomized, controlled, experimental studies (e.g., Campbell & Stanley, 1966).

These three factors shed light on the most recent iteration of the drug court controversy being discussed on Join Together Online. In an August 2004 commentary, Kevin Whiteacre took to task the National Drug Court Institute (NDCI) and the White House Office of National Drug Control Policy for their "National Report Card" on drug courts in the U.S. (Huddleston et al., 2004). Mr. Whiteacre pointed out, correctly, that (1) the majority of drug-court program evaluations have used either no comparison group, or a biased comparison group such as offenders who refused or failed the drug-court program; (2) the majority of evaluations reported analyses only for program graduates (i.e., the most successful

cases) as opposed to the original "intent-to-treat" cohort; and (3) the GAO has issued reports faulting the data-collection methods used in the drug-court grantee self-report surveys administered by the former Drug Courts Program Office (DCPO).

These are valid points that have been echoed by other drug-court researchers, including my colleagues and myself at the Treatment Research Institute (Belenko, 1998, 1999, 2001, 2002; Marlowe, DeMatteo, & Festinger, 2003). Unfortunately, Mr. Whiteacre went beyond these appropriate criticisms to conclude that the "jury's still out" on the impact of drug courts. On this latter point, I believe he is mistaken.

It is true that many drug-court program evaluations are of such poor quality that the results cannot be interpreted from a scientific perspective. However, there are at least three randomized, controlled, experimental studies published in peer-reviewed journals reporting superior results for drug courts over traditional probationary conditions. These studies were conducted in the Maricopa County (Ariz.) Drug Court (Turner et al., 1999), the Baltimore City Drug Treatment Court (Gottfredson & Exum, 2002; Gottfredson et al., 2003), and the Las Cruces

(N.M.) DWI Court (Breckenridge et al., 2000). Among other positive findings, these studies revealed significant reductions in post-program criminal recidivism for drug-court participants lasting up to two and three years post-admission. A fourth experimental study of the Summit County (Ohio) Juvenile Drug Court also provided evidence for the superiority of drug court over standard adjudication; however, the small sample sizes in that study rendered the findings preliminary.

There have also been several "parametric" studies that are beginning to isolate the effects of the various "key components" (NADCP, 1997) of drug courts. For instance, using a randomized, controlled design, Adele Harrell, John Roman, and their colleagues at The Urban Institute have demonstrated that imposing graduated sanctions for positive urine drug-screens improved outcomes over standard pre-trial drug-court supervision (Harrell, Cavanagh, & Roman, 1998). Further, in a series of experimental studies, our research group demonstrated that frequent judicial status hearings improved outcomes for high-risk drug offenders who had more severe drug-use histories or a comorbid diagnosis of antisocial personality disorder (Festinger et al., 2002; Marlowe, Festinger, & Lee, 2003, 2004; Marlowe, Festinger, Lee, et al., 2003). These findings were replicated in three different jurisdictions, located in both urban and rural communities and serving both misdemeanor and felony drug offenders.

The latter studies are particularly relevant for establishing the efficacy of drug courts. It is very difficult to conduct the type of randomized studies with no-treatment control conditions that are necessary to scientifically prove the efficacy of an intervention. An alternative approach, however, to assessing the efficacy of drug court is to evaluate the effects of manipulating its core ingredients. Demonstrating that judicial status hearings have a significant bearing on drug-court outcomes establishes that drug courts have a unique mechanism of action. This provides scientific support for the utility of drug courts, and perhaps the only practicably obtainable evidence that the GAO and other stakeholders would be willing to accept.

Taken together, the results of these experimental studies prove the efficacy of drug courts beyond peradventure. The Food and Drug Administration (1998) requires only two experimental clinical trials to establish the efficacy of a new medication. It makes little sense to hold drug courts to a higher standard of scientific proof than we hold, say, cancer medicines. The fact that some program evaluation studies have been poorly implemented does nothing to detract from the scientific integrity of these well-designed studies. It may, however, raise questions about the effectiveness, nationally, of drug courts in day-to-day practice. Dozens of well-designed program evaluations have demonstrated the effectiveness of particular drug-court programs; however, the vast majority of drug courts in this country are not collecting the data elements necessary to document their services or outcomes.

The responsibility now falls to the drug-court field to establish performance benchmarks and best practices for drug-court programs, and to develop accreditation procedures that can be used to document whether a particular program is in compliance with professionally accepted standards of practice. Failing to do so would be a potential waste of money and a breach of consumers' trust. It would not, however, detract from the scientific evidence favoring the efficacy of drug courts. In short, the jury may still be out on how the universe of drug-court programs in this country is operating in practice, but the verdict is long overdue on the efficacy of drug courts.