A Process and Impact Evaluation of a Federal Drug Court

Joseph Dule, PhD, Ohio Northern University

David L. Myers, PhD, University of New Haven

Photo by Noralí Nayla on Unsplash

Federal correctional populations, and particularly the number of individuals incarcerated or supervised for drug-related offenses, are a nationwide concern. At the federal level, recent data indicate there are over 222,000 individuals held in secure confinement, with nearly 162,000 sentenced to the Federal Bureau of Prisons and the other 60,000 being housed in pretrial detention (Motivans, 2021). Of the Federal Bureau of Prison population, nearly half (48%) exhibited a drug charge for their most serious offense. In comparison, about 6% were imprisoned for violent offenses, about 5% were incarcerated for property crimes, and around 18% were in federal prison for weapons crimes. In addition to the incarcerated population, the total number of adults under federal supervision in the community stands at 150,000 (Motivans, 2021). Of these individuals, about 15,500 (or 10%) are on federal probation, while approximately 111,000 (74%) are on supervised release following a period of incarceration. As with federal incarceration, nearly half of adults under federal supervision were convicted of a drug crime as their most serious offense. Furthermore, research indicates persistently high recidivism rates among known offenders. To illustrate, one study tracked over 25,000 federal offenders over an 8-year period, beginning in 2005 (Hunt & Dumville, 2016). About half were rearrested, almost one-third were reconvicted, and one-quarter were reincarcerated.

The federal government has demonstrated strong support for the drug court model, primarily through financial support of drug court programs, research, and other initiatives. For instance, each year the Bureau of Justice Assistance (BJA) and the Substance Abuse and Mental Health Administration (SAMHSA) distribute grants to states and localities to support the creation and enhancement of drug courts. In fiscal year 2017, over $100 million in federal funding was appropriated for drug courts (Sacco, 2018). More recently, the Biden administration publicly committed to end all incarceration for drug use alone and divert these individuals to drug courts and other treatment (Lekhtman, 2020), and to expand available funding for federal, state, and local drug courts (JoeBiden.com, 2021). Despite existing research findings and governmental support, little empirical research exists that focuses on federal drug courts. Consequently, there remains considerable need for further evaluation of federal drug courts and other specialized court programs. The current research aimed to address this gap in the literature.

The Program

This evaluation examined a statewide effort to process and treat federal offenders in the United States District Court of Connecticut (Dule et al., 2021a; Dule et al., 2021b). Beginning in 2009 and operating in three locations (Bridgeport, Hartford, and New Haven), the federal Support Court aims to identify drug-involved clients and assist them with addressing their risks and needs, by providing a rehabilitative atmosphere that empowers participants to lead law-abiding lives. Support Court requires regular attendance in court sessions before the Judge and Treatment Team, with enhanced community supervision, substance abuse treatment, random drug testing, and pro-social community activities expected (Connecticut Support Court Policies and Procedures Manual, 2016).

The target population of Support Court consists of individuals who are struggling with substance abuse and are at heightened risk for drug/alcohol use relapse, or whose past or current charged criminal conduct is attributed reasonably to drug and alcohol addiction. While the Support Court program itself is voluntary, post-conviction Support Court graduates are eligible for up to one year off of their supervised release term, and pretrial graduates may be eligible for favorable consideration in their criminal cases. Disqualifying criteria include offenders with mental health problems, serious medical issues, and those with histories of sex-related crimes, arson, serious firearm charges, violent crime, and any pending state felony charges. Each of the three divisional Support Courts is capped at 16 participants. The length of programming is at least one year; however, duration ultimately is determined by each participant’s progress.

In addition to treatment, counseling, and pro-social activities, each Support Court Judge addresses participant behavior through a range of incentives and timely sanctions. This may include verbal reprimand, increased reporting, community service, phase evaluation, no credit earned for week in phase, phase demotion, additional homework or writing assignments, or an overnight stay or day in detention. Participants themselves or team members can recommend sanctions, but ultimately the Support Court Judge makes a final determination on which incentives or sanctions are implemented. Conversely, a variety of incentives are possible, including certificates, verbal praise, applause, standing to be acknowledged, all-star participant of the month acknowledgement, gift cards, considerations for pretrial diversion, sentence reduction, and time-off of supervision.  

The Connecticut Support Court Policies and Procedures Manual (2016) does not specify the frequency of drug testing, but it does emphasize that drug testing is done frequently. The manual stipulates that during Phases I-IV, each participant will undergo “random drug testing.” In order to advance from each phase within the program, there is a required amount of “clean time.” In addition, prior to being admitted into the program, clients may be directed to take an initial drug or alcohol test. Participants may be required to be drug and alcohol free for up to 30 days prior to admission.

Process Evaluation

Initial research activities centered on conducting a process evaluation of Support Court operations and services, along with a descriptive assessment of various participant outcomes (Dule, 2021a). To guide the process evaluation of federal Support Court in the District of Connecticut, four research questions were specified:

1. Is the intended target population being served?
2. Are the intended personnel/stakeholders in place and engaged?
3. Are the intended services being provided?
4. Are the anticipated outcomes being achieved?

To answer these research questions, a mixed-methods research design was employed, involving the collection and use of both quantitative and qualitative data (Creswell & Creswell, 2017; Creswell & Plano Clark, 2017; Welsh & Harris, 2016). Data were collected from four sources:

Observational Data from Treatment Team Meetings and Support Court Sessions

During the spring and summer of 2018, members of the research team visited and directly observed each of the three federal Support Courts operating in Connecticut. Direct observations were made of team meetings, followed by observations of the court sessions. 

Service Record Data Pertaining to Support Court Participants

Members of the research team collected quantitative data from the federal Probation and Pretrial Services Automated Tracking System (PACTS). This case management system contains information on all federal defendants, including those processed in the District of Connecticut. Data pertaining to Support Court participants from 2009-2017 were collected and coded for statistical analysis. Types of variables included demographics, Support Court duration and outcome, risk scores, history of substance use and treatment, mental health history, previous criminal and violent behavioral patterns, sentencing information, housing, employment, drug testing, pre- and post-conviction conditions, and recidivism data.

Survey Data from Support Court Participants

During summer 2018, members of the research team collected survey data from current Support Court participants (N = 22). Survey items captured participant perceptions of the Support Court judge, probation officers, treatment staff, assistant U.S. attorney, federal defense attorney, numerous aspects of Support Court, and participant preparedness for future success.

Survey Data from Support Court Stakeholders

Also, during summer 2018, members of the research team collected survey data from members of the Treatment Team at each location (N = 20). Survey items captured stakeholder perceptions about Support Court operations, interactions among team members, treatment and services offered, use of data and evidence-based approaches, and use of incentives and rewards.

Process Evaluation Findings

Is the intended target population being served?

According to the Connecticut Support Court Policies and Procedures Manual (2016), the target population consists of individuals experiencing either pretrial or post-conviction supervision who exhibit current or past substance abuse and/or are at heightened risk of relapse. Official data derived from PACTS confirmed that both pretrial (N=69) and post-conviction (N=140) participants were being served. In terms of various risk factors exhibited by the target population, the average number of prior arrests for Support Court participants was 5.55, with a statistically significant difference (p< .05) between pre-trial (mean of 4.49 arrests) and post-conviction (mean of 6.07 arrests) participants. Consistent with Drug Court best practices (NADCP, 2018), Connecticut Support Court utilizes evidence-based assessment tools (i.e., PTRA, PCRA, and RPI) to assess participant risk scores. Among all Support Court participants, the average RPI score was 4.38 (RPI scores can range from 0-9); the average PTRA score was 3.42 (between Category 3 and 4, with Categories ranging from 1-5) and the average initial PCRA score was 2.43, while the ending PCRA average score was 2.3 (between low/moderate and moderate, with four possible risk categories). These scores suggest that Support Court clients are generally moderate risk. It is likely that some of the disqualifying criteria such as sex-related crimes, arson, serious firearm charges, & violent crime, may exclude many “high risk” offenders from program participation.

In terms of participant drug and alcohol risk factors, the study found that substance use generally began at a fairly young age (mean age of 14.63 years), suggesting that the target population (whose mean age was 36.77 years at program start date) typically exhibited a long-term or chronic history of drug and/or alcohol use. Approximately two-thirds (68.9%) of the participants were convicted of a drug charge, and 91.9% were convicted of a felony offense. Finally, from the Support Court stakeholder perspective, survey results (N=20) indicated that stakeholders “agree” (mean score of 4.05 out of 5.0) that the program is reaching its defined target population.

Are the intended personnel/stakeholders in place and engaged?

The Treatment Team members in Support Court include the U.S. District Judge at each Support Court location, the United States Attorney’s Office, the Federal Defenders Office, the United States Probation Office, and treatment providers. On-site observations confirmed that representatives from all intended stakeholders were present for team meetings and Support Court sessions. Despite a few stylistic differences (e.g., the level of formality within a particular court session), each treatment team appeared highly committed to its mission and engaged in its work. Team member survey results confirmed perceptions that the Support Court team works cohesively, members understand each other’s perspective, decisions are reached collaboratively, and team members value other team member’s input and recognize their contributions to the success of the program.

Are the intended services being provided?

Service record data from PACTS revealed a majority of Support Court participants were currently involved in outpatient treatment (61%), while 10% were receiving in-inpatient treatment, and 16% were involved in self-help treatment. About a quarter of participants (23%) were receiving mental health treatment, while 11% received therapy within the past year. Half of the participants (50%) presented no evidence of a mental health condition, while about a quarter (24%) had a history of mental health issues, but no active problems. Survey data further revealed that stakeholders believe participants are receiving the appropriate dosage of treatment, treatments are individualized, needs are being met, the Judge values treatment-providers’ recommendations, and that the Support Court has a satisfactory network of treatment providers in place.

In terms of the Support Court program’s goal of improving participant access to education and employment, the results from the participant survey were somewhat less favorable than in the area of treatment. More precisely, participant survey scores pertaining to questions about the program’s assistance with education, employment, and housing were comparatively lower than perceptions of treatment scores, indicating less consensus about how successful the program has been in these areas. In terms of drug testing, stakeholders agreed that drug tests are occurring on a timely basis, results are communicated quickly to the Support Court team, and that sufficient precautions are in place to prevent drug test tampering. Participants also agreed that drug testing was helpful for their sobriety.

Are the anticipated outcomes being achieved?

Results from the process evaluation also indicated that several of the anticipated individual and community outcomes identified in the Policy and Procedures Manual (2016) were being achieved. Survey responses of participants pertaining to sobriety and remaining drug-free, being involved in a recovery lifestyle, employment, independence, housing, life skills, mental and physical well- being, coping skills, trust, self-worth, and family relationships suggested that the program was making a positive impact. Participants also generally believed that the program had a positive impact on how their family viewed the criminal justice system. Analysis of PACTS data further revealed that participant housing stability and employment status significantly improved while participants were in the program. Finally, about half of all Support Court participants either formally graduated or otherwise left the program successfully. While this finding is similar to completion rates uncovered in other drug court research (Brown, 2010), it also means that about half of the Support Court participants were terminated or otherwise left the program unsuccessfully.

Impact Evaluation Description

A subsequent impact evaluation (Dule et al., 2021b) entailed an assessment of three recidivism measures contained in PACTS (arrest, drug test failure, and revocation of supervision). Support Court participants were compared with similar federal justice-involved individuals who did not participate in Support Court. The comparison group members experienced post-conviction supervision in the United States District Court for the District of Connecticut during the period of 2010 through the first 6 months of 2018, and they had drug and alcohol treatment ordered as part of their supervision conditions. Importantly, they did not experience Support Court during their pretrial services or post-conviction supervision periods. In general, the 232 individuals in this non-Support Court group were compared to the 182 Support Court participants from 2009 to 2017 who had progressed to post-conviction supervision by the start of 2018 (i.e., recidivism was assessed during the post-conviction supervision period for both groups). Propensity score matching was used to create two groups that were statistically equivalent on measured demographic and legal variables (Apel & Sweeten, 2010; Beal & Kupzyk, 2014; Rosenbaum & Rubin, 1983). Utilizing one-to-one matching procedures, 116 Support Court participants were closely matched with 116 comparison group members. This technique was used to produce individually matched subjects in the treatment and comparison groups, whereby each pair of matched subjects had nearly identical propensity scores.

Impact Evaluation Findings

During the post-conviction supervision period, the Support Court participants were: 

• Significantly less likely to fail a drug test (42% versus 56%).
• Significantly less likely to be arrested (33% versus 47%) and experienced a significantly lower hazard of rearrest by about 44 percent.
• Insignificantly more likely to have their supervision revoked (22% versus 13%), primarily due to being insignificantly more likely to be revoked on technical violations (13% versus 5%).

An additional set of analyses considered whether there were differences in recidivism outcomes for successful Support Court participants (i.e., those who formally graduated or otherwise left Support Court successfully) and unsuccessful Support Court participants (i.e., those who were terminated or otherwise left Support Court unsuccessfully), as compared to members of the comparison group. Based on the matched treatment and comparison groups (N=232), odds ratios from the logistic regression models indicated:

• Unsuccessful Support Court participants were significantly more likely to have their supervision revoked (over 3 times more likely) compared to non-Support Court group members.
• Successful Support Court participants were significantly less likely to fail a drug test (approximately 65% less likely) compared to non-Support Court group members.
• Successful Support Court participants were significantly less likely to be arrested (approximately 61% less likely) compared to non-Support Court group members.

Conclusions and Recommendations

Overall, results from the process and impact evaluations were quite favorable for Support Court participants. Overall, the evaluation revealed strong adherence to NADCP best practices and to the program’s stated policies and objectives. Importantly, compared to the matched non-Support Court individuals (N=116), the Support Court participants (N=116) were significantly less likely to be arrested and significantly less likely to have failed a drug test, and they experienced significantly longer times to rearrest. Successful Support Court participants were approximately 65% less likely to fail a drug test and approximately 61% less likely to be arrested than non-Support Court group members.

While the findings of the process and impact evaluations were generally favorable, various areas for improvement were identified. Recommendations included the following:

1. Support Court personnel should review and discuss the available target population and program size. Emphasis should be placed on identifying and enrolling medium and higher risk participants, as research suggests these individuals can benefit the most from effective treatment.
2. Support Court personnel should review and discuss the use of data and evidence-based programs and practices. Data-driven decision-making is a key aspect of evidence-based organizations, and evidence-based programs and practices are those that have attained the highest degree of research support. Use of risk and needs assessment to identify medium and higher risk individuals, and then match them with appropriate and effective services, is one key principle for evidence-based corrections.
3. Support Court personnel should review and discuss participant perceptions of treatment versus their perceptions of housing, education, and employment. Survey data indicated participants expressed high satisfaction with treatment services, but somewhat less satisfaction with housing, education, and employment services.
4. Support Court personnel should review and discuss frequency of drug testing and drug test results. On average, Support Court participants experienced about one drug test every two weeks, or two drug tests per month. Depending on cost, along with participant availability and other obligations, more frequent drug testing may be worthwhile, particularly for those participants in earlier stages of Support Court.
5. Support Court personnel should review and discuss the data and findings on program

completion rates. Although program completion rates are in-line with findings from existing drug court literature, it is possible that increasing successful program completion would have a beneficial impact on other behavioral outcomes.

6. Finally, it should be stressed that future drug court programs should strive to embrace recognized “best practices,” many of which have been established in state and local drug court evaluations. Programs that emphasize these strategies and techniques consistently demonstrate stronger positive effect sizes (Marlowe et al., 2016).

References

Apel, R. J., & Sweeten, G. (2010). Propensity score matching in criminology and criminal justice. In A. R. Piquero & D. Weisburd (Eds.), Handbook of quantitative criminology (pp. 543–562). Springer.

Beal, S. J., & Kupzyk, K. A. (2014). An introduction to propensity scores: What, when, and how. Journal of Early Adolescence, 34(1), 66–92.

Brown, R. T. (2010). Systematic review of the impact of adult drug-treatment courts. Translational Research, 155(6), 263-274.

Connecticut Support Court Policies and Procedures Manual (2016). The United States District Court for the District of Connecticut.

Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Thousand Oaks, CA: Sage Publishing

Creswell, J. W., & Plano Clark, V. L. (2017). Designing and conducting mixed-methods research (3rd ed.). Thousand Oaks, CA: Sage Publishing.

Dule, J., Myers, D. L., Earl, K., Wang, M., & Daty, T. (2021a). An evaluation of federal Support Court in Connecticut. EBP Quarterly, 6(1).

Dule, J., Myers, D. L., & Kringen, J. (2021b). Assessing the impact of the District of Connecticut’s Support Court using propensity score analysis. Federal Probation, 85(3), 14-20.

Drug Courts Program Office, US Department of Justice. (1997). Defining drug courts: The key components. Retrieved from https://www.ncjrs.gov/pdffiles1/bja/205621.pdf

Hunt, K. S., & Dumville, R. (2016). Recidivism among federal offenders: A comprehensive overview. https://www.ussc.gov/sites/ default/files/pdf/research-and-publications/ research-publications/2016/recidivism_ overview.pdf

JoeBiden.com. (2021). The Biden Plan for strengthening America’s commitment to justice. https://joebiden.com/justice/

Lekhtman, A. (2020). Joe Biden’s unwelcome plan to expand coerced treatment and drug courts. https://filtermag.org/joe-biden- forced-drug-treatment/

Lowenkamp, C.T., Holsinger, A.M., & Latessa, E.J. (2005).  Are drug courts effective: A meta-analytic review.  Journal of Community Corrections, 15(1), 5-11.

Marlowe, D. B., Hardin, C. D., & Fox, C. L. (2016). Painting the current picture: A national report on drug courts and other problem-solving courts in the United States. http://www.ndci.org/wp-content/ uploads/2016/05/Painting-the-Current- Picture-2016.pdf

Motivans, M. (2021). Federal justice statistics, 2017–2018. Washington, DC: U.S. Department of Justice, Bureau of Justice Statistics.

National Association of Drug Court Professionals. (2018). Adult Drug Court Best Practice Standards. Vol. II. Retrieved from http://www.nadcp.org/wp- content/uploads/2018/03/Best-Practice-Standards-Vol.-II.pdf

Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity scores in observational studies for causal effects. Biometrika, 70, 41–55.

Sacco, Lisa. (2018). Federal support for drug courts: In brief. 20 March. https://fas.org/sgp/crs/misc/R44467.pdf

Welsh, W. N., & Harris, P. W. (2016). Criminal justice policy and planning: Planned change. New York: Routledge.