Evaluation Results from the Somerset County Day Reporting Center

• David L. Myers, PhD, University of New Haven
• Daniel R. Lee, PhD, Indiana University of Pennsylvania
• Dennis M. Giever, PhD, New Mexico State University

In September 2015, Somerset County, PA, received a 3-year “Smart Supervision: Reducing Prison Populations, Saving Money, and Creating Safer Communities” grant from the Bureau of Justice Assistance (under the Department of Justice, Office of Justice Programs). The goals of this funding program, now known as the “Innovations in Supervision Initiative” (ISI), are to develop and test innovative strategies and implement evidence-based probation and parole approaches. In turn, ISI seeks to improve supervision success rates and increase community safety, by effectively addressing client risk, needs, and recidivism. Receipt of grant funding in Somerset County followed previous successful efforts directed at justice system strategic planning, cross-systems mapping, and implementation of evidence-based approaches.

As a result of the federal funding, the Somerset County Day Reporting Center (DRC) was created. Subsequently, in providing services to clients, risk and needs assessment was utilized, evidence-based programs and practices were provided, client data were collected, quality assurance tools were employed, and process and outcome evaluation occurred. Evaluation results were produced and reported on previously (Myers, Lee, & Giever, 2018a, 2018b).

Overall, during the three years of federal grant funding, the Somerset County DRC was implemented effectively. Data collected from multiple sources suggested DRC programming and practices were implemented as intended, organizational culture was positive, and participants benefited from the DRC experience. Program participants received a variety of evidence-based services, resulting in improved perceptions of the criminal justice system, lowered risk of recidivism, and a lowered likelihood of rearrest, particularly for participants who graduated from the program. This report provides updated data and findings based on participants admitted to the DRC through March 2019. Data were obtained through DRC participant surveys, risk assessment using the COMPAS tool, and publicly available criminal records. The findings provide additional support for DRC implementation and effectiveness.

Program Participant Survey

Table 1 presents the results of a survey administered to participants who completed specific programs offered through the DRC. The first six items on this survey were scored from 1 (strongly disagree with the statement) to 6 (strongly agree with the statement), while the final item was scored from 1 (very bad) to 5 (very good). Higher scores are associated with more positive views of DRC programming.

Overall, the results of the participant survey were favorable, with all average scores on the first six statements being above 4.0 (slightly agree), and 90 of the 114 (79%) average scores being 5.0 (agree) or better. In addition, all average scores based on the final statement are 4.0 (good) or higher. Although these participant survey results were positive, survey findings were used by DRC staff and service providers to make collaborative adjustments to DRC programming and improve program implementation and fidelity.

Table 1: Participant Survey Results

Notes: AM = Anger Management; FC = Family Center; FS = Family Strengthening;

PREP = Prepared Renters Program; STEPS = Steps Toward Employment Program Success;

IY = Incredible Years; RP = Relapse Prevention; SC = Stages of Change;

WRAP = Wellness Recovery Action Plan; MRT = Moral Reconation Therapy

Numbers in parentheses represent number of completed surveys for each program.

Other numbers in table represent average score for each survey item.

First 6 items were scored from 1 (strongly disagree) to 6 (strongly agree)

Last item was scored from 1 (very bad) to 5 (very good).

Criminal Justice Perceptions Survey

A pre-test/post-test survey was administered that focused on perceptions of treatment by the criminal justice system. Initially, the survey was completed anonymously by participants entering the DRC, and the follow-up survey was completed anonymously at discharge. Pre- and post-test responses cannot be linked or compared for specific individuals; however, pre- and post-test group responses can be assessed. Items on this survey were scored from 1 (strongly disagree with the statement) to 6 (strongly agree with the statement), except for one item scored from 1 (very bad) to 5 (very good). Higher scores are associated with more favorable perceptions.

The results of the client satisfaction survey for the first 139 participants to enter the DRC, including the first 78 participants who were discharged (either successfully or unsuccessfully), appear in Table 2. Of the 78 individuals included in the post-test, 55 (70%) successfully completed DRC programming, and 23 (30%) were unsuccessfully discharged.

Table 2: Client Satisfaction Survey Results

** p < .001

Note: Items were scored from 1 (strongly disagree) to 6 (strongly agree), except for the one item that was scored from 1 (very bad) to 5 (very good)

Results from the criminal justice perceptions survey were positive, with average scores on the four comparable items improving significantly from the pre-test to post-test. Post-test scores all indicate consistent and strong agreement with the statements provided, suggesting high client satisfaction with DRC services and staff, as well as perceptions of fair treatment.

In addition to these findings, qualitative comments provided by participants on the pre-test and post-test surveys support the quantitative results and indicate a high degree of optimism associated with DRC programming and personnel. To illustrate, on the pre-test survey, several individuals voiced displeasure with the criminal justice system:

My rights were never read to me. When questioning the courts, what jurisdiction they were judging under, they refused to give me the cause and nature of the charges against me. Also, I asked who the injured party was, as outlined by the rule of Corpus Delecti, which states that an injured party must be present. The courts refused to answer me. I feel that my rights have been violated with no compensation to me.

I feel that the justice system should be updated more regularly and people with addiction problems should receive more help rather than more time in prison and should not be grouped in with people who have more serious crimes. Prison doesn't help addicts with no new charges and have been doing well. Addiction is always going to be a problem for an addict no matter how much clean time they have, and they should not be incarcerated for long amounts of time for falling short of staying clean.

I feel like you should not be represented by people that work for the people that are accusing you. You can’t be rightfully represented, in my case my lawyer lied to me the whole time, and when we got to the court room it was a completely different story, so I feel like they don't care about the problem or the solution. They just want the circle to keep going so they can make more money.

Could have helped me more instead of just throwing me in jail

They judge you without knowing who you are.

During the pre-test survey, however, there also were various positive comments offered regarding the DRC. For example:

The criminal justice system has given me an opportunity to make things right. I have struggled with addiction my whole life, and between Twin Lakes and this, it gives me an opportunity to better my life, live addiction free, and live my life to the fullest.

I am thankful in a way for this program because I feel optimistic that I will be able to learn to live a clean life and be a more productive member of the community. I know it’s a little early to tell, but I really hope and think my life is going to start down a new path.

I love how you are starting to look at different aspects to use for our system, other than just jail and prison, and just to cage a human for making a few mistakes in his or her life is not just or fair. We need to find other solutions to the problems at hand and fix the broken and needy. Yours truly a former inmate.

Everyone here has been great!! I just want to say thank you for this 2nd chance to get my life back on track. I see there is no reason at all why this program wouldn't work for ANYONE!!! It's out there if you want it!! But you have to want it!! Thanks again guys all of you at the DRC center... hands down, hats off... keep up the great work!!

Finally, participant post-test statements were uniformly positive and indicative of strong satisfaction with DRC programming and personnel. To illustrate:

All the staff at DRC are very helpful and really want to help you and see you succeed.

[Program Staff] are great people and they keep to their word on helping others based on help wanted. It was a great program.

[Program Staff] were especially fair and really respectful to me, and [Staff Member] was also a very amazing guy. I honestly wish the rest of the judicial system had people like these men!!! I am very satisfied with the amount of respect I had... THANK YOU SO MUCH.

DRC was a very good stepping stone. It helped me better myself and my decisions. Overall DRC was a very good place for me to be.

I thought that all the staff was very helpful, respectful, and willing to help me with anything I needed.

Changes in COMPAS Scores

With regard to changes in COMPAS scores, prior to entering the DRC, participants were assessed for recidivism risk by probation officers through the use of the COMPAS tool. When participants complete phase 2 of DRC programming, the COMPAS tool is readministered, and this takes place again when participants complete phase 3 of DRC programming and are discharged. Quantitative analysis of changes in these scores, particularly for the dynamic risk factors, is presented below.

Tables 3 and 4 provide the results of dependent samples t-tests for differences in means between risk factors at Time 1 (prior to enrollment in DRC programming) and Time 2 (when DRC participants transition from phase 2 to phase 3 of programming). In Table 3, each pair of risk factors is presented in the first column, and the “Mean” column presents the average participant scores for General Recidivism Risk and the 17 dynamic risk factors listed in the table. As of March 31, 2019, 62 DRC participants had transitioned from phase 2 to phase 3 and had COMPAS risk factor scores for both Time 1 and Time 2.

As shown in Tables 3 and 4, 12 of 18 (67%) of the mean differences in risk factors were positive, meaning the average scores were lower at Time 2 (suggesting recidivism risk was lowered). One-third of the mean differences were negative, meaning the average scores were higher at Time 2 (suggesting recidivism risk was increased). In Table 4, the findings highlighted in yellow are those with a statistically significant difference in means. In highlighting these findings, we used a p-value of .05 for purposes of identifying significant changes. Of the seven highlighted/significant findings, six were positive (i.e., lower recidivism risk) and one was negative (i.e., higher recidivism risk).

From Time 1 to Time 2, the following risk factors declined significantly: Criminal Opportunity, Leisure and Recreation, Criminal Personality, Criminal Thinking, Anger, and Employment Problems. In contrast, mean scores for Substance Abuse increased significantly. In addition, the General Recidivism Risk score remained stable. In combination with subsequent changes in risk scores between Time 1 and Time 3, and between Time 2 and Time 3, the leadership team discussed the results presented in Table 4. One possibility for the lack of positive change in General Recidivism Risk, as well as the mixed results for the more specific risk factors, is that more time for change was needed for participants in the program. Another possibility is that DRC participants simply became more truthful at Time 2, meaning their responses were more accurate after participating in DRC programming and developing a relationship with DRC staff. Regardless of the explanation, six risk factors did improve significantly from Time 1 to Time 2, setting the stage for further COMPAS score analysis.

Tables 5 and 6 provide the results of t-tests for differences in means between risk factors at Time 1 (prior to enrollment in DRC programming) and Time 3 (when DRC participants transition from phase 3 and graduate from the program). In Table 5, each pair of risk factors is presented in the first column, and the “Mean” column presents the average participant scores for General Recidivism Risk and the 17 dynamic risk factors listed in the table. As of March 31, 2019, 51 DRC participants had transitioned from phase 3 and had COMPAS risk factor scores for Time 1 and Time 3.

As shown in Tables 5 and 6, 14 of the 18 mean differences in risk factor scores were positive, meaning the average scores were lower at Time 3 (suggesting recidivism risk was lowered). Four of the mean differences were negative, indicating the average scores were higher at Time 3 (suggesting recidivism risk increased). In Table 6, the findings highlighted in yellow are those with a significant difference in means. Again, in highlighting these findings, we used a p-value of at or below .05 for purposes of identifying significant changes. Nine of the 11 highlighted findings were positive, indicating lowered recidivism risk. The General Recidivism Risk score declined significantly, along with Criminal Opportunity, Leisure and Recreation, Social Isolation, Criminal Personality, Criminal Thinking, Anger, Cognitive Behavioral, and Financial. Substance Abuse and Socialization Failure both increased significantly, indicating higher risk at Time 3 versus Time 1.

Finally, Tables 7 and 8 provide the results of t-tests for differences in means between risk factors at Time 2 (when DRC participants transition from phase 2 to phase 3) and Time 3 (when DRC participants transition from phase 3 and graduate from the program). Again, as of March 31, 2019, 51 DRC participants had transitioned from phase 3 and had COMPAS risk factor scores for Time 2 and Time 3.

As shown in Tables 7 and 8, 16 of the 18 mean differences in risk factor scores were positive, meaning the average scores were lower at Time 3 (suggesting recidivism risk was lowered). Two of the mean differences were negative, indicating the average scores were higher at Time 3 (suggesting recidivism risk increased). In Table 8, the findings highlighted in yellow are those with a significant difference in means. The General Recidivism Risk score declined significantly, along with Criminal Associates/Peers, Social Isolation, Substance Abuse, Cognitive Behavioral, Financial, and Social Adjustment Problems. Socialization Failure increased significantly, indicating higher risk at Time 3 versus Time 2.

Overall, the changes in risk scores are positive and indicate a beneficial impact from DRC programming, particularly for the 51 participates who progressed to complete phase 3 of the program. A large of majority of risk score changes were in a beneficial direction, and many of these changes were statistically significant. This sets the stage for the next area of statistical analysis, focused on recidivism.

Recidivism Assessment

Finally, to evaluate the impact of DRC programming and practices on the future behavior of participants, a recidivism assessment was conducted. Official arrest data were accessed through The Unified Judicial System of Pennsylvania Web Portal (https://ujsportal.pacourts.us/). This website contains information on all criminal court cases filed in Pennsylvania, beginning with charges filed by police in magisterial district court following the arrest of a defendant.

Initially, recidivism data were collected on DRC participants when they were discharged (either successfully or unsuccessfully) from DRC programming. Then, in order to utilize a quasi-experimental research design (specifically a non-equivalent treatment and comparison group design), rearrest data were collected on an historical group of non-DRC probationers and paroles. In the analyses discussed below, the comparison group includes non-DRC offenders on probation or parole supervision in 2015 and 2016, for which the COMPAS tool was completed prior to supervision. This comparison group excludes such offenders as those transferred for supervision from another jurisdiction, those who were sanctioned under Accelerated Rehabilitative Disposition, and those under short-term DUI supervision.

The current recidivism analysis focuses on rearrest data at 6 and 12 months following initiation of either DRC programming (i.e., the treatment group) or more typical probation/parole supervision (i.e., the comparison group). First, a descriptive summary of the recidivism data for DRC participants is presented below. Then, a comparative assessment of recidivism figures for the DRC participants and non-DRC probationers/parolees is presented. Finally, analysis is presented that compares DRC graduates to non-graduates, along with the probationers/parolees in the comparison group.

As of March 31, 2019, 95 individuals had been discharged (either successfully or unsuccessfully) from DRC programming and had at least 12 months in the follow-up period. Of these DRC participants, slightly more than half (49) graduated from the program, and slightly less than half (46) were dismissed unsuccessfully. Based on the entire group of 95 participants:

• 9 (11.8%) were rearrested within 6 months of entering the DRC
• 17 (17.9%) were rearrested within 12 months of entering the DRC
• Only 3 (3.2%) were rearrested on a felony within 12 months of entering the DRC

Of the 49 graduates of the DRC:

• 3 (6.1%) were rearrested within 6 months of entering the DRC
• 4 (8.2%) were rearrested within 12 months of entering the DRC
• 2 (4.1%) were rearrested on a felony within 12 months of entering the DRC

At face value, these figures represent a rather low level of recidivism, particularly for DRC graduates and while considering the elevated risk levels of most DRC participants. However, use of comparison group data strengthens the conclusions that can be made regarding the impact of DRC programming on recidivism. To utilize more comparable groups of DRC and non-DRC subjects, individuals in both groups with General Recidivism Risk scores of 4 or higher (from initial COMPAS tool administration) were selected for analysis. This ensured that the two groups were not significantly different based on initial General Recidivism Risk, and enabled the recidivism analysis to focus on medium and higher risk individuals (i.e. the DRC target population). Tables 9 through 11 provide initial results of this comparative analysis.

Based on the figures presented in Tables 9 through 11, DRC participants exhibited slightly lower general recidivism at 6 months (10.3% versus 12.7%) and slightly lower felony recidivism at 12 months (3.8% versus 6.3%), but higher general recidivism at 12 months (19.2% versus 16.5%). However, none of these differences were statistically significant, meaning the recidivism of all DRC participants versus comparable Non-DRC probationers and parolees was similar.

The next set of statistical findings, presented in Tables 12 through 14, is based on comparing DRC graduates with non-DRC individuals. These analyses generally indicate lower recidivism on the part of DRC graduates. More specifically, rearrest rates were lower for DRC graduates than for non-DRC individuals at 6 months (4.9% versus 12.7%) and 12 months (7.3% versus 16.5%). The felony rearrest rate at 12 months was similar for the two groups, but slightly lower for DRC graduates (4.9% versus 6.3%). Although the 6-month and 12-month rearrest rates were lower for DRC graduates, the differences, while noticeable, were not statistically significant. This is likely due to the low group sizes employed in this analysis, which makes findings of statistical significance difficult to achieve.

The next set of findings, shown in Tables 15 through 17, is based on comparing individuals who graduated from the DRC with participants who were dismissed unsuccessfully from the DRC. All 95 DRC participants were included in this analysis. The findings again indicate lower general recidivism on the part of DRC graduates, at both 6 months (6.1% versus 13.0%) and 12 months (8.2% versus 28.3%). The felony rearrest rate at 12 months was slightly higher for DRC graduates (4.1% versus 2.2%), but only 3 total DRC participants were rearrested for a felony at 12 months. In addition, the difference in general recidivism at 12 months was statistically significant (chi-square = 6.523, p < .05), meaning DRC graduates were significantly less likely to be rearrested at 12 months, as compared to DRC participants who were dismissed unsuccessfully.

Finally, logistic regression analysis was used to assess further the impact of DRC programming on recidivism. These models contained three independent variables: “Graduated from DRC” and “Dismissed from DRC” (with Non-DRC probationers/parolees used as the reference group), along with the “General Recidivism Risk” score for each individual. Essentially, these models assess the likelihood of recidivism for DRC graduates versus the comparison group and for DRC dismissals versus the comparison group, while controlling for general recidivism risk.

Table 18 shows the logistic regression results with rearrest at 6 months used as the dependent variable. The results indicate DRC graduates were less likely to recidivate, as compared to non-DRC probationers and parolees, while controlling for general recidivism risk. Although the simple odds of rearrest were about 65% lower for DRC graduates, this effect on recidivism did not reach statistical significance, again likely due to the rather small group sizes employed in the model. In addition, there was relatively little difference in recidivism for participants dismissed from the DRC versus individuals in the comparison group, and general recidivism risk was not significantly associated with the likelihood of rearrest at 6 months.

Table 18: Logistic Regression for Rearrest at 6 Months

Table 19 provides the logistic regression results with rearrest at 12 months used as the dependent variable. The results again indicate DRC graduates were less likely to recidivate, as compared to non-DRC probationers and parolees, while controlling for general recidivism risk. In this case, the simple odds of rearrest were about 60% lower for DRC graduates, but again this effect on recidivism was not statistically significant. In addition, the likelihood of recidivism was greater (but not statistically significant) for participants dismissed from the DRC versus individuals in the comparison group. General recidivism risk also had an insignificant association with the likelihood of arrest at 12 months.

Table 19: Logistic Regression for Rearrest at 12 Months

Table 20 presents the logistic regression results with felony rearrest at 12 months used as the dependent variable. As in the previous two models, the results suggest DRC graduates were less likely to recidivate, as compared to non-DRC probationers and parolees, while controlling for general recidivism risk. Here, the simple odds of felony rearrest were about 20% lower for DRC graduates, but this effect on recidivism was not statistically significant. In addition, the likelihood of felony rearrest was lower (but not statistically significant) for participants dismissed from the DRC versus individuals in the comparison group, and general recidivism risk had an insignificant association with the likelihood of felony arrest at 12 months.

Table 20: Logistic Regression for Felony Rearrest at 12 Months

Summary and Conclusions

Overall, prior evaluation research indicated the Somerset County DRC was implemented as planned during the 3 years of federal funding, and the results of implementation monitoring and outcome assessment were favorable. Key findings from the current analysis can be summarized as follows:

1. Program participants received a wide variety of services, and participant survey data were highly favorable regarding perceptions of the programming received. In addition, pre-test/post-test results indicated improved perceptions of the criminal justice system because of DRC participation.
2. Analysis of COMPAS data revealed a number of statistically significant improvements in risk scores over time, particularly from Time 1 to Time 3 and from Time 2 to Time 3. These positive changes pertain primarily to DRC participants who were successful in the program and eventually graduated. For these individuals, the findings suggest the DRC was successful in lowering recidivism risk across a number of dynamic risk factors.
3. Analysis of recidivism data and use of a comparison group suggested DRC graduates exhibited lower levels of official recidivism (measured by rearrest) at 6 and 12 months. Felony recidivism at 12 months was very low across all DRC participants. Obtaining statistically significant findings was challenging, due to the rather small groups of DRC graduates and non-graduates examined. When combined with the results of the COMPAS data analysis, the findings indicate program graduates demonstrated lowered recidivism risk and a lowered likelihood of rearrest, but program non-graduates did not exhibit this same pattern.

References

Myers, D. L., Lee, D. R., Giever, D. M. (2018a). Second Chance Act in action: A case study of evidence-based approaches and a researcher-practitioner partnership. EBP Quarterly, 3(1), 1-4.

Myers, D. L., Lee, D. L., & Giever, D. M. (2018b). Bureau of Justice Assistance, Innovations in Supervision Initiative; Somerset County, PA Day Reporting Center: Final Report on Implementation and Evaluation. Indiana, PA: Justice Performance Consultants, LLC.

Photo by ray sangga kusuma on Unsplash