Challenges of and Tips for Conducting Randomized Trials in a Criminal Court Setting
Posted on May 31, 2011
Michael Rempel (bio) of the Center for Court Innovation discusses the challenges of and offers tips for conducting randomized clinical trials in criminal court settings.
Next, we faced the problem of upholding due process rights for the offenders without compromising the integrity of the study. Any study protocol involving the criminal justice system has to take into account defendants' constitutional rights, as well as the responsibility judges have to uphold court mandates. We learned from our experiences in the Bronx that defense attorneys would be unlikely to accept any study protocol that might result in a defendant receiving a more onerous sentence than he would have received were he not involved in the study. We also learned that judges felt strongly it was their legal responsibility to impose the exact terms of sentence -- thus, randomization could not take place after the sentencing process. Eventually, we settled upon in-court allocution and assignment, in which the sentencing judge would fully explain to the defendant the ramifications of pleading guilty, after which randomization would take place at a bench conference. The judge would then inform the defendant of the selected condition as part of the sentencing process. This method worked well, and we implemented a similar randomization process in Rochester.
In addition, we were not sufficiently skeptical about court volume estimates at either the Bronx or Rochester sites. Estimates regarding case volume at both were significantly higher than actual cases, and, in the case of the Rochester site, we had to extend the study there by a full year in order to meet our targeted recruitment goal.
Finally, we learned that stakeholder collaboration and buy-in was essential. In Rochester especially, we encountered a tight-knit stakeholder community that was unfamiliar with our work. Although we had agreed to conduct the trial with the judiciary, we could not implement several key components (mainly the victim and defendant interviews) without the support of the local prosecutor and victim advocacy community, neither of which we had contacted in the initial study planning phase. In our case, we were eventually able to work together with all of the involved stakeholders, although this outcome cannot always be guaranteed. In general, it's best to let every major stakeholder know as early as possible that a randomized trial might be taking place and to solicit input from those stakeholders.
Second, having an on-site research presence can encourage fidelity to the implementation plan and provide immediate resolutions to challenges as they arise. It also conveys to stakeholders a level of commitment to the process that, ultimately, contributes to long-term buy-in and cooperation.
Third, the study itself may precipitate unintended effects. It is important to watch for such consequences and address any attendant changes in practice. For example, we wondered if prosecutors would be less likely to offer a plea bargain, because the defendants would have only a 50 percent chance of being assigned to a batterer program (the Bronx) or receiving judicial monitoring (Rochester). In the Bronx, we collected pre-study data to assess major differences between the characteristics of the pre-study sample of defendants versus the participant sample. We found no significant differences. In Rochester, we monitored the overall distribution of sentences in the domestic violence courtroom to catch any changes happening as a direct result of the study.
Finally, close monitoring of data collection is critical. For example, a problem in the Bronx site only came to light during the final data analysis, at which point we discovered that the judicial hearing officer had not been consistently applying the graduated monitoring protocols (noncompliant participants were supposed to be assigned a more stringent reporting schedule; in most cases, this did not take place). Although we could not fix this problem, we could at least take it into account and make appropriate adjustments when analyzing our data.
In Rochester, as noted previously, the challenges related to stakeholder buy-in were even more severe. In general, having learned from the Bronx experience, we involved more stakeholders in Rochester prior to securing funding. For instance, we reached out to the local judiciary in Rochester from the outset rather than planning the study initially with only a statewide administrative judge. Yet, we still encountered resentment from the local prosecutor and victim advocacy community in Rochester, who believed that they should have been able to provide input at the earliest possible time. In general, it is difficult to achieve the right balance on this question, because it is obviously inefficient to hold multiple meetings with a wide array of individuals when funding is not yet in place. We can only recommend extreme care in thinking through whom to notify and what level of detail to provide at the proposal stage.
Once funding is secured, it then becomes important to initiate broader outreach sooner rather than later. In a third and more recent trial, as we did in Rochester, we obtained the buy-in of both the statewide and local judiciary prior to obtaining funding; but unlike Rochester, we sought in-depth meetings with prosecutors, the defense bar, and line staff as soon as possible after funding was secured, so that their input could be obtained at a time when we could feasibly incorporate it into the final study implementation plan.
Adapted from the 2010 article "Lessons Learned from the Implementation of Two Randomized Trials in a Criminal Court Setting," by Melissa Labriola, Michael Rempel, and Amanda Cissner.
Please note that the feedback is viewed only by 4researchers staff and is not intended for communication with individual contributors.
Use the form below to submit feedback about this article. If you would like a response, please be sure to include your e-mail address.