Zaslow: There's a different team in each of the sites, and there the challenge is coming up with a common protocol and training people so that they will have inter-rater reliability on the measures in an acceptable way and that's uniform enough across the sites and developing protocols for checking on reliability that's uniform across the sites. Creating a mechanism for pooling the data across the sites and entering it so there is a single data management system.

The data have to be entered in the same way. You have to figure out a way to transmit the data securely to protect the privacy of participants, so there's this uniformity element, and that's counterbalanced with the richness of learning about the uniqueness in each of the sites. We need to know if an on-site consultation model works differently or the same in sites that have a lot of regulation on the ground for family childcare and center care and virtually none for family childcare, where the policy contexts are very different. It's really wonderful to be carrying out a study like this in the multiple sites because you don't mistakenly think that the pattern you see in one site is the same in all the others, and you learn a lot from the other teams as you talk about how this is playing out on the ground.

Helsel: We have different contractors, so you're trying to align your interests and your goals across each of the contractors, but you're dealing with very different regions of the country. Obviously, one staff person can't go out and do all the interviews, the data collection, and just getting the sites on board for the study. You have to get consistency in your protocols in getting people on board, but you also have to be very sensitive to the regional differences of the different, in this case the different Head Start programs all the way from the executive staff down to the teachers and the parents in the programs. It's kind of a quick study type work when you're working in sites just figuring out the differences between each of the parties involved.

Zaslow: We talked before about collaborations that were highly uniform where the same protocol was implemented, but there are also looser confederations that are brought together where each site is doing something different, but the intent is to make sure that they communicate across them. An example of that is a Kellogg-funded project we're involved in that involves school readiness, Supporting Partnerships for Ready Kids or SPARK. That's taking place in eight states that are doing very different things, and the intent there was to come up with a common logic model, but the states are not required to follow the same evaluation protocol except for a few small elements that they are required to do, but that's a looser confederation where the intent is to help the sites learn from each other, which happens all the time.

« Back to Article