Connecting Research to Policy
Posted on March 14, 2008
Greg J. Duncan (bio) and Katherine Magnuson (bio) describe cost-benefit analysis and how to use it when approaching policymakers.
|
Q: Why is cost-benefit analysis important in intervention research?
A: A cost-benefit approach asks whether the value of the outcomes generated by an intervention program is sufficient to justify the costs associated with it. This is a key question for practitioners and policymakers. Researchers often report effect sizes, which are certainly useful. But effect sizes alone do not convey the time and money resources required to generate the effect. Cost-benefit analysis provides policymakers with crucial information about a program's social benefits relative to the cost of bringing them about.
Q: Whose costs and benefits are included?
A: A cost-benefit analysis compares the costs of a program to the benefits generated by that program. Although policymakers often focus just on taxpayers' costs and benefits, economic logic suggests that policymakers should take a broader look at both taxpayers' and participants' costs and benefits. Total social costs, regardless of who pays them, and total social benefits, regardless of who receives them, are most telling about how a society should invest its resources.
Q: Can you give me an example?
A: The Perry Preschool Program provides a good example of a cost-benefit analysis. The Perry Program was developed by David Weikart and his colleagues in 1962 as part of federal initiatives by Head Start and the Bureau for the Education of the Handicapped. Founded with the Ypsilanti Public Schools in Ypsilanti, Michigan, the intervention provided 1 or 2 years of enriched early care education for 58 low-income African American children. For evaluation purposes, the researchers tracked the progress of preschoolers attending the program and a randomly-assigned group of children who did not attend the program.
The costs of the program are reasonably simple to estimate: in Perry’s case, when expressed in today's dollars, the costs of staff and facilities were about $15,166 per child.
The financial benefits of the program included both money saved by taxpayers as well as higher earnings of participants. Perry children were less likely to need special education services than the control group (specifically 1.1 years versus 2.8 years of special education). This and other schooling differences amount to a taxpayer savings of about $8,434 in staff and facilities costs. Perry participants were more likely to complete high school, attend college, and, as a result, earned an estimated $50,000 per child than their control group counterparts. Finally, Perry students were less likely to become involved in criminal activity, particularly violent crime. There are huge costs to victims and the criminal justice system associated with violent crimes, generating a per-child savings that exceeded $150,000.
The chart below shows the total costs and benefits for participants and taxpayers, which add up to the total social cost and benefit figures shown in the final column. Perry's benefits far exceeded its costs; many profitable intervention programs may have much lower benefit-cost ratios.
| For Participants | For Public | Total |
| Costs | $0 | $15,166 | $15,166 |
| Benefits | $38,924 | $169,510 | $208,434 |
Ratio Costs-Benefits | | | 13.74:1 |
Q: How specific do my monetary estimates need to be for cost benefit analyses?
A: Specific calculations are best, but even order-of-magnitude estimates are helpful. Knowing whether per-participant costs total $500, $5,000, or $50,000 can help you or your audience understand the comparative size of program costs.
Q: How do I produce a rough estimate of costs or benefits?
A: Direct service program costs consist mostly of staff salary expenses, so tracking the number of hours of time spent with each child or family or group is an excellent starting point for estimating costs. Thorough analysis of kinds of costs is even more helpful, e.g. one-on-one versus group interaction, specific tools needed to perform tests. Again, if exact calculations are not possible, order-of-magnitude monetary estimates are still useful.
Precise estimation of benefits can be difficult, but understanding and using estimates available in the field and from other studies is recommended. National per pupil costs of education, the costs of prosecuting minor and major crimes, the impacts of test score gains on future income, the effects of substance use on lifetime earnings and health costs: All these kinds of already-available data can serve as benchmarks for approximate figures.
Q: How does a cost benefit analysis help me make a case to policymakers?
A: Solid cost-benefit analysis allows policymakers to understand a program’s "rate of return" on its costs. A program that produces benefits with a small dollar value might still have an excellent benefit-cost ratio if it does not cost much. If programs with high benefits have even great costs, then they may not be worthy social investments. A brief job search program, for example, might produce less gain in earnings than weeks of vocational training, but the reduced costs of the "less impressive" program need to be taken into account when choosing which programs to fund.
Further, explaining benefits in dollar form can produce a clear link for policymakers about the multifaceted and long-term benefits associated with programs. Grade repetition or special education costs, for example, are easy to measure, and if a program can reduce those, the benefits are clear both as an intervention and as fiscal and political strategy.
Q: What particular issues might come up when speaking to policymakers?
A: Focus on describing important findings in policy-relevant units of measurement. Some indexes or scales, such as the child care quality measures developed by the National Institute of Child Health and Human Development, are difficult to interpret on their own. A comparison of children in the top and bottom quartiles of the quality measure amounts to a 2.5 standard deviation quality "change" far greater than changes that might actually be produced by a practical policy. The best scaling reflects a practical, financially-feasible goal. In terms of the NICHD example, how much would it cost to bring about a 1 or even .5 standard deviation quality change?
Also try to consider "spillover" benefits for individuals who are not themselves program participants. When the program affects people other than participants this is sometimes viewed as "contamination" but external benefits are potentially important for policymakers. School-based programs that reduce behavior problems, for example, might also make classroom time more productive not only for participants, but for the teachers and other students as well.
Cost-benefit calculations are also useful for making clear to policymakers how an intervention should be designed. For example, a program created to reduce acts of violence among high schoolers might not generate benefits that exceed costs if implemented across all students, but highly beneficial if targeted to students with particular discipline problems during elementary school. Cost-benefit analysis can prove to policymakers why a program might need a very specific form of implementation.
Q: Are there other reasons to use cost-benefit analysis?
A: This analysis can facilitate the judicious expansion and replication of the program. If the data indicate that a program is not effective on a certain scale or without certain cultural requirements, then cost-benefit analysis can prevent poor implementation. In a worst-case scenario, cost-benefit analysis might point out that a program or intervention simply isn't fiscally sound and needs restructuring. Although frustrating for a program developer, the recognition of a program's limits can save time, money, and energy better used elsewhere.
Q: What are the limits of cost-benefit analysis?
A: Although all successful programs provide at least some financially-measurable benefits, even the most experienced economists would have problems assigning a dollar value to some types of benefits. For example, it is hard to estimate the dollar value of increasing tolerance or civic participation. One way of handling this kind of result is to compare the cost-effectiveness across programs. In a cost-effective framework, you scale outcomes in their natural units rather than dollars and then see which program can produce a given increase in that outcome, such as civic participation, at the lowest cost.
Based on:
Duncan, G. J., & Magnuson, K. (2007). Penny wise and effect size foolish. Child Development Perspectives, 1, 46-51.
See also:
Schweinhart, L. J., Montie, J., Xiang, Z., Barnett, W. S., Belfield, C. R., & Nores, M. (2005). Lifetime effects: The High/Scope Perry Preschool study through age 40. Monographs of the High/Scope Educational Research Foundation, 14.
Barnett, W. S. (1996). Lives in the balance: Age-27 benefit-cost analysis of the High/Scope Perry Preschool Program. Monographs of the High/Scope Educational Research Foundation, 11.
Schweinhart, L. J., Barnes, H. V., & Weikart, D. P. (1993). Significant benefits: The High/Scope Perry Preschool study through age 27. Monographs of the High/Scope Educational Research Foundation, 10.
More About "Policymakers" Related Topics | |