A: A cost-effectiveness study compares established programs, usually in a controlled setting, to determine which produces the better results given the expense. A cost assessment, which is what we were conducting in this study, is a descriptive analysis of program costs and does not analyze any data on outcomes associated with programs. In our study, we attempted to determine all costs that accrued during the implementation of a school-based mental health program. The study was conducted in a real world setting, outside the context of a research study. We collected data on direct and indirect costs, along with what are called "societal" costs, like shared space and use of volunteer staff. Although a cost assessment cannot generate estimates of cost per unit of effect, this kind of study still can provide useful information for program planning. In addition, information on costs can be used as preliminary data for a future cost-effectiveness evaluation.

A: This was a screening and treatment program in a low-income, mostly Latino middle school in NYC. A health center was already set up in the school to provide medical, dental, and psychological services. The psychological services were provided by social workers and social work interns.

Originally, the system of referral for psychological services was informal ñ children were referred by teachers, assistant principals, parents, other classmates, or by the students themselves. Because there was not formal system set up for referrals, many kids who needed help were not being served, especially the 'internalizing' kids — the quiet ones who didn't complain and didn't cause disruption.

Thus, a new screening program was put in place to screen all of the 6th ñ 8th graders for anxiety, depression, and/or drug use, and then to refer to treatment all those who met specific criteria. The staff for the new program included a program director and case manager (both full-time), interviewers, and 3 psychiatrists (one on-site and 2 coming in as needed).

A: Using economic cost analysis methods, we calculated the costs from a societal perspective, not just from the school's perspective — so the analysis included the obvious things (salaries, fringe benefits, supplies) but also things that the school wouldn't necessarily consider to be costs, such as depreciation of equipment, indirect costs such as billing, payroll, custodial services, and the use of the physical space.

We wanted to know what it cost to add the screening program to the school, and how much more the new treatment program cost than the original program. In Year 1, we got baseline data, with the original treatment program in place and screening in start-up. We then collected data for Year 2 and Year 3, when both the screening and the treatment programs were in place.

For the analysis, we deducted the Year 1 costs (baseline) from costs of Years 2 and 3. That way, we could tell how much the screening program alone cost.

A: We obtained:
Labor costs: salaries, stipends, fringe benefits
Operating costs: supplies and materials, minor equipment and furniture, contracted services, postage, telephone, staff training, local travel
Indirect costs: depreciation - capital equipment, custodial services
Societal costs: school space, opportunity cost of capital equipment, student volunteers

A: Gathering these data in a real-world setting is not easy — most programs are not designed for cost analysis research — the indirect and societal costs are not on record anywhere, so you have to estimate them.

The data we had available to us came from accounting records maintained by the university hospital and community agency that funded and administered the programs. The majority of data on standard program costs, such as costs associated with staff salaries, fringe benefits, and supplies, could be obtained from these records. In some cases, however, we only had access to budgets and not expenditures. In those cases, we checked payroll records and interviewed program staff to ensure that budgeted funds were spent as planned.

Data were not available on societal costs — costs of resources used by the programs that were not paid for explicitly. We estimated those costs using interviews and publicly available data. For example, to estimate the cost of the school space used by the programs, we had to measure the space and then calculate the rental charges associated with that space. We estimated the rental rate per square foot for the space based on estimates from local realtors and the facilities department of the university hospital that was one of the sponsors of the programs. Another example is volunteer labor. The program did not keep track of or record in-kind expenses associated with volunteers (other than social work student interns). We interviewed the program director to find out who volunteered at the program, the education level of these individuals, and how many hours the volunteers worked. We then estimated what the school district would have paid these volunteers, based on their education level and entry level salary information provided from the NYC Board of Education. Many other examples are provided in the paper.

A: Cost analysis can provide very useful information when you are studying implementation in the real world. For example, our findings suggested that it was worth the cost when there was a high number of kids being screened (i.e. a large population school) or where there was a high incidence of anxiety, depression, and/or drug use; for smaller schools, the program might need to be modified by setting up a central screening/treatment program that serves several schools. This is important to know when you are designing a school-based program.

Cost analysis in the real-world setting is challenging but it is feasible, particularly if you plan to do it ahead of time. Keep good financial records; keep track of all resources used, even if they don't 'cost' anything outright (if they are donated or shared). You can use standard accounting procedures, or set up a protocol for keeping track of direct, indirect, and societal costs. Ideally, it makes sense to build a school-based infrastructure to prospectively track the costs of mental health programs. This effort pays off in the long-run, since the data generated by such a system can inform decision making.