A Commitment to Measurement Posted on March 14, 2006 J. Steven Reznick (bio) warns against using sloppy measurement development techniques, and discusses developing a measure of behavioral inhibition with Jerome Kagan.

The advice that I would give researchers and that I try to instill in my students in graduate courses and in students that work with me, is the importance of a commitment to measurement.

I really believe that if you look at the journals, what we find is that we are under a pressure to publish. We all know that. That's the coin of the realm. But just because you got it published doesn't mean it's good. Indeed, sometimes spending time working on measurement, development of measurement, is not rewarded at all. It's very hard to get grants that are methodologically oriented. It's not so easy to publish work that is methodologically oriented.

But the bottom line is if you aren't measuring what it is that you're trying to measure, the rest of it is a waste of time. For my students, for the people I have fate control over, I try to hold their feet to the fire and my own feet in saying, "I would like to be talking about theory and process of working memory development, but if I can't measure working memory in infants or toddlers, I can't do that. I'm going to have to keep going back and back and back until the measurement is successful.

I think that mantra would make a better field, would allow us to build a psychology, and I think the fact that we haven't accrued a field of psychology where things are built one on the other is because of this Tower of Babel that we have of claiming to be measuring a construct but each person doing it in an idiosyncratic way and not even necessarily measuring what they think they're measuring.

I'll give an example of the difficulty of measurement. I spent a number of years working with Jerry Kagan at Harvard, and our lab was focused on this construct behavioral inhibition, how children respond in a state of uncertainty. We would, in ultimately four longitudinal cohorts that we followed, and there would be an assessment, and we would bring together all the graduate students and staff. We've got the three-year visit coming up, and we would outline every procedure we could think of that would be affected by behavioral inhibition. "OK, well, let's try this. We'll put them in a room where there's risky things. Let's see what they do. We'll ask them while they're interviewed with an examiner, and we'll see how much they talk." We were all focused on measuring this construct at each age, and then we said we'll go to schools and we'll do school visits. We staged these birthday parties, but again it's all focused on that. And the conclusions that we drew, and I think that Jerry's books and articles have shown that that turned out to be a pretty successful endeavor

But then I'll be reading a journal and here is this study, and it's on temperament, and there's a variable "approach/withdrawal", and it's based on the parent's report on nine questions in the broader context of a temperament interview. And I have to think, "No. That is not the same thing. It is not that easy." Then you see someone has a structural equation model, and there's a box that says "temperament". No. That is not going to get us anywhere.

« Back to Article