Post-Hoc Probing of Mediator and Moderator Effects
Posted on October 4, 2006
Grayson N. Holmbeck (bio) explains why post-hoc tests are important, and provides examples.
Q: My advisor has suggested that I conduct post hoc probing, but if I've already found significance for mediation and moderation effects, why do I need more analyses?
A: Conducting post hoc analyses of mediation and moderation is important because it prevents the investigator from drawing false positive or false negative conclusions about the relationships among the variables. If you've done a multiple regression and found a moderation effect, for example, you know that the relationship between the predictor variable and the outcome varies across levels (or groups) of the moderator, but you don't know exactly when ths relationship is significant (e.g., when there are low levels of the moderator or high levels of the moderator). For mediation tests, you need to determine if the association between predictor and outcome is significantly reduced when the mediator is introduced into the model. It is important to keep in mind that the "drop to nonsignificance" criterion is flawed and may result in incorrect conclusions about the data.
Testing Moderational Effects
Example: Two-way interaction between a dichotomous variable and a continuous variable (see Holmbeck, 2002, for more information).
To probe a significant interaction, you need to compute two new conditional moderator variables, then run two regressions using the new variables:Step 1: Create two conditional moderator variables. For the first, code one of the moderator groups 0, and the other group 1. For the second conditional variable, code the first group as -1 and the second group as 0 (see Holmbeck, 2002, for an explanation). Step 2: Compute the 2 interactions using the 2 new conditional variables from Step 1.Step 3: Run 2 post hoc regressions by simultaneously entering the main effect for A (predictor variable ñ see diagram), one of the conditional group variables (Bcond), and the interaction of the two (A and Bcond).Step 4: Run a post hoc regression using the other conditional group variable (the one that was not used in the first regression).The results will produce significance tests (t) for each slope.
Q: What happens if you don't do a post hoc for moderation effects?
A: When you test the significance of interaction terms, you learn that the relationship between A and C is a moderated by B (see diagram), but you cannot know how B affects the relationship between A and C. Visual inspection of the regression line slopes is not sufficient and may result in false positives. By conducting a post hoc analysis, you can see how each level or group of the moderator effects the association between predictor and outcome.
Testing Mediational Effects
Once you've found a significant mediational effect, you should conduct a post hoc analysis to test the significance of the indirect effect (see Holmbeck, 2002 for details).Step 1: From the computer output, get the unstandardized path coefficients from the model, and their standard errors.Step 2: Calculate the standard error of the indirect effect (see Holmbeck, 2002).Step 3: Divide the unstandardized beta for the indirect effect by the standard error of the indirect effect to get a z-score.
Q: What mistakes can you make in interpretation if you do not conduct post hocs for mediational effects?
A: You can draw both false positive and false negative conclusions. False negative conclusions may be due to limited power or poor reliability of measures (Type II errors), of course. But both false negatives and false positives can be a result of the "drop to nonsignificance" criterion. This strategy assumes that there is mediation when the A-C relationship drops from significance to nonsignificance with the addition of the mediator. This is a flawed strategy, however, because it is possible that no mediation is present even when the relationship changes from significant to nonsignificant. Moreover, it is also possible that mediation has occurred even when there is no drop to non-significance (see Holmbeck, 2002 for further discussion).
Holmbeck, G. N., (2002). Post-hoc probing of significant moderational and mediational effects in studies of pediatric populations. Journal of Pediatric Psychology, Special Issue on Methodology and Design 27(1), 87-96.
Please note that the feedback is viewed only by 4researchers staff and is not intended for communication with individual contributors.
Use the form below to submit feedback about this article. If you would like a response, please be sure to include your e-mail address.