Effect stability: (2) Simple mediation designs

In my last post, I described how a significant estimate need not be close to its population value, and how, using a clever method developed by Schönbrodt and Perugini (2013), one can estimate the sample size required to achieve stability for an estimator through simulation.

Schönbrodt's and Perugini's method defines a point of stability (POS), a sample size beyond which one is reasonably confident that an estimate is within a specified range (labeled the corridor of stability, or COS) of its population value.  For more details on how the point of stability is estimated, you can read either my previous post or Schönbrodt's and Perugini's paper.

By adapting Schönbrodt's and Perugini's freely available source code, I found that, in two-group, three-group, and interaction designs, statistical stability generally requires sample sizes around 150-250.  In this post, I will apply this same method to simple mediation designs.

Effect stability: (1) Two-group, three-group, and interaction designs

When planning the sample size to estimate a population parameter, most psychology researchers choose the size that could allow an inference that the parameter is non-zero -- in other words, researchers attempt to maximize statistical significance.  However, both practical and scientific interest often centers around whether the estimate is good or stable -- that is, close to its population parameter.

These two criteria, significance and stability, are not the same.  Indeed, with a sample size of 20, a correlation of $r$=.58, which has a $p$-value of .007, could plausibly range between .18 and .81.