Output from summary(lmerMod)
Compare ANOVAs and LMER for an individual simulation
Aggregated by Items
Aggregated by Subject
The p-value for the main effect in LMER will be near-identical to the by-stimuli ANOVA if the random slope for the main effect is set to zero (i.e., where between-subject variation in the effect of stimulus type is 0).
Compare False Positive Rate and Power
This function will run the number of simulations with the parameters you've set and report the proportion of runs that gave a significant effect of condition (given the alpha you set). It will also report the false positive rate for the same simulations with the main effect of condition set to 0. If you set the main effect of condition to 0, then power will be equal to the false positive rate.
It is not an error that the false positive rate for the by-subjects ANOVA is very high. With this type of within-subjects, between-items design, you can get very high false positive rates if items have some variation in their mean DV (i.e., where faces tend to vary in expressiveness). For this type of design (no between-subject factors), the by-items ANOVA will have a closer-to-nominal false positive rate, but will have a more inflated false positive rate for designs with between-subject factors where subjects have some random variation in their mean repsonses.
If you set the Item Intercept SD to 0, you will see that the by-subjects ANOVA has a false positive rate closer to the nominal alpha (defaults to 0.05). However, this models a very unrealistic situation where the variation in expressiveness of faces is 0.
Simulated Effect Size Distribution
This app is a companion to Understanding mixed effects models through data simulation by Lisa M. DeBruine and Dale J. Barr.Preprint on PsyArXiv | Example R code | Code for this app
Set the parameters in the sidebar menu for a crossed design where raters (subjects) classify the emotional expression of faces (items) as fast as possible. Faces are either from an ingroup or an outgroup category (X_i). The hypothesis is that people will classify the emotions of ingroup faces more quickly than outgroup faces. Learn more about what these parameters mean below.
Click on Simulating LMER in the sidebar menu to view the output of the lmer summary and see how the parameters you specified affect the output. Click on Compare ANOVA & LMER to compare the results of the mixed effect model with by-subject and by-item aggregated ANOVA. Click on Power & False Positives to run a power analysis using your parameters and compare power and false positive rate between lmer and ANOVA.