This is a toolbox for running and plotting bayesian mixed models. Here’s how to get the latest version of the toolbox. Note that there are hefty packages that will be needed for this to work, like rstanarm, so installation might take a while.
devtools::install_github("dcr-eyethink/eyethinkbayes")
library(eyethinkbayes)
#> Loading required package: eyethinkdata
#> Loading required package: ggplot2
#> Loading required package: data.tableUsually, I use the wrapper command bayes_mm() to call a series of functions that run, analyse, report and plot a model in all one step. In this vignette I’m going to first show you each of those functions in turn to break down what they do. Then we’ll return to the wrapper function that can do it in one step.
We’re going to work on some behavioural data from an experiment in which people watched TV shows and adverts. We are interested in whether they prefer and remember ads that are ‘addressable’. This means that the ads are related to their personal interests in some way. First let’s read in the trial data
| pid | dim | adcon | tv | rating | age | agecat | dprime |
|---|---|---|---|---|---|---|---|
| 6700131 | car | addressable | choice | 1.00 | 27 | 18_34 | 0.9674216 |
| 6700131 | cruise | addressable | choice | 3.25 | 27 | 18_34 | 0.0000000 |
| 6700131 | dog | non-address | choice | 4.75 | 27 | 18_34 | 1.2496377 |
| 6700131 | games | non-address | choice | 0.25 | 27 | 18_34 | 0.2822161 |
| 6700745 | car | non-address | choice | 0.00 | 43 | 35.54 | 0.0000000 |
| 6700745 | cruise | non-address | choice | -3.00 | 43 | 35.54 | 1.0037630 |
Participants of difference ages saw four ads in for four different dimensions (dim) that could be addressed to the interests or not (adcon). They watched a TV show that they chose or not (tv). Here we have summary data for each dimension: how much they liked the adverts (rating), and their memory for them when shown stills later (dprime).
Running a model
First we are going to run a bayesian mixed model on their rating data, to see if they liked the addressable ads more. This command will create a formula for the model using the conditions you specify as fixed effects, and the groups as random effects. Rstanarm is used to run the model. This may take some minutes to run. There is a parameter here, explore_browser, which if set to TRUE will launch a browser window with an interactive interface that you can use to explore plots and parameters of your model, check its convergence and so on.
memory_bmm <- bayes_mm_run(data = data,dvname="dprime",conditions = c("adcon","age","tv"),groups = c("pid","dim"),
explore_browser = F)
#> [1] "running model"
#> Warning: There were 1 divergent transitions after warmup. See
#> https://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
#> to find out why this is a problem and how to eliminate them.
#> Warning: Examine the pairs() plot to diagnose sampling problemsNow that we have run the model, we can pass the output to different functions for the plotting and reporting.
Main effects
First, we can look at the main effects. The function bayes_mm_mainFX() will give you means tables, MPEs and contrast for all the main effects you ask for. This will be saved to a text file. The plots show you the observed data (violin) and the distribution of the estimate of the mean (smaller violins). If we don’t specify the mainfx arg it will give us all main effects, but here I’m looking at the effect of adcon.
mfx <- bayes_mm_mainFX(bmm = memory_bmm,mainfxs = "adcon")
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Main effects means: adcon
#> Estimated Marginal Means
#>
#> adcon | Mean | 95% CI | pd
#> ----------------------------------------
#> addressable | 0.77 | [0.58, 0.97] | 100%
#> non_address | 0.60 | [0.41, 0.80] | 100%
#>
#> Marginal means estimated at adcon
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Main effects contrasts: adcon
#> Marginal Contrasts Analysis
#>
#> Level1 | Level2 | Difference | 95% CI | pd | % in ROPE
#> ------------------------------------------------------------------------
#> addressable | non_address | 0.17 | [0.09, 0.25] | 100% | 2.55%
#>
#> Marginal contrasts estimated at adcon
#> NOTE: Results may be misleading due to involvement in interactions
mfx$plot_mainfx
Here, the MPE is given inbetween the two plots, and is the probability that there is non zero difference between those two conditions. We can also represent the difference between conditions using sparkline. It shows, beneath the means plots, the distribution of the estimate of the condition difference. 95% of the distribution is shaded in grey, and zero is shown by a vertical black line. So if the black line is outside the grey shaded area, there is a greater than 95% chance that there is a non zero difference between the means. In the plot below I’ve turned on the sparkline, and turned off the mpes and the predicted distribution of the means. I can also change the plotting here by passing parameters to pirateye(). So here I’ve drawn bars, and they’ve been plotted on a composite plot. These plots are also saved to files, and the w and h parameters control the height and width of the saved pdfs.
bayes_mm_mainFX(bmm = memory_bmm,mainfxs = c("adcon","tv"),bars=T,violin=F,contrast_sparkline = T,show_mpe = F,show_preddist = F)
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Main effects means: adcon
#> Estimated Marginal Means
#>
#> adcon | Mean | 95% CI | pd
#> ----------------------------------------
#> addressable | 0.77 | [0.58, 0.97] | 100%
#> non_address | 0.60 | [0.41, 0.80] | 100%
#>
#> Marginal means estimated at adcon
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Main effects contrasts: adcon
#> Marginal Contrasts Analysis
#>
#> Level1 | Level2 | Difference | 95% CI | pd | % in ROPE
#> ------------------------------------------------------------------------
#> addressable | non_address | 0.17 | [0.09, 0.25] | 100% | 2.55%
#>
#> Marginal contrasts estimated at adcon
#> NOTE: Results may be misleading due to involvement in interactions
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Main effects means: tv
#> Estimated Marginal Means
#>
#> tv | Mean | 95% CI | pd
#> -----------------------------------
#> choice | 0.68 | [0.48, 0.89] | 100%
#> random | 0.69 | [0.49, 0.89] | 100%
#>
#> Marginal means estimated at tv
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Main effects contrasts: tv
#> Marginal Contrasts Analysis
#>
#> Level1 | Level2 | Difference | 95% CI | pd | % in ROPE
#> -----------------------------------------------------------------
#> choice | random | -9.41e-03 | [-0.14, 0.12] | 55.70% | 89.97%
#>
#> Marginal contrasts estimated at tv
#> NOTE: Results may be misleading due to involvement in interactions
#> $plot_mainfx
Interactions
We can plot interactions in the a similar way. We specify the interactions that we want separated by “:”. You can ask for any number of 2 or 3 way interactions. Note that the order matters. The first condition listed will be used for the colour split, the second for the x-axis split, and the 3rd, if used, will split the plot into 2 facets. Interaction plots will default to a split violin for the mean estimates, and I’ve used bars here again
bayes_mm_interact(memory_bmm, interacts = "adcon:tv",bars=T,violin=F)
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Interactions: means of levels of adcon within levels of tvEstimated Marginal Means
#>
#> adcon | tv | Mean | Median.1 | 95% CI | pd
#> ------------------------------------------------------------
#> addressable | choice | 0.75 | 0.75 | [0.54, 0.96] | 100%
#> non_address | choice | 0.62 | 0.62 | [0.41, 0.83] | 100%
#> addressable | random | 0.80 | 0.80 | [0.59, 1.01] | 100%
#> non_address | random | 0.59 | 0.59 | [0.38, 0.80] | 100%
#>
#> Marginal means estimated at adcon
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Interations: contrasts between levels of adcon within levels of tvMarginal Contrasts Analysis
#>
#> Level1 | Level2 | tv | Difference | 95% CI | pd | % in ROPE
#> -----------------------------------------------------------------------------------
#> addressable | non_address | choice | 0.13 | [0.02, 0.24] | 98.70% | 28.21%
#> addressable | non_address | random | 0.20 | [0.09, 0.32] | 99.98% | 1.79%
#>
#> Marginal contrasts estimated at adcon
#> NOTE: Results may be misleading due to involvement in interactions
#> $`adcon:tv`
We can also look at how continuous variables interact at different levels of a factor. The model plots predictions at 10 points along the x axis by default (you can specify different values if pred_values), and reports the probability that there is a non-zero slope to the lines. The plot below shows that there is strong evidence (98%) for a positive slope for age with addressable content, but weaker evidence (92%) for non-addressable content.
bayes_mm_interact(memory_bmm, interacts = c("adcon:age"))
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Interactions: means of levels of adcon within levels of ageEstimated Marginal Means
#>
#> adcon | age | Mean | Median.1 | 95% CI | pd
#> -------------------------------------------------------------
#> addressable | 18.00 | 0.63 | 0.63 | [0.40, 0.87] | 100%
#> non_address | 18.00 | 0.50 | 0.50 | [0.27, 0.74] | 99.95%
#> addressable | 23.56 | 0.66 | 0.66 | [0.45, 0.89] | 100%
#> non_address | 23.56 | 0.53 | 0.53 | [0.31, 0.75] | 99.98%
#> addressable | 29.11 | 0.70 | 0.70 | [0.50, 0.91] | 100%
#> non_address | 29.11 | 0.55 | 0.55 | [0.35, 0.76] | 100%
#> addressable | 34.67 | 0.74 | 0.74 | [0.54, 0.94] | 100%
#> non_address | 34.67 | 0.58 | 0.58 | [0.38, 0.78] | 100%
#> addressable | 40.22 | 0.77 | 0.77 | [0.58, 0.97] | 100%
#> non_address | 40.22 | 0.61 | 0.61 | [0.41, 0.80] | 100%
#> addressable | 45.78 | 0.81 | 0.81 | [0.61, 1.01] | 100%
#> non_address | 45.78 | 0.63 | 0.63 | [0.43, 0.83] | 100%
#> addressable | 51.33 | 0.85 | 0.85 | [0.64, 1.05] | 100%
#> non_address | 51.33 | 0.66 | 0.66 | [0.45, 0.87] | 100%
#> addressable | 56.89 | 0.88 | 0.88 | [0.66, 1.10] | 100%
#> non_address | 56.89 | 0.68 | 0.68 | [0.46, 0.91] | 100%
#> addressable | 62.44 | 0.92 | 0.92 | [0.69, 1.16] | 100%
#> non_address | 62.44 | 0.71 | 0.71 | [0.46, 0.95] | 100%
#> addressable | 68.00 | 0.96 | 0.96 | [0.70, 1.22] | 100%
#> non_address | 68.00 | 0.74 | 0.74 | [0.47, 1.00] | 100%
#>
#> Marginal means estimated at adcon
#> NOTE: Results may be misleading due to involvement in interactions
#> Level1 Level2 age Difference CI_low CI_high pd
#> <char> <char> <num> <num> <num> <num> <num>
#> 1: addressable non_address 18.00000 0.1248645 -0.046332652 0.2927846 0.92625
#> 2: addressable non_address 23.55556 0.1363050 -0.003206269 0.2742869 0.97125
#> 3: addressable non_address 29.11111 0.1479390 0.034583957 0.2581088 0.99600
#> 4: addressable non_address 34.66667 0.1582427 0.067653244 0.2466966 0.99975
#> 5: addressable non_address 40.22222 0.1685004 0.086140263 0.2493725 1.00000
#> 6: addressable non_address 45.77778 0.1781791 0.089145282 0.2696086 1.00000
#> 7: addressable non_address 51.33333 0.1897335 0.079566484 0.3050638 0.99975
#> 8: addressable non_address 56.88889 0.2000721 0.059931885 0.3472299 0.99800
#> 9: addressable non_address 62.44444 0.2089790 0.041949456 0.3897843 0.99300
#> 10: addressable non_address 68.00000 0.2186418 0.018632529 0.4350996 0.98525
#> ROPE_Percentage
#> <num>
#> 1: 0.37526316
#> 2: 0.28973684
#> 3: 0.18736842
#> 4: 0.08236842
#> 5: 0.02500000
#> 6: 0.01684211
#> 7: 0.03184211
#> 8: 0.05526316
#> 9: 0.07631579
#> 10: 0.10342105
#> NOTE: Results may be misleading due to involvement in interactions
#> $`adcon:age`
We can split the data up further using the third factor. This shows a three way interaction, where the adcon:age relationship is split by tv. As well as reporting the MPE for the slope, the function has looked at the evidence for there being a difference at each of the 10 intervals on the x axis. If there is no evidence (MPE<90) that there is a difference between conditions, it colours the region in grey. So the plot below shows that in the tv=choice condition on the left, there is no difference between addressability conditions for people under 30. But a difference emerges after that, since as people get older, there is strong evidence that their memory for addressable content gets better (98.8%), but no evidence (71.7%) for a memory effect with non-addressable content.
bayes_mm_interact(memory_bmm, interacts = "adcon:age:tv")
#>
#> Interactions: means of levels of adcon within levels of age tvEstimated Marginal Means
#>
#> adcon | tv | age | Mean | Median.1 | 95% CI | pd
#> ----------------------------------------------------------------------
#> addressable | choice | 18.00 | 0.52 | 0.52 | [0.24, 0.81] | 99.92%
#> non_address | choice | 18.00 | 0.55 | 0.55 | [0.28, 0.85] | 99.98%
#> addressable | random | 18.00 | 0.73 | 0.73 | [0.45, 1.02] | 100%
#> non_address | random | 18.00 | 0.45 | 0.45 | [0.16, 0.74] | 99.75%
#> addressable | choice | 23.56 | 0.58 | 0.58 | [0.33, 0.84] | 100%
#> non_address | choice | 23.56 | 0.57 | 0.57 | [0.33, 0.82] | 99.98%
#> addressable | random | 23.56 | 0.75 | 0.75 | [0.50, 1.01] | 100%
#> non_address | random | 23.56 | 0.49 | 0.49 | [0.23, 0.74] | 99.90%
#> addressable | choice | 29.11 | 0.63 | 0.63 | [0.42, 0.87] | 100%
#> non_address | choice | 29.11 | 0.59 | 0.59 | [0.36, 0.82] | 100%
#> addressable | random | 29.11 | 0.76 | 0.76 | [0.54, 1.00] | 100%
#> non_address | random | 29.11 | 0.52 | 0.52 | [0.29, 0.75] | 99.95%
#> addressable | choice | 34.67 | 0.69 | 0.69 | [0.49, 0.92] | 100%
#> non_address | choice | 34.67 | 0.60 | 0.60 | [0.39, 0.82] | 100%
#> addressable | random | 34.67 | 0.78 | 0.78 | [0.57, 1.00] | 100%
#> non_address | random | 34.67 | 0.55 | 0.55 | [0.34, 0.76] | 99.98%
#> addressable | choice | 40.22 | 0.75 | 0.75 | [0.54, 0.97] | 100%
#> non_address | choice | 40.22 | 0.62 | 0.62 | [0.41, 0.83] | 100%
#> addressable | random | 40.22 | 0.80 | 0.80 | [0.59, 1.01] | 100%
#> non_address | random | 40.22 | 0.59 | 0.59 | [0.38, 0.80] | 100%
#> addressable | choice | 45.78 | 0.81 | 0.81 | [0.59, 1.03] | 100%
#> non_address | choice | 45.78 | 0.63 | 0.63 | [0.41, 0.85] | 100%
#> addressable | random | 45.78 | 0.81 | 0.81 | [0.61, 1.02] | 100%
#> non_address | random | 45.78 | 0.63 | 0.63 | [0.41, 0.84] | 100%
#> addressable | choice | 51.33 | 0.87 | 0.87 | [0.63, 1.10] | 100%
#> non_address | choice | 51.33 | 0.65 | 0.65 | [0.41, 0.89] | 100%
#> addressable | random | 51.33 | 0.83 | 0.83 | [0.61, 1.05] | 100%
#> non_address | random | 51.33 | 0.66 | 0.66 | [0.44, 0.89] | 100%
#> addressable | choice | 56.89 | 0.92 | 0.92 | [0.66, 1.18] | 100%
#> non_address | choice | 56.89 | 0.67 | 0.67 | [0.40, 0.93] | 100%
#> addressable | random | 56.89 | 0.85 | 0.85 | [0.60, 1.09] | 100%
#> non_address | random | 56.89 | 0.70 | 0.70 | [0.45, 0.95] | 100%
#> addressable | choice | 62.44 | 0.98 | 0.98 | [0.68, 1.27] | 100%
#> non_address | choice | 62.44 | 0.69 | 0.69 | [0.38, 0.98] | 100%
#> addressable | random | 62.44 | 0.86 | 0.86 | [0.58, 1.14] | 100%
#> non_address | random | 62.44 | 0.73 | 0.73 | [0.45, 1.02] | 100%
#> addressable | choice | 68.00 | 1.04 | 1.04 | [0.70, 1.37] | 100%
#> non_address | choice | 68.00 | 0.70 | 0.70 | [0.35, 1.03] | 100%
#> addressable | random | 68.00 | 0.88 | 0.88 | [0.56, 1.20] | 100%
#> non_address | random | 68.00 | 0.77 | 0.77 | [0.45, 1.09] | 100%
#>
#> Marginal means estimated at adcon Level1 Level2 age tv Difference CI_low CI_high
#> <char> <char> <num> <fctr> <num> <num> <num>
#> 1: addressable non_address 18.00000 choice -0.033691426 -0.26325647 0.1982663
#> 2: addressable non_address 23.55556 choice 0.006604848 -0.18108262 0.1936423
#> 3: addressable non_address 29.11111 choice 0.048971710 -0.10464222 0.1991199
#> 4: addressable non_address 34.66667 choice 0.091158991 -0.03614086 0.2143361
#> 5: addressable non_address 40.22222 choice 0.131661277 0.01592892 0.2441353
#> 6: addressable non_address 45.77778 choice 0.174191796 0.04502552 0.2999803
#> 7: addressable non_address 51.33333 choice 0.215229996 0.05409199 0.3773216
#> 8: addressable non_address 56.88889 choice 0.255778648 0.05658343 0.4627878
#> 9: addressable non_address 62.44444 choice 0.295375303 0.05392302 0.5519639
#> 10: addressable non_address 68.00000 choice 0.336151606 0.04851867 0.6433499
#> 11: addressable non_address 18.00000 random 0.281901608 0.04007245 0.5350621
#> 12: addressable non_address 23.55556 random 0.263415985 0.06535460 0.4686238
#> 13: addressable non_address 29.11111 random 0.244598852 0.08191462 0.4086310
#> 14: addressable non_address 34.66667 random 0.225020912 0.09151119 0.3579055
#> 15: addressable non_address 40.22222 random 0.204529841 0.08742394 0.3227781
#> 16: addressable non_address 45.77778 random 0.184173605 0.06026805 0.3071385
#> 17: addressable non_address 51.33333 random 0.163554779 0.01282568 0.3169512
#> 18: addressable non_address 56.88889 random 0.143462904 -0.04876202 0.3352845
#> 19: addressable non_address 62.44444 random 0.122814041 -0.11284721 0.3594360
#> 20: addressable non_address 68.00000 random 0.102662823 -0.18013330 0.3814483
#> Level1 Level2 age tv Difference CI_low CI_high
#> pd ROPE_Percentage
#> <num> <num>
#> 1: 0.60900 0.60631579
#> 2: 0.53250 0.72921053
#> 3: 0.73700 0.74868421
#> 4: 0.92425 0.56263158
#> 5: 0.98725 0.27973684
#> 6: 0.99500 0.11000000
#> 7: 0.99550 0.05394737
#> 8: 0.99425 0.03552632
#> 9: 0.99125 0.03078947
#> 10: 0.98850 0.02921053
#> 11: 0.98900 0.04421053
#> 12: 0.99550 0.03210526
#> 13: 0.99875 0.01631579
#> 14: 0.99975 0.01000000
#> 15: 0.99975 0.01789474
#> 16: 0.99800 0.07342105
#> 17: 0.98325 0.19263158
#> 18: 0.92950 0.31342105
#> 19: 0.85250 0.40894737
#> 20: 0.77025 0.43421053
#> pd ROPE_Percentage
#> $`adcon:age:tv`
We can get more fine grained information about those interval
differences by displaying them all in a ladder with all the MPEs
bayes_mm_interact(memory_bmm, interacts = "adcon:age:tv",ladder=T,ribbon=F)
#>
#> Interactions: means of levels of adcon within levels of age tvEstimated Marginal Means
#>
#> adcon | tv | age | Mean | Median.1 | 95% CI | pd
#> ----------------------------------------------------------------------
#> addressable | choice | 18.00 | 0.52 | 0.52 | [0.24, 0.81] | 99.92%
#> non_address | choice | 18.00 | 0.55 | 0.55 | [0.28, 0.85] | 99.98%
#> addressable | random | 18.00 | 0.73 | 0.73 | [0.45, 1.02] | 100%
#> non_address | random | 18.00 | 0.45 | 0.45 | [0.16, 0.74] | 99.75%
#> addressable | choice | 23.56 | 0.58 | 0.58 | [0.33, 0.84] | 100%
#> non_address | choice | 23.56 | 0.57 | 0.57 | [0.33, 0.82] | 99.98%
#> addressable | random | 23.56 | 0.75 | 0.75 | [0.50, 1.01] | 100%
#> non_address | random | 23.56 | 0.49 | 0.49 | [0.23, 0.74] | 99.90%
#> addressable | choice | 29.11 | 0.63 | 0.63 | [0.42, 0.87] | 100%
#> non_address | choice | 29.11 | 0.59 | 0.59 | [0.36, 0.82] | 100%
#> addressable | random | 29.11 | 0.76 | 0.76 | [0.54, 1.00] | 100%
#> non_address | random | 29.11 | 0.52 | 0.52 | [0.29, 0.75] | 99.95%
#> addressable | choice | 34.67 | 0.69 | 0.69 | [0.49, 0.92] | 100%
#> non_address | choice | 34.67 | 0.60 | 0.60 | [0.39, 0.82] | 100%
#> addressable | random | 34.67 | 0.78 | 0.78 | [0.57, 1.00] | 100%
#> non_address | random | 34.67 | 0.55 | 0.55 | [0.34, 0.76] | 99.98%
#> addressable | choice | 40.22 | 0.75 | 0.75 | [0.54, 0.97] | 100%
#> non_address | choice | 40.22 | 0.62 | 0.62 | [0.41, 0.83] | 100%
#> addressable | random | 40.22 | 0.80 | 0.80 | [0.59, 1.01] | 100%
#> non_address | random | 40.22 | 0.59 | 0.59 | [0.38, 0.80] | 100%
#> addressable | choice | 45.78 | 0.81 | 0.81 | [0.59, 1.03] | 100%
#> non_address | choice | 45.78 | 0.63 | 0.63 | [0.41, 0.85] | 100%
#> addressable | random | 45.78 | 0.81 | 0.81 | [0.61, 1.02] | 100%
#> non_address | random | 45.78 | 0.63 | 0.63 | [0.41, 0.84] | 100%
#> addressable | choice | 51.33 | 0.87 | 0.87 | [0.63, 1.10] | 100%
#> non_address | choice | 51.33 | 0.65 | 0.65 | [0.41, 0.89] | 100%
#> addressable | random | 51.33 | 0.83 | 0.83 | [0.61, 1.05] | 100%
#> non_address | random | 51.33 | 0.66 | 0.66 | [0.44, 0.89] | 100%
#> addressable | choice | 56.89 | 0.92 | 0.92 | [0.66, 1.18] | 100%
#> non_address | choice | 56.89 | 0.67 | 0.67 | [0.40, 0.93] | 100%
#> addressable | random | 56.89 | 0.85 | 0.85 | [0.60, 1.09] | 100%
#> non_address | random | 56.89 | 0.70 | 0.70 | [0.45, 0.95] | 100%
#> addressable | choice | 62.44 | 0.98 | 0.98 | [0.68, 1.27] | 100%
#> non_address | choice | 62.44 | 0.69 | 0.69 | [0.38, 0.98] | 100%
#> addressable | random | 62.44 | 0.86 | 0.86 | [0.58, 1.14] | 100%
#> non_address | random | 62.44 | 0.73 | 0.73 | [0.45, 1.02] | 100%
#> addressable | choice | 68.00 | 1.04 | 1.04 | [0.70, 1.37] | 100%
#> non_address | choice | 68.00 | 0.70 | 0.70 | [0.35, 1.03] | 100%
#> addressable | random | 68.00 | 0.88 | 0.88 | [0.56, 1.20] | 100%
#> non_address | random | 68.00 | 0.77 | 0.77 | [0.45, 1.09] | 100%
#>
#> Marginal means estimated at adcon Level1 Level2 age tv Difference CI_low CI_high
#> <char> <char> <num> <fctr> <num> <num> <num>
#> 1: addressable non_address 18.00000 choice -0.033691426 -0.26325647 0.1982663
#> 2: addressable non_address 23.55556 choice 0.006604848 -0.18108262 0.1936423
#> 3: addressable non_address 29.11111 choice 0.048971710 -0.10464222 0.1991199
#> 4: addressable non_address 34.66667 choice 0.091158991 -0.03614086 0.2143361
#> 5: addressable non_address 40.22222 choice 0.131661277 0.01592892 0.2441353
#> 6: addressable non_address 45.77778 choice 0.174191796 0.04502552 0.2999803
#> 7: addressable non_address 51.33333 choice 0.215229996 0.05409199 0.3773216
#> 8: addressable non_address 56.88889 choice 0.255778648 0.05658343 0.4627878
#> 9: addressable non_address 62.44444 choice 0.295375303 0.05392302 0.5519639
#> 10: addressable non_address 68.00000 choice 0.336151606 0.04851867 0.6433499
#> 11: addressable non_address 18.00000 random 0.281901608 0.04007245 0.5350621
#> 12: addressable non_address 23.55556 random 0.263415985 0.06535460 0.4686238
#> 13: addressable non_address 29.11111 random 0.244598852 0.08191462 0.4086310
#> 14: addressable non_address 34.66667 random 0.225020912 0.09151119 0.3579055
#> 15: addressable non_address 40.22222 random 0.204529841 0.08742394 0.3227781
#> 16: addressable non_address 45.77778 random 0.184173605 0.06026805 0.3071385
#> 17: addressable non_address 51.33333 random 0.163554779 0.01282568 0.3169512
#> 18: addressable non_address 56.88889 random 0.143462904 -0.04876202 0.3352845
#> 19: addressable non_address 62.44444 random 0.122814041 -0.11284721 0.3594360
#> 20: addressable non_address 68.00000 random 0.102662823 -0.18013330 0.3814483
#> Level1 Level2 age tv Difference CI_low CI_high
#> pd ROPE_Percentage
#> <num> <num>
#> 1: 0.60900 0.60631579
#> 2: 0.53250 0.72921053
#> 3: 0.73700 0.74868421
#> 4: 0.92425 0.56263158
#> 5: 0.98725 0.27973684
#> 6: 0.99500 0.11000000
#> 7: 0.99550 0.05394737
#> 8: 0.99425 0.03552632
#> 9: 0.99125 0.03078947
#> 10: 0.98850 0.02921053
#> 11: 0.98900 0.04421053
#> 12: 0.99550 0.03210526
#> 13: 0.99875 0.01631579
#> 14: 0.99975 0.01000000
#> 15: 0.99975 0.01789474
#> 16: 0.99800 0.07342105
#> 17: 0.98325 0.19263158
#> 18: 0.92950 0.31342105
#> 19: 0.85250 0.40894737
#> 20: 0.77025 0.43421053
#> pd ROPE_Percentage
#> $`adcon:age:tv`
Bayes_mm wrapper function to run and analyse data
All the functions above can be carried out by a single, wrapper function, bayes_mm(). This will first run the model, then plot all the main effects and all the interactions. It will also output various model metrics that can be used in reporting, and plot the traces if asked. Here is an analysis for how people rated the adverts, run in one line. I’ve also asked for plots of the traces to check convergence. If you wanted to see more, you can use explore_browser=TRUE.
rating_bmm <- bayes_mm(data = data,dvname="rating",conditions = c("adcon","dim","tv"),groups = c("pid"),plotcheck = T)
#> [1] "running model"
#> Bayes Mixed model on rating by adcon dim tv nested in pid
#> ================================================================================
#>
#> stan_glmer
#> family: gaussian [identity]
#> formula: rating ~ adcon * dim * tv + (1 | pid)
#> observations: 804
#> ------
#> Median MAD_SD
#> (Intercept) 1.8 0.4
#> adconnon_address -0.8 0.6
#> dimcruise 0.4 0.5
#> dimdog -0.5 0.6
#> dimgames 0.9 0.5
#> tvrandom -1.0 0.6
#> adconnon_address:dimcruise -0.5 0.9
#> adconnon_address:dimdog -0.7 0.9
#> adconnon_address:dimgames -3.6 0.9
#> adconnon_address:tvrandom 0.7 0.9
#> dimcruise:tvrandom 0.6 0.8
#> dimdog:tvrandom 1.4 0.8
#> dimgames:tvrandom 0.2 0.8
#> adconnon_address:dimcruise:tvrandom -1.1 1.2
#> adconnon_address:dimdog:tvrandom -0.1 1.2
#> adconnon_address:dimgames:tvrandom 0.6 1.2
#>
#> Auxiliary parameter(s):
#> Median MAD_SD
#> sigma 2.7 0.1
#>
#> Error terms:
#> Groups Name Std.Dev.
#> pid (Intercept) 2.0
#> Residual 2.7
#> Num. levels: pid 201
#>
#> ------
#> * For help interpreting the printed output see ?print.stanreg
#> * For info on the priors used see ?prior_summary.stanreg
#>
#> Priors for model 'bmm'
#> ------
#> Intercept (after predictors centered)
#> Specified prior:
#> ~ normal(location = 0.89, scale = 2.5)
#> Adjusted prior:
#> ~ normal(location = 0.89, scale = 8.7)
#>
#> Coefficients
#> Specified prior:
#> ~ normal(location = [0,0,0,...], scale = [2.5,2.5,2.5,...])
#> Adjusted prior:
#> ~ normal(location = [0,0,0,...], scale = [17.44,20.14,20.14,...])
#>
#> Auxiliary (sigma)
#> Specified prior:
#> ~ exponential(rate = 1)
#> Adjusted prior:
#> ~ exponential(rate = 0.29)
#>
#> Covariance
#> ~ decov(reg. = 1, conc. = 1, shape = 1, scale = 1)
#> ------
#> See help('prior_summary.stanreg') for more details
#>
#> Parameter | Median | 95% CI | pd | Rhat | ESS | Prior
#> -----------------------------------------------------------------------------------------------------------------
#> (Intercept) | 1.78 | [ 0.96, 2.60] | 100% | 1.000 | 1037.00 | Normal (0.89 +- 8.73)
#> adconnon_address | -0.79 | [-1.96, 0.42] | 90.33% | 1.003 | 908.00 | Normal (0.00 +- 17.44)
#> dimcruise | 0.45 | [-0.62, 1.55] | 79.27% | 1.001 | 1249.00 | Normal (0.00 +- 20.14)
#> dimdog | -0.49 | [-1.65, 0.70] | 79.62% | 1.002 | 1232.00 | Normal (0.00 +- 20.14)
#> dimgames | 0.87 | [-0.18, 1.94] | 94.83% | 1.003 | 1309.00 | Normal (0.00 +- 20.14)
#> tvrandom | -1.02 | [-2.24, 0.19] | 95.10% | 1.002 | 821.00 | Normal (0.00 +- 17.45)
#> adconnon_address:dimcruise | -0.50 | [-2.22, 1.16] | 72.25% | 1.002 | 1049.00 | Normal (0.00 +- 25.78)
#> adconnon_address:dimdog | -0.67 | [-2.42, 1.14] | 77.03% | 1.002 | 1035.00 | Normal (0.00 +- 25.99)
#> adconnon_address:dimgames | -3.62 | [-5.34, -1.96] | 100% | 1.002 | 1134.00 | Normal (0.00 +- 26.54)
#> adconnon_address:tvrandom | 0.68 | [-0.94, 2.34] | 79.22% | 1.002 | 797.00 | Normal (0.00 +- 20.24)
#> dimcruise:tvrandom | 0.59 | [-0.98, 2.13] | 77.78% | 1.002 | 1193.00 | Normal (0.00 +- 26.54)
#> dimdog:tvrandom | 1.37 | [-0.19, 2.92] | 95.50% | 1.002 | 1085.00 | Normal (0.00 +- 26.54)
#> dimgames:tvrandom | 0.22 | [-1.36, 1.79] | 60.20% | 1.002 | 1128.00 | Normal (0.00 +- 26.54)
#> adconnon_address:dimcruise:tvrandom | -1.12 | [-3.40, 1.32] | 81.73% | 1.002 | 1052.00 | Normal (0.00 +- 35.46)
#> adconnon_address:dimdog:tvrandom | -0.05 | [-2.53, 2.30] | 51.73% | 1.002 | 957.00 | Normal (0.00 +- 37.94)
#> adconnon_address:dimgames:tvrandom | 0.58 | [-1.78, 3.03] | 67.47% | 1.003 | 1004.00 | Normal (0.00 +- 36.12)
#>
#> Uncertainty intervals (equal-tailed) and p-values (two-tailed) computed
#> using a MCMC distribution approximation.
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Main effects means: adcon
#> Estimated Marginal Means
#>
#> adcon | Mean | 95% CI | pd
#> -------------------------------------------
#> addressable | 1.74 | [ 1.36, 2.12] | 100%
#> non_address | 0.03 | [-0.36, 0.41] | 55.27%
#>
#> Marginal means estimated at adcon
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Main effects contrasts: adcon
#> Marginal Contrasts Analysis
#>
#> Level1 | Level2 | Difference | 95% CI | pd | % in ROPE
#> ------------------------------------------------------------------------
#> addressable | non_address | 1.72 | [1.35, 2.09] | 100% | 0%
#>
#> Marginal contrasts estimated at adcon
#> NOTE: Results may be misleading due to involvement in interactions
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Main effects means: dim
#> Estimated Marginal Means
#>
#> dim | Mean | 95% CI | pd
#> --------------------------------------
#> car | 1.04 | [ 0.58, 1.48] | 100%
#> cruise | 1.26 | [ 0.80, 1.74] | 100%
#> dog | 0.90 | [ 0.43, 1.36] | 100%
#> games | 0.36 | [-0.12, 0.82] | 93.38%
#>
#> Marginal means estimated at dim
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Main effects contrasts: dim
#> Marginal Contrasts Analysis
#>
#> Level1 | Level2 | Difference | 95% CI | pd | % in ROPE
#> -----------------------------------------------------------------
#> car | cruise | -0.22 | [-0.76, 0.29] | 79.62% | 22.34%
#> car | dog | 0.15 | [-0.35, 0.66] | 70.65% | 25.79%
#> car | games | 0.68 | [ 0.16, 1.20] | 99.35% | 0%
#> cruise | dog | 0.37 | [-0.14, 0.90] | 91.92% | 12.63%
#> cruise | games | 0.91 | [ 0.39, 1.42] | 99.98% | 0%
#> dog | games | 0.54 | [ 0.00, 1.07] | 97.42% | 3.68%
#>
#> Marginal contrasts estimated at dim
#> NOTE: Results may be misleading due to involvement in interactions
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Main effects means: tv
#> Estimated Marginal Means
#>
#> tv | Mean | 95% CI | pd
#> -------------------------------------
#> choice | 0.99 | [0.50, 1.46] | 100%
#> random | 0.79 | [0.31, 1.26] | 99.80%
#>
#> Marginal means estimated at tv
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Main effects contrasts: tv
#> Marginal Contrasts Analysis
#>
#> Level1 | Level2 | Difference | 95% CI | pd | % in ROPE
#> -----------------------------------------------------------------
#> choice | random | 0.21 | [-0.48, 0.87] | 73.10% | 20.61%
#>
#> Marginal contrasts estimated at tv
#> NOTE: Results may be misleading due to involvement in interactions
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Interactions: means of levels of adcon within levels of dimEstimated Marginal Means
#>
#> adcon | dim | Mean | Median.1 | 95% CI | pd
#> -----------------------------------------------------------------
#> addressable | car | 1.27 | 1.27 | [ 0.67, 1.86] | 100%
#> non_address | car | 0.82 | 0.82 | [ 0.15, 1.44] | 99.20%
#> addressable | cruise | 2.01 | 2.01 | [ 1.37, 2.67] | 100%
#> non_address | cruise | 0.51 | 0.51 | [-0.10, 1.13] | 95.35%
#> addressable | dog | 1.47 | 1.47 | [ 0.83, 2.09] | 100%
#> non_address | dog | 0.31 | 0.31 | [-0.31, 0.96] | 83.25%
#> addressable | games | 2.24 | 2.24 | [ 1.65, 2.85] | 100%
#> non_address | games | -1.53 | -1.53 | [-2.16, -0.90] | 100%
#>
#> Marginal means estimated at adcon
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Interations: contrasts between levels of adcon within levels of dimMarginal Contrasts Analysis
#>
#> Level1 | Level2 | dim | Difference | 95% CI | pd | % in ROPE
#> ------------------------------------------------------------------------------------
#> addressable | non_address | car | 0.44 | [-0.37, 1.29] | 85.72% | 11.55%
#> addressable | non_address | cruise | 1.50 | [ 0.67, 2.35] | 100% | 0%
#> addressable | non_address | dog | 1.16 | [ 0.28, 1.97] | 99.65% | 0%
#> addressable | non_address | games | 3.77 | [ 2.97, 4.60] | 100% | 0%
#>
#> Marginal contrasts estimated at adcon
#> NOTE: Results may be misleading due to involvement in interactions
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Interactions: means of levels of adcon within levels of tvEstimated Marginal Means
#>
#> adcon | tv | Mean | Median.1 | 95% CI | pd
#> -------------------------------------------------------------------
#> addressable | choice | 1.99 | 1.99 | [ 1.44, 2.53] | 100%
#> non_address | choice | 5.32e-04 | 5.32e-04 | [-0.54, 0.54] | 50.08%
#> addressable | random | 1.51 | 1.51 | [ 0.97, 2.04] | 100%
#> non_address | random | 0.06 | 0.06 | [-0.49, 0.62] | 59.40%
#>
#> Marginal means estimated at adcon
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Interations: contrasts between levels of adcon within levels of tvMarginal Contrasts Analysis
#>
#> Level1 | Level2 | tv | Difference | 95% CI | pd | % in ROPE
#> ---------------------------------------------------------------------------------
#> addressable | non_address | choice | 1.99 | [1.49, 2.50] | 100% | 0%
#> addressable | non_address | random | 1.45 | [0.93, 1.97] | 100% | 0%
#>
#> Marginal contrasts estimated at adcon
#> NOTE: Results may be misleading due to involvement in interactions
#>
#> Interactions: means of levels of adcon within levels of dim tvEstimated Marginal Means
#>
#> adcon | dim | tv | Mean | Median.1 | 95% CI | pd
#> --------------------------------------------------------------------------
#> addressable | car | choice | 1.78 | 1.78 | [ 0.96, 2.60] | 100%
#> non_address | car | choice | 0.99 | 0.99 | [ 0.04, 1.90] | 97.82%
#> addressable | cruise | choice | 2.22 | 2.22 | [ 1.33, 3.11] | 100%
#> non_address | cruise | choice | 0.93 | 0.93 | [ 0.09, 1.76] | 98.50%
#> addressable | dog | choice | 1.29 | 1.29 | [ 0.30, 2.21] | 99.62%
#> non_address | dog | choice | -0.17 | -0.17 | [-1.00, 0.65] | 65.35%
#> addressable | games | choice | 2.65 | 2.65 | [ 1.80, 3.51] | 100%
#> non_address | games | choice | -1.77 | -1.77 | [-2.67, -0.90] | 100%
#> addressable | car | random | 0.76 | 0.76 | [-0.13, 1.61] | 95.38%
#> non_address | car | random | 0.65 | 0.65 | [-0.24, 1.51] | 93.05%
#> addressable | cruise | random | 1.81 | 1.81 | [ 0.88, 2.73] | 100%
#> non_address | cruise | random | 0.09 | 0.09 | [-0.77, 1.00] | 58.33%
#> addressable | dog | random | 1.65 | 1.65 | [ 0.81, 2.46] | 100%
#> non_address | dog | random | 0.80 | 0.80 | [-0.14, 1.74] | 95.08%
#> addressable | games | random | 1.84 | 1.84 | [ 0.97, 2.70] | 100%
#> non_address | games | random | -1.30 | -1.30 | [-2.21, -0.44] | 99.85%
#>
#> Marginal means estimated at adcon
#> Interations: contrasts between levels of adcon within levels of dim tvMarginal Contrasts Analysis
#>
#> Level1 | Level2 | dim | tv | Difference | 95% CI | pd | % in ROPE
#> ---------------------------------------------------------------------------------------------
#> addressable | non_address | car | choice | 0.79 | [-0.42, 1.96] | 90.33% | 5.71%
#> addressable | non_address | cruise | choice | 1.29 | [ 0.14, 2.47] | 98.62% | 0%
#> addressable | non_address | dog | choice | 1.46 | [ 0.27, 2.62] | 99.20% | 0%
#> addressable | non_address | games | choice | 4.41 | [ 3.28, 5.54] | 100% | 0%
#> addressable | non_address | car | random | 0.12 | [-1.04, 1.25] | 57.23% | 13.08%
#> addressable | non_address | cruise | random | 1.70 | [ 0.53, 2.88] | 99.85% | 0%
#> addressable | non_address | dog | random | 0.85 | [-0.33, 2.01] | 92.07% | 4.68%
#> addressable | non_address | games | random | 3.13 | [ 1.97, 4.33] | 100% | 0%
#>
#> Marginal contrasts estimated at adcon
#> $`adcon:dim`
#>
#> $`adcon:tv`
#>
#> $`adcon:dim:tv`
Since we have saved the output of the model, we can then do follow up analyses to replot and look at things of interest, by passing the model back through one of our plotting functions. Here’s the three way interaction:
bayes_mm_interact(rating_bmm,interacts = "adcon:dim:tv",violin=F,contrast_sparkline = T,show_mpe = F)
#>
#> Interactions: means of levels of adcon within levels of dim tvEstimated Marginal Means
#>
#> adcon | dim | tv | Mean | Median.1 | 95% CI | pd
#> --------------------------------------------------------------------------
#> addressable | car | choice | 1.78 | 1.78 | [ 0.96, 2.60] | 100%
#> non_address | car | choice | 0.99 | 0.99 | [ 0.04, 1.90] | 97.82%
#> addressable | cruise | choice | 2.22 | 2.22 | [ 1.33, 3.11] | 100%
#> non_address | cruise | choice | 0.93 | 0.93 | [ 0.09, 1.76] | 98.50%
#> addressable | dog | choice | 1.29 | 1.29 | [ 0.30, 2.21] | 99.62%
#> non_address | dog | choice | -0.17 | -0.17 | [-1.00, 0.65] | 65.35%
#> addressable | games | choice | 2.65 | 2.65 | [ 1.80, 3.51] | 100%
#> non_address | games | choice | -1.77 | -1.77 | [-2.67, -0.90] | 100%
#> addressable | car | random | 0.76 | 0.76 | [-0.13, 1.61] | 95.38%
#> non_address | car | random | 0.65 | 0.65 | [-0.24, 1.51] | 93.05%
#> addressable | cruise | random | 1.81 | 1.81 | [ 0.88, 2.73] | 100%
#> non_address | cruise | random | 0.09 | 0.09 | [-0.77, 1.00] | 58.33%
#> addressable | dog | random | 1.65 | 1.65 | [ 0.81, 2.46] | 100%
#> non_address | dog | random | 0.80 | 0.80 | [-0.14, 1.74] | 95.08%
#> addressable | games | random | 1.84 | 1.84 | [ 0.97, 2.70] | 100%
#> non_address | games | random | -1.30 | -1.30 | [-2.21, -0.44] | 99.85%
#>
#> Marginal means estimated at adcon
#> Interations: contrasts between levels of adcon within levels of dim tvMarginal Contrasts Analysis
#>
#> Level1 | Level2 | dim | tv | Difference | 95% CI | pd | % in ROPE
#> ---------------------------------------------------------------------------------------------
#> addressable | non_address | car | choice | 0.79 | [-0.42, 1.96] | 90.33% | 5.71%
#> addressable | non_address | cruise | choice | 1.29 | [ 0.14, 2.47] | 98.62% | 0%
#> addressable | non_address | dog | choice | 1.46 | [ 0.27, 2.62] | 99.20% | 0%
#> addressable | non_address | games | choice | 4.41 | [ 3.28, 5.54] | 100% | 0%
#> addressable | non_address | car | random | 0.12 | [-1.04, 1.25] | 57.23% | 13.08%
#> addressable | non_address | cruise | random | 1.70 | [ 0.53, 2.88] | 99.85% | 0%
#> addressable | non_address | dog | random | 0.85 | [-0.33, 2.01] | 92.07% | 4.68%
#> addressable | non_address | games | random | 3.13 | [ 1.97, 4.33] | 100% | 0%
#>
#> Marginal contrasts estimated at adcon
#> $`adcon:dim:tv`
If you run into any problems, please raise an issue with github!