16  GLM: Repeated Measures ANOVA

Repeated Measures ANOVA is used to analyze data collected in within-participants designs, where the same outcome measure is collected from the same individuals multiple times.

A study design in which the same participants are assessed repeatedly is called a Within-Participants Design. Within-participants designs have distinct advantages in comparison to between-participants designs. In these designs, participants serve as their own control, eliminating variability due to individual differences from the error term. This intrinsic control enhances statistical power and efficiency. These designs are used, for example, in longitudinal studies, test-retest designs, diary studies, and repeated physiological assessments.

While within-participants designs offer significant advantages, they also present challenges that require careful consideration. Order effects, where the sequence of experimental conditions influences results, are a common concern. Differential order effects, where the influence of order varies across different sequences, can further complicate data interpretation. An example of a differential order effect is wen the effect of a drug administered before a placebo condition persists into the placebo phase of the experiment. To mitigate order effects, researchers often employ the Latin square design. This experimental design ensures each condition appears once in every position within the order, thus minimizing the influence of sequence on outcomes. By controlling for order effects, researchers enhance the internal validity of their experiments.

Beyond order effects, within-participants designs are also affected by learning- and historical effects. A learning effect occurs when participants’ increasing familiarity with questionnaires affects their subsequent responding. Historical effects occurr when external events happen during the study, and influence participants’ responses. Finally, the effect of time is often confounded with the effect of experimental conditions.

16.1 Two Repeated Measurements

The paired samples t-test is suitable for scenarios where participants are measured before and after an intervention. This technique simply analyzes the difference score between pretest and posttest scores.

16.2 More Than Two Measurements

For scenarios with more than two repeated measurements, there are two potential solutions: the linear mixed model, and the multivariate approach. The linear mixed model, treats all repeated measurements as a single variable with multiple observations per participant. Thus, if one participant gave four repeated measurements, we would have four rows in the data for that participant. The multivariate approach treats the repeated measurements as correlated outcomes. Each measurement occasion is analyzed while controlling for the other measurement occasions.

16.3 Sphericity Assumption

The linear mixed model assumes sphericity, which is analogous to the assumption of homogeneity of error variance. Sphericity implies that the variances of the differences between all combinations of repeated measures are equal.

If you do not, or can not, assume sphericity, you can use a corrected test for the linear mixed model, or switch to the multivariate approach.

16.4 Mixed Designs

A mixed design involves both within-participants and between-participants factors. This factorial design allows researchers to examine interactions between these factors, such as the interplay between time and exposure conditions. Post hoc analyses can be used to understand the direction and significance of these interactions.

17 Lecture

18 Formative Test

A formative test helps you assess your progress in the course, and helps you address any blind spots in your understanding of the material. If you get a question wrong, you will receive a hint on how to improve your understanding of the material.

Complete the formative test ideally after you’ve seen the lecture, but before the lecture meeting in which we can discuss any topics that need more attention

Question 1

In Repeated Measures ANOVA, what type of experimental design involves the same participants being exposed to multiple conditions?

Between-participants designLongitudinal designWithin-participants designCross-sectional design

Question 2

Which of these is NOT a methodological concern in within-participants designs?

Historical effectsLearning effectsOrder effectsInteraction effects

Question 3

How does the Latin square design address order effects in experiments?

Randomly assigns participants to different conditionsSelects a single condition for all participantsControls for order effects by ensuring each condition appears once in every positionManipulates the order of conditions for experimental purposes

Question 4

What is the primary advantage of within-participants designs in comparison to between-participants designs?

Larger sample sizesGreater external validityLower costElimination of variability due to individual differences from the error term

Question 5

What statistical technique is commonly used to analyze data collected in within-participants designs with two repeated measurements?

Independent samples t-testChi-square testPaired samples t-testAnalysis of variance

Question 6

What assumption of the general linear model is violated when analyzing data with repeated measurements?

Linearity of relationshipsNormal distribution of errorsIndependence of errorsHomogeneity of variances

Question 7

What is the purpose of using a multivariate approach in analyzing data with more than two repeated measurements?

To perform a test that is robust to violations of sphericityTo ignore the repeated measurementsTo assess the order effects in the dataTo calculate mean differences between measurements

Question 8

What is the key assumption in the multivariate approach for analyzing data with repeated measurements?

Normality assumptionSphericity assumptionHomoscedasticity assumptionIndependence assumption

Question 9

In mixed design ANOVA, what type of factors are considered?

Categorical and continuous factorsOnly within-participants factorsBoth within-participants and between-participants factorsOnly between-participants factors

Question 10

What does the term ‘sphericity’ refer to in the context of repeated measures ANOVA?

The shape of the distribution of the outcomesThe distribution of the residualsEqual variances and correlations of differences scores of all pairs of repeated measurementsThe distribution of the predictor variables

Question 1

In a within-participants design, the same participants are exposed to different conditions, allowing for the comparison of outcomes within the same individuals.

Question 2

Order effects refer to the potential impact of the sequence in which conditions are presented on the observed outcomes. Learning effects imply that participants respond to a questionnaire differently when they already know the questions. Historical effects mean that something external happens while you are running the experiment. Interaction effects are a statistical term.

Question 3

The Latin square design helps control for order effects by ensuring that each condition appears in each position within the order an equal number of times.

Question 4

Within-participants designs allow each participant to serve as their own control, effectively removing variability due to individual differences from the error term. The cost is often indeed lower, but that’s not the primary advantage.

Question 5

The paired samples t-test is used to analyze the differences between two related measurements, such as pretest and posttest scores.

Question 6

Repeated measurements within the same individuals violate the assumption of independence of errors, as observations from the same participant are likely to be correlated.

Question 7

A multivariate approach is robust to the assumption of sphericity because it considers the interrelationships between different repeated measurements, treating them as correlated outcomes.

Question 8

The sphericity assumption assumes that the variances and correlations among all pairs of repeated measurements are equal, which is essential for accurate results.

Question 9

Mixed design ANOVA involves the consideration of both within-participants and between-participants factors to understand the interactions between these factors on the outcomes.

Question 10

Sphericity refers to the assumption that the variances and correlations among all difference scores between pairs of repeated measurements are equal, which is crucial for accurate analysis.

19 Tutorial

19.1 Repeated Measures ANOVA

In this tutorial, we will explore how to perform a repeated-measures ANOVA using SPSS to assess the effect of repeated measurements of depression symptoms in a sample of military veterans. The primary objective is to determine whether there are significant changes in depression symptom scores across multiple time points.

Load the dataset called depression.sav containing depression symptom scores at different time points for each participant.

3. Click on “Analyze” in the top menu and select “General Linear Model” and then “Repeated Measures.”

19.1.1 Defining the Within-Subjects Factor

1. In the “Repeated Measures” dialog box, name your within-subjects factor as “time.”
2. Specify the number of levels as 4 (since there are four repeated measurements).
3. Click the “Add” button.

19.1.2 Defining Within-Subjects Variables

1. Click on the “Define” button to configure within-subjects variables.
2. In the “Repeated Measures” dialog box, move the variables corresponding to each time point (e.g., scl1, scl2, scl3, scl4) to the “Within-Subjects Variables” box while maintaining their correct order.

Configuring Options

1. Click the “Options” button.
2. Check the boxes for “Descriptive statistics” and “Estimate of effect size.”
3. Click “Continue.”

Running the Test

1. Click “OK” to run the repeated-measures ANOVA.
2. The result will appear in the Output Viewer.

Interpreting the Result

Descriptive Statistics

The descriptive statistics provide insight into the direction of any potential effect. The means comparison shows the average depression symptom scores at different time points.

True or false: There is an increase in symptoms over time. TRUE FALSE

Assumption of Sphericity

SPSS tests assumption of sphericity using Mauchly’s test of sphericity.

True or false: In this analysis, the assumption of sphericity is met. TRUE FALSE

True or false: According to the Huyn-Feldt estimate of epsilon, the deviation from sphericity is small. TRUE FALSE

Let’s assume sphericity for now. Choose the appropriate test and correction based on this assumption.

What is the appropriate F-value for the chosen test?

What is the appropriate df for the chosen test?

19.2 Pairwise Comparisons

Examine the table of pairwise comparisons.

Which difference is smallest? T2 v T4 T3 v T4 T2 v T3 T1 v T2

If you were to use Bonferroni correction to control for multiple comparisons, you would divide the experiment-wise alpha level by the number of comparisons. How many comparisons are you making here?

Report your results. Make sure to reference both the RM-ANOVA test, and post hoc comparisons with Bonferroni correction. Then, check your answer.

“A repeated-measures ANOVA revealed a significant effect of time on depression symptom scores, F(3, 2931) = 7.29, p < .001. For post hoc pairwise comparisons, we applied a Bonferroni correction. Since there are 6 comparisons between 4 time points, we established the alpha level as .05/6 = .008. Using this alpha level, we found that the mean depression symptom score increased significantly from T1 to T3 (Mean difference = .29, p = .003), and from T1 to T4 (Mean difference = .41, p < .001). These results suggest that depression symptoms increased significantly over time for the military veteran sample.”