GLM Repeated Measures:
One Within, One Between

Reading: SPSS Advanced Models 9.0: 2. Repeated Measures
Homework:
Download: glm_1w1b.sav      (Download Tips)

1. Overview

This is a continuation of our discussion of within-subjects designs. In the previous set of notes, GLM: Repeated-measures designs - One within-subjects factor, we discussed how to set up GLM to run a repeated measures design with one within-subjects factor. In this set of notes we extend the model to include a between-subjects factor.

top

2. The Data

The previous set of notes described a treatment outcome study that used Eye Movement Desensitization and Reprocessing (EMDR) as a treatment for psychological trauma (Wilson, Becker, & Tinker, 1995, 1997). In this set of notes we extend the analysis of that study by comparing participants who were randomly assigned to the immediate treatment condition with those randomly assigned to the delayed treatment condition. Assignment to treatment conditions becomes the between-subjects factor in the design.

The design is shown in Table 1. Participants in the immediate treatment condition received three 90-minute EMDR sessions between the two assessments. Participants in the delayed treatment condition received no treatment between the two assessments. The delayed treatment condition acts as control for history and regression to the mean artifacts. For example, perhaps people responded to the ads for the study because they were feeling particularly disturbed by their trauma. And maybe they would naturally feel better with the mere passage of time. The delayed treatment condition controls for the possibility of that "regression to the mean" threat to validity of the study. (Participants in the delayed treatment condition received the EMDR treatment following the postdelay assessment. Those results are not analyzed in this set of notes.)

The data are stored in the file glm_1w1b.sav. The variables in that file are shown in Table 2. The dependent measure is the Avoidance score of the Impact of Events scale (IES; Horowitz, Wilner, & Alvarez, 1979). Avoidance symptoms include trying not to think about the traumatic event and staying away from situations and people that remind you of the event.

top

3. Select the GLM Repeated Measures Procedure

The GLM Repeated Measures dialog box is opened by clicking

Analyze
      General Linear Model
            Repeated Measures...

GLM - Repeated Measures Define Factor(s)

The within-subject factors are defined in the opening window. Lets call the within-subjects factor in this example prepost. The prepost factor has two levels. Add prepost(2) to the within-subject factor window.

Lets also name the measure this time so that the output will not refer to the measure as measure_1. Lets name the measure ies_avd, the avoidance score of the IES. You do not need to name the measure, it just makes the output easier to read when we see ies_avd rather than measure_1.

GLM - Repeated Measures

The next step is to define each of the two levels of the time within-subjects variable. Move each of the variables to the Within-Subjects Variables (time): window. Make sure that the conceptual order of the original variables is preserved when you move them to the within-subjects variables window, that is, iesa_pre should moved to the __?__(1,ies_avd) space, iesa_pos should be moved to the __?__(2,ies_avd) space.

There is a between-subjects factor in this design so we need to move that factor, txcond, to the Between-Subjects Factor(s): window.

top

4. Transformations for the Within-Subjects Factor

There is one degree of freedom for the prepost within-subjects factor. GLM will create two transformed scores, one for the average of the repeated measures variables, iesa_pre & iesa_pos, and one for the prepost main effect.

The coefficients used for the creating the transformed variables can be displayed by clicking the Display...Transformation matrix option in the Options... dialog box.

top

5. The Basic Output

The basic output is similar as for a repeated measures design with one within-subject factor. The following tables are included in the output: Within Subjects Factors, see Table 5; Between Subjects Factors, see Table 6; Mauchly's Test of Sphericity, see Table 7; Tests of Within Subjects Effects, see Table 8; Tests of Within-Subjects Contrasts, see Table 8, and Tests of Between-Subjects Effects, see Table 9, and Multivariate Tests, see Table 10.

Within- and Between Subject Factor Definition

Univariate Tests

Univariate Assumptions

As was noted in the previous set of notes for repeated measures, the within-subject effect is analyzed in GLM by first transforming the original variables into single degree of freedom tests of the null hypothesis. In this example the prepost within-subject effect has a single degree of freedom so there is only one single degree of freedom estimate of the prepost effect.

Because GLM does not have to average multiple single degree of freedom tests to get an overall test of the effect, the assumptions about the correlations between the transformed variables being zero and the variances of the transformed variables being equal are not pertinent. The Mauchly sphericity test will still be printed, see Table 7, but the Mauchly Wstatistic will be 1.00, the df will be 0 and the significance level will not be printed. In addition the epsilons will all be 1.00, meaning that the corrected df are the same as the original df.

Univariate ANOVA Results

Univariate tests of the time within-subject effect are shown in Table 8. The sphericity assumed row indicates that the statistics in that row have not been adjusted by any of the epsilons. It is appropriate in this example because with 1 df tests the sphericity assumption is not relevant.

Measure: IES_AVD

Tests of individual contrasts are shown in Table 9.  The default contrasts are orthogonal polynomial (linear, quadratic, cubic, etc.) contrasts.  Because there is only 1 df for these effects only one contrast is shown, the linear effect.  These linear effects are the same effects that are shown in the "Tests of Within-Subject Effects" table. There are no corrections for sphericity in the tests of within-subjects contrasts because each contrast is a single degree of freedom test.

Measure: IES_AVD

The results of the tests of the within-subject effects indicate significant effects for both the prepost main effect, F(1, 74) = 73.11, p < .0005, and the prepost by treatment condition interaction, F(1, 74) = 41.45, p < .001. For this study, the interaction was critical.  It was hypothesized that avoidance symptoms would decrease from pretreatment to posttreatment for the participants in the immediate treatment condition and that there would no change for participants in the delayed treatment condition because they have not yet received treatment.

Univariate tests of the main effect of treatment condition (a between-subjects effect) are shown in Table 10. Recall that the between-subjects effects are tested by the transformed variable called average.

The intercept tests whether the transformed variable average is different from zero. It is not of interest to us.

The main effect for treatment condition is not significant, F(1, 74) = 2.48, p = .12.

Multivariate Tests

Multivariate tests of the within-subject effects are also displayed by the GLM procedure, see Table 11. When the within-subjects tests are single degree of freedom tests, as in this example, then the multivariate Fs will be identical to the univariate Fs.

Table 11. Multivariate Tests^b

top

6. Interpreting Significant Effects: Displaying the Means

There are three different ways to display the means: 
(a) display the descriptive statistics; 
(b) display the estimated marginal means, and 
(c) display profile plots or plots with error bars.

Descriptive Statistics

Estimated Marginal Means

Profile Plots

To display a profile plot go to the Plots... dialog box. The dependent variable is always on the y-axis. In a repeated measures design the within-subjects factor is typically placed on the horizontal axis, move prepost to the horizontal axis window. Then move the between subjects factor, txcond, to the separate lines window The resulting profile plot is shown in Figure 1. The plots show a large decrease in the IES avoidance scores for the immediate treatment group from pretreatment to posttreatment. The change for the delayed treatment group from pretreatment to postdelay is small.

top

7. Interpreting Significant Effects: Tests for Main Effects

If there are no significant interactions, then it is appropriate to interpret any significant main effects. Main effect tests are available from the Compare main effects option in the Estimated Marginal Means section of the Options... dialog box and from the Post Hoc dialog box.

Estimated Marginal Means

The Compare main effects option provides pairwise main-effect tests for either within- or between-subjects factors. You can select uncorrected (LSD) confidence intervals for the pairwise comparisons or you could correct the confidence intervals using either the Bonferroni or Sidak adjustments. 

The estimated marginal means are the unweighted means. These are the appropriate means for designs with either equal ns or unequal ns.

In other words, the tests provided by the Compare main effects option is a general solution to making pairwise comparisons that is appropriate for every situation.

Post Hoc Dialog Box

The Post Hoc dialog box provides a wide array of post hoc tests for between subjects main effects. It will not provide post hoc tests for within-subjects factors.

In our example you could run post-hoc tests on the txcond between-subjects main effect.

These post hoc tests use the unweighted means so the tests are not appropriate if you have more than one between subjects factor and you have unequal ns.

So when is it appropriate to use these tests?

As a general rule of thumb I would stick with the more general set of comparisons that are available from the Compare main effects option within the Estimated Marginal Means section of the Options... dialog box. 

8. Interpreting Significant Interactions: Simple Main Effects

The hypotheses for this study were that EMDR treatment would result in a significant decrease in PTSD symptoms, and that there would be no change in PTSD symptoms for participants in the delayed treatment condition prior to treatment. This pair of hypotheses imply that the interaction would be significant, which it is. But the hypotheses state a specific form of the interaction.  These two specific hypotheses can be tested as simple main effects the change from pretreatment to posttreatment within each treatment condition.

Simple main effects for a mixed design such as this (i.e., a design with both within- and between-subjects factors) are computed in the same way as for a between-subjects design (see the notes for GLM: Simple Main Effects). Go to the Estimated Marginal Means section of the Options... dialog box. Make sure that the interaction term is displayed in the Display Means for: window and press Continue. The next step is to edit the syntax for the /EMMEANS subcommand. To do that press the Paste button in the GLM - Repeated Measures dialog box, the Syntax Editor window will open.

Two types of output are displayed, the ANOVA for the two simple main effects, see Table 15, and pairwise comparisons, see Table 16. The ANOVA table presents a set of multivariate tests of the simple main effects. Normally people would report the F value rather than any of the multivariate test values. In this example the IES avoidance score at posttreatment (M = 6.64, SE = 1.426) was significantly lower than the pretreatment score (M = 21.03 , SE = 1.458 ), F(1, 74) = 115.37, p < .0005. There was no difference between the postdelay (M = 15.70, SE = 1.464 ) and predelay scores (M = 17.73, SE = 1.497) for participants who had not received EMDR treatment, F(1,74) = 2.17, p = .145.

You come to the same conclusion by examining the pairwise comparisons for the simple main effects. Even the significance levels are the same because only two means are being compared for each simple main effect.

Statistical Note:  Simple main effects for 2-way interactions in a two-way, mixed design always compare either (a) between-subject means (e.g. simple main effects of treatment group at pretreatment and at posttreatment) or (b) within-subject means (e.g., simple main effects of pre- and posttreatment measures within each treatment condition).  When between subject means are compared, the comparisons are between subject t tests.  The comparisons are made on the individual means using the standard errors of each mean. When within subject means are compared, the comparisons are paired t tests.  The the comparisons are made on the difference scores using the standard error of the difference score.

Writing up the Results of the Simple Main Effects Analysis

The description of the results would begin by describing the overall analysis of variance.  The description should include the dependent variable(s) analyzed and each of the factors in the design.  When the design is mixed then each factor should be described as either within-subjects or between subjects.     followed by the significant effects from that analysis. The first time you use the term "analysis of variance" you should write out the whole phrase and put the abbreviation (ANOVA) in parentheses.  Subsequently you should just use the abbreviation, ANOVA.

Next the significant effects from that analysis would be described.

Finally,  you would describe the simple main effects analysis and the results from that analysis. Write out your description of the simple main effects analysis in the box that follows and then press SUBMIT. Please press the Enter key at the end of each line of text. We will discuss your (anonymous) contributions in class.

Click here:  View Descriptions to read your descriptions of the simple main effects analysis.

top

9. Interpreting Significant Interactions: Tukey's HSD

Pairwise comparisons for all between subjects effects or all within subjects effects

Tukey's HSD test could be used to make all pairwise comparisons for significant main effects.

The accepted rule is use the MSerror term that was used to test the main effect.

If you want to use the HSD to make all pairwise comparisons in a 2-way interaction where the factors are both within subjects or both between subjects, then use the MSerror that was used to test the interaction.

Pairwise comparisons for mixed between- and within-subjects interactions

If you want to use the HSD to make all pairwise comparisons in a 2-way interaction where one of the factors is a between subjects and the other factor is within subjects then things become more complicated.  One might expect that the MSerror that was used to test the interaction would be appropriate. Dr. Klebe recommends using that approach.  Here are the values of the components of the HSD formula --

MSerror = 34.974,
v (df for the appropriate error term) = 74,
alpha = .05,
p (the number of means to be compared) = 4,
n = the harmonic mean of the n = 37.97.

Nh = 4 / (1/39 + 1/37 + 1/39 + 1/37) = 37.97

q (from the Studentized Range Statistic table) with alpha = .05, p (the number of means) = 4, and v (df for error) = 74  is 3.39.

The critical value for Tukey's HSD using the MSerror for the within subject effect is --

HSD = 3.39 * SQRT(34.974/37.97) = 3.25

If the difference between any pair of means is greater than or equal to 3.25 they are significantly different using this Tukey's HSD procedure. 

However that is not always the recommended procedure. Shavelson (1996) recommends using a pooled MSerror.  Pooling is accomplished by adding the Sums of Squares for the two error terms and dividing by the sum of the degrees of freedom for the two error terms --

MS_PW = (SS_{error.between} + SS_error.within) / (df_{error.between} + df_error.within)

In this example --

MS_PW = (2588.102 + 9414.874) / (74 + 74)
            = 12,002.976 / 148
            =  81.101

The harmonic mean of the ns = 37.97

Look up q with alpha = .05, p (the number of means) = 4, and v (df for error) = 74 + 74 = 148 in the Studentized Range table --

q = 3.68

Putting it all together the critical value for Tukey's HSD is --

HSD = 3.68 * SQRT(81.101/37.97) = 5.37.

If the difference between any pair of means is greater than or equal to 5.37 they are significantly different using Tukey's HSD procedure.  The pooling approach typically is much more conservative than the using the MSerror from the within subjects error term.

I suggest using a simple main effects analysis rather than making all pairwise comparisons using the Tukey's HSD.

top

10. References

Horowitz, M. J., Wilner, N., & Alvarez, W. (1979). Impact of event scale: A measure of subjective distress. Psychosomatic Medicine, 41, 209-218.

Shavelson, R. J. (1996). Statistical reasoning for the behavior sciences (3rd ed.). Boston: Allyn and Bacon.

Wilson, S. A., Becker, L. A., & Tinker, R. A. (1995). Eye movement desensitization and reprocessing (EMDR) treatment for psychologically traumatized individuals. Journal of Consulting and Clinical Psychology, 63, 928-937.

Wilson, S. A., Becker, L. A., & Tinker, R. A. (1997). Fifteen-month follow-up of eye movement desensitization and reprocessing (EMDR) treatment for posttraumatic stress disorder and psychological stress. Journal on Consulting and Clinical Psychology, 65, 1047-1056.

top

©Lee A. Becker, 1997-1999    -revised 12/01/99