Analysis of Covariance

Nine subjects are randomly assigned to one of three groups instructed by three teaching methods. The covariate is a measure of math aptitude obtained before the experiment begins. The dependent variable is a measure of math achievement obtained after the experiment is completed.

• Independent variable: Method with three levels (There are three independent groups.)

• Covariate: Math aptitude test

• Dependent variable:  Math achievement test

Can we reasonably attribute the group difference in math achievement to teaching methods? If not, how to separate the effect due to the initial difference in aptitude scores from the experimental effect?

To control the source of variation due to the aptitude scores, the effect of the covariate will be removed from the achievement scores by using the regression method. Then the F test can be performed on the adjusted achievement scores.

SPSS for Windows

Create three variables: method, apt, and ach. Enter data.

Test  the Assumption of Equal Slopes

The dependent variable is ach. The covariate is apt. The independent variable (factor) is method.

Before conducting an ANCOVA, we need to test the assumption of equal slopes first. The ANCOVA assumes a linear relationship between the covariate and the dependent variable and there is no interaction between the covariate and treatments.

 Research Questions

Is there a strong linear relationship between the achievement and aptitude scores within all three teaching methods? Are the slopes of the lines relating achievement and aptitude the same for the three methods? (Is there a significant method-by-aptitude interaction? )

 

1. Choose  Analysis \ General Linear Model \ Univariate.

Select ach and move it to the Dependent Variable box.

Select method and move it to the Fixed Factor(s) box.

Select apt and move it to the Covariate(s) box.

2. Next, click Options to display descriptive statistics and tests.

Select method in the Factor(s) and Factor Interactions box and move it to the Display Means for box.

In the Display box, select Descriptive statistics , Estimates of effect size and homogeneity tests as shown below. Click Continue.

3. Third, click the Model button. Click Custom.

In the Factors & Covariates box, select method(F) and move it to the Model box.

In the Factors & Covariates box, select apt(C) and move it to the Model box.

In the Factors & Covariates box, select both method(F) and apt(C) by holding down the Ctrl key. Note that the option Interaction is shown in the Build Term(s) box.

Click the right arrow to move the interaction term to the Model box. Click Continue and OK.

   

SPSS outputs

Descriptive Statistics

Examine the interaction term: METHOD*APT.

 

 

Conclusion

The interaction is not significant, F(2,3) = .177, p = .846, partial eta square = .105.
 

Data Visualization

 Produce a scatterplot of achievement and aptitude scores for three methods.

1. From the menus choose: Graphs \ Scatter

2. Choose the default Simple picture button.

First, click on Define.
Next, move ach to the Y Axis box and move apt to the X Axis box.
Highlight the variable `method`. Move it to the Set Markers by box. Thus, each method has a different color or marker symbol on the scatterplot.

3. Modify the chart to produce three linear regression lines.

Double-click on the scatterplot. The SPSS Chart Editor window will appear.

Instructions for SPSS 11.5

From the Chart Editor Window choose: Chart \ Options.

Click on the Subgroups check box. Click on Fit Options.
Fit Method. Linear Regression is the default method.

In the Regression Options box, click the Display R-square in legend check box.
Click Continue. Click OK.
To add an title, choose Chart \ Title. Type in an appropriate title and click OK. Close the Chart Editor window.
 

Instructions for SPSS 12.0

(1) Double-click on the scatterplot. The SPSS Chart Editor window will appear.

(2) Click on one of the data dots in blue (method 1). All the blue data dots will be highlighted.

(3) From the Chart Editor window menus choose: Chart/Add Chart Element/Fit  Line at Total.

(4) Click on on of the data dots in green. All the green data dots (Method 2) will be highlighted.

(5) From the Chart Editor window menus choose: Chart/Add Chart Element/Fit  Line at Total.

(6)  Click on on of the data dots in red. All the red data dots (Method 3) will be highlighted.

(7) From the Chart Editor window menus choose: Chart/Add Chart Element/Fit  Line at Total.

SPSS Output

The scatterplot of aptitude against achievement scores for three methods can be shown below

The covariate (aptitude scores) is linearly related to the dependent variable (achievement scores) within all levels of the teaching methods. The assumption of equal slopes was met and we could proceed to conduct a one-way ANCOVA.

One-Way ANCOVA

Differences in achievement scores are attributable not only to differences among the teaching methods but also to the initial differences in aptitude scores. In order to control this source of variability (the covariate variable), the statistical technique - analysis of covariance can be used. Analysis of covariance adjusts achievement scores based on aptitude scores (the covariate).
 

1. Choose Analyze \ General Linear Model \ Univariate. Note that we have selected one dependent variable, one fixed factor, and one covariate in the previous section.

2. Click Model. Select Full Factorial.

Click Continue and OK. 

SPSS Output

1. The three adjusted means are reported as 5.209, 8.684, and 10.107. 

2. Test the differences among the three adjusted means.
 

Examine the METHOD source.

The results of the ANCOVA indicated that there were significant differences among the three adjusted means, F(2,5) = 54.288, p < .001, and the partial eta square of .956 suggested a strong relationship between achievement scores and teaching methods, controlling for aptitude scores.

Post Hoc Pairwise Comparisons

Since there were significant differences among the three adjusted means, we may proceed to conduct post hoc pairwise comparisons. 

1. Choose Analyze \ General Linear Model \ Univariate. Note that we have selected one dependent variable, one fixed factor, and one covariate in the previous section.

2. Click Paste and the syntax will appear in the Syntax Editor window. 

3. Delete the following lines

4. Specify three sets of coefficients to compare the adjusted means for Methods 1 and 2, Methods 1 and 3, Methods 2 and 3.

 
Identify specific comparisons of interest and assign values to be directly compared. 

Examine the above contrasts. Three pairwise mean differences are  -3.47,  -4.90, and -1.42.

Contrast #1 and Contrast #2 might be significant since the mean differences are larger.

5. Type three /LMATRIX commands. 

6. Click Run and select ALL.

SPSS Output

Examine the results.

Contrast #2

Contrast #3

There are three pairwise comparisons. The Bonferroni procedure was used to control for type I error across three comparisons. .05 / 3 = .0167.

In this example, two comparisons are significant.

There were significant differences in the adjusted means between Method 1 and Method 2, F(1,5) = 71.205, p < .0167.

There were significant differences in the adjusted means between Method 1 and Method 3, F(1,5) = 90.952, p < .0167.

There were no significant differences in the adjusted means between Method 2 and Method 3, F(1,5) = 9.215, p > .0167.