Cruise Scientific   ¨   Visual Statistics Studio   ¨   Table of Contents    ¨   Library

Krus, D. J. & Ney, R. G. (1978) Convergent and discriminant validity in item analysis. Educational and Psychological Measurement, 38, 135-137.

CONVERGENT AND DISCRIMINANT VALIDITY IN ITEM ANALYSIS 

David J. Krus and Robert G. Ney
Arizona State University

 

Described is an algorithm for item analysis of multiple-choice tests in which item discrimination indices have been defined for item distractors in addition to their traditional definition for the scored alternative. Also, the Campbell and Fiske's concept or convergent and discriminant validity was reconccptualized from the test to the item level and proposed as a key concept for interpretation of results of item analysis of multiple-choice tests.

 

Traditionally, various discrimination indices in item analysis are computed for the scored alternative only. Computerization of item analytic procedures opens an interesting possibility: calculation of item discrimination coefficients also for item distractors. Inclusion of distractors into analysis extends its scope and opens new possibilities for the item analysis interpretation. Instead of judging the magnitude of the item discrimination index only, item-test correlation patterns can be evaluated for the scored alternatives together with their distractors. Several criteria can he used for this interpretation.

 

First, the discrimination index for the scored alternative should be significantly greater than zero and or practical significance. Second, the discrimination coefficient for the scored alternative should be high and positive (at least in achievement and ability measures), and the discrimination coefficients for the distractors should be high and negative. Also, this alternating pattern of high positive-high negative coefficients should be evenly distributed among all employed alternatives. With respect to the total test, to be internally consistent, all items should show the consistent pattern with respect to both criteria just out lined.

 

Obviously, these interpretative criteria bear strong resemblance to Campbell and Fiske’s (1959) concepts of convergent and Discriminant validity. One may conceptualize the scored alternatives as different “methods” of measuring a construct and the item distractors as “methods” designed to separate the construct under consideration from other constructs. The first interpretative criterion then becomes that of convergent validity, and the second criterion follows the logic of discriminant validation. The principal change is the reconceptualization of these concepts from the test to the item level.

 

Conceptualization of item analysis of multiple-choice tests

 

As shown in Table 1, item difficulty indices were computed as proportions of total response frequencies, assigned to each test alternative; the scored alternative was marked with an asterisk. Item discrimination indices were computed as point biserial correlations between each alternative and total score of the test; the asterisk marks the largest positive, nonzero value of he discrimination index.

 

Table 1. Conceptualization of Item Analysis for Multiple-Choice Tests

 

 

Item Difficulty: Scored alternative is highlighted.
Item Discrimination: Largest significant positive value of the discrimination
index coinciding with the scored alternative is highlighted
,
Largest positive value of the discrimination index,
not coinciding with the scored alternative or not significant
is marked by the magenta color,

 

Items one to three in the Table 1 show varying degrees of both convergent and discriminant validity, item four shows the lack of convergent validity; item five, the lack of discriminant validity; and item six, the lack of both convergent and discriminant validity.

 

REFERENCE

 

Campbell, D. T. and Fiske, D. W. (1959) Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin,  56, 81-105.

 

Cruise Scientific   ¨   Visual Statistics Studio   ¨   Table of Contents    ¨   Library