Cruise Scientific   ¨   Visual Statistics Studio   ¨   Foundations of Visual Statistics

  Krus, D. J. & Kennedy, P. H. (1977) Normal scaling of dominance matrices: The domain-referenced model. Educational and Psychological Measurement, 37, 189-193.

NORMAL SCALING OF DOMINANCE MATRICES: THE DOMAIN REFERENCED MODEL
David J. Krus and Patricia H. Kennedy
Arizona State University

 

Described is an alternative algorithm for Thurstone’s pair comparisons method. This algorithm provides an improved capacity for handling indeterminate proportions of 1.00 and .0.00 as well as estimates of population scale values that are not truncated.

Samuelson (1968, pp.1522-1523) demonstrates that when the true variance is used in calculation of the z values, then the upper limit for the maximum value of z is . This observation suggests that n should play a role in scaling algorithms using the transformations into the standard scores, as e.g., in the Thurstone’s method of scaling the pair comparisons.

Thurstone's method for scaling data obtained from pair comparisons of stimuli can be considered a prototype of a normal distribution based method for scaling-dominance matrices. Even though the theory behind this method is quite complex (Thurstone, 1927a), the algorithm itself is straightforward. For the basic Case V, the frequency dominance matrix is translated into proportions and interfaced with the standard scores. The scale is then obtained as a left-adjusted column marginal average of this standard score matrix (Thurstone, I 927b).

The principal difficulty with this algorithm is its indeterminacy with respect to one-zero proportions, which return z values as plus or minus infinity, respectively. The inability of the pair comparisons algorithm to handle these cases imposes considerable limits on the applicability of the method. The most frequent recourse when the 1.00-0.00 frequencies are encountered is their omission. Thus, e.g., Guilford (1954, p. 163) has recommended not using proportions more extreme than .977 or .023, and Edwards (1957, pp. 41-42) has suggested that

“if the number of judges is large, say 200 or more, then we might use p_ij values of .99 and .01, but with less than 200 judges, it is probably better to disregard all comparative judgments for which p_ij is greater than .98 or less than .02."’

Since the omission of such extreme values leaves empty cells in the Z matrix, the averaging procedure for arriving at the scale values cannot be applied, and an elaborate procedure for the estimation of unknown parameters is usually employed (Edwards, 1957, pp. 42-46).

An alternative solution can be suggested as follows. Consider the following dominance matrix, generated by five subjects judging the palatability of peach brandy versus rum raisin ice cream, as shown in Table 1.

Table 1. A Dominance Matrix of Frequencies

 

Application of Thurstone’s Case V algorithm to this dominance matrix would necessitate the conversion of this frequency matrix into a matrix of proportions with adjusted diagonals (Table 2)

Table 2.  A Dominance Matrix of Proportions

 

and translation of these proportions into z scores through using standard normal distribution tables. Unfortunately, for the example, the z score for both brands of ice cream happens to be infinity, which would stretch the scale a bit. On the other hand, application of Edwards’ (1957) algorithm (i.e., deleting the one-zero proportions) would not leave any scale at all. A possible solution to this dilemma is to estimate the population’s values of the z scores from the sample data, by using

 

(1)

 

for the matrices of frequencies, or

 

(2)

 

for the matrices of proportions. In both formulae, b and c stand for the symmetric off-diagonal entries of a dominance matrix. For the example, the matrix of z scores was computed from the frequencies dominance matrix and entered to the off-diagonal elements of Table 3 by using formula (1) as

 

 

Table 3. A Dominance Matrix in Standard Scores

Identical results could have been obtained by applying formula (2) to the matrix of proportions; i.e.,

 

 

To compare the efficiency of the domain-referenced algorithm with the extant pair comparisons model, data obtained by using the method of pair comparisons to develop an attitude scale (Edwards, l957_, p. 42, Table 2.10) and reproduced here in Table 4 were reanalyzed.

Table 4. Data Matrix of Proportions for a Study Involving Scaling of Nine Stimuli

 

The adjusted z-scale obtained by the domain-referenced algorithm is listed in the last column of Table 5. Its corresponding scale, computed by Edwards’ modification of the pair comparisons’ algorithm, is reproduced in the third column of the same table (Edwards, 1957, p. 45, Table 2.12). Both scales were adjusted by setting the lowest scale value to zero to avoid negative scale values. However, this adjustment is optional and transformations to different scale forms are often used.
 

Table 5. Comparison of the Pair Comparison and the Domain-Referenced Models for
 Scaling of One-Dimensional Dominance Matrices

The Pair Comparison Scales are reproduced from Edwards (1957. p.43. Table 2.21)

The obtained scales were plotted in z-scores in the Figure 1 where the scale obtained by using the pair-comparison method is shown by using the light shaded bars and the scale obtained by the domain-referenced method is shown by using the darker shades of gray.

Fig. 1. Comparison of the pair-comparison and the domain-referenced methods.

Comparison of the non-adjusted scales for both solutions, especially in terms of the end points of -1.52z and +1.52z for the Edwards’ scale and -7.30z and +5.73z for the domain-referenced scale reveals a marked truncation of the scale values returned by Edwards’ algorithm. As two entries of the analyzed dominance matrix were equal to one and 33% of the dominant entries were greater than .9357, the -1.52z and +1.52z end points of the Edwards’ scale are not likely to be of the correct magnitude. Also, the markedly larger spread of the scale values, obtained by the domain-referenced model, allows for finer discriminations between scaled stimuli.

Irrespective of the differences in observed magnitudes, the relative order of stimuli, as returned by both models, was identical. However, by using the domain-referenced model, extreme proportions do not have to be deleted. Also, the number of observations is included as a parameter of the scaling algorithm and no assumptions about the values of the missing parameters are necessary.

References

Edwards, A. L. (1957) Techniques of attitude scale construction. New York: Appleton-Century- Crofts.

Guilford, J. P. (1954) Psychometric methods. New York: McGraw-Hill.

Samuelson, P.A. (1968) How deviant can you be? American Statistical Association Journal, 63, 1522-1525.

Thurstone, L. L. (1927a) A Law of comparative judgment. Psychological Review, 34, 273-286.

Thurstone, L. L. (1927b) The method of paired comparisons for social values. Journal of Abnormal and Social Psychology. ,21, 384-400.

Cruise Scientific   ¨   Visual Statistics Studio   ¨   Foundations of Visual Statistics