Cruise Scientific                                  Visual Statistics Studio                              Measurement and Scaling

  Krus, D. J. (2006) Reliability and homogeneity. Journal of Visual Statistics, 6, August, 2006.

Reliability and Homogeneity
David J. Krus
Arizona State University

Coefficients of reliability and homogeneity pertain to properties of data matrices within the framework of the theory of tests and measurements, where the coefficient of reliability indicates the degree to which a data matrix approximates an internally consistent scale and the coefficient of homogeneity indicates the degree to which a data matrix approximates a hierarchical scale. Note that in statistics, the word homogeneity is sometimes used as a synonym of homoscedasticity, however within the framework of theory of tests and measurements the word homogeneity has a meaning of its own.

Consider a scale X, for instance [0, 1, 2, 3] obtained as a row-wise composite of elements of a data matrix containing responses to a set of items by a group of subjects. This “subjects by items” matrix is a special case of data matrices containing numerical data about attributes (column marginal referents) and entities (row marginal referents) of some quantified phenomena. Let us assume that this “subjects by items” matrix consists of responses to three binary (yes-no, agree-disagree) items made by groups of subjects of indefinite size n and let us describe the abstract forms of this type matrices which define the limiting cases (0, 1) of the coefficients of reliability and homogeneity.

Idealized data structures of tautological relationships
The abstract data structure for three binary items, p, q, and r, for which the coefficient of reliability

 and the coefficient of homogeneity

 will both be equal to zero would have to contain all possible (eight,

, in general

 where k is the number of binary items) patterns of responses to the p, q, and r binary items by a fictional group of eight subjects (a through h). This k by

 matrix of all possible response patterns to the k number of items is called plenum, and for the discussed instance of items p, q, and r can be written as

 

A theoretical data structure, defined by catenated tautologies, can be obtained from the plenum by the logical function  rectified by the  function. The rectifying function  replicates the source response pattern if its value is 1 and omits the source response pattern if its value is 0. In the case of logical functions of tautologies the rectifying function will contain only the 1 (true) value and thus replicate the source plenum, as

 

The resulting data matrix is visualized in Figure 1.

Fig. 1. Idealized data structure containing only uncorrelated
tautological relationships.

The correlations between variables p, q, and r, are:

 

The coefficient of the internal consistency reliability and the coefficient of homogeneity for this theoretical data structure are equal to zero.

Idealized data structures of perfectly correlated relationships,
corresponding to logical functions of catenated equivalences

Analyzing the plenum of all possible responses for three binary items p, q, and r by using the rectified logical function  is shown in the template below

 

and the resulting structure is visualized in Figure 2.

 

Fig. 2. Abstract data structure obtained by logical analysis of a
plenum for three items by catenated logical functions of equivalence.

The above abstract data structure can be fitted into the 3-item plenum by repeated insertions of the a and h response patterns n/2 times, as within the discussed theoretical model there are no constraints on the size of the group of respondents, n, and there exists a formal proof that the internal consistency reliability is maximized by item variabilities approximating .25. This structure is outlined in the template below and visualized in Figure 3.

 

Fig. 3. Abstract structure of logical equivalencies fitted to a 3-item
plenum.

Scale based on measurements along the continuum of real numbers within the same confines is suggested as

and visualized in Figure 4.

Fig.4. Abstract data structure for perfectly correlated variables containing
real numbers.

Intercorrelations of these types of abstract data visualized in Figures 3 and 4 are

 

For the structures described above, the coefficient of the internal consistency reliability is one and the coefficient of homogeneity is less than one.

Idealized data structures corresponding to
catenated logical functions of implications

Analyzing the plenum of all possible responses for three binary items p, q, and r by the rectified logical function of catenated implications  

 

returns the data matrix visualized in Figure 5.

Fig. 5. Idealized data structure conformable to catenated
logical relationships of implications, characteristic of the
if-then hierarchical scales.

This data structure defines a hierarchical scale X as

 

The correlations between items p, q, and r are:   

 

The coefficient of the internal consistency reliability in this case is less than one and the coefficient of homogeneity equals one.

Tetrad criterion

Intercorrelations of items within the implicational data structures are compliant with the Spearman’s tetrad criterion. The Spearman's tetrad criterion, is characteristic of hierarchical uni-dimensional structures. The tetrad criterion is based on computations of products and differences of four (from Gr. prefix τετρα-, four) elements of correlation matrices. If the result equals zero or is close to zero, the tetrad criterion is met. For the above matrix of correlations, the tetrad criterion is satisfied, since .57(.57) -1.000(.33) = 0. Spearman tetrads are in fact the 2 x 2 minors of a matrix. In factor analysis, the number of common factors is one less than the order of the lowest-order minor that will vanish. In the case of implicatory scales, even minors with order equal to two (tetrads) vanish, thus these data structures are uni-dimensional. This explains the enduring controversy surrounding the Spearman’s (1927) original formulation of factor analysis, capturing the general factor of intelligence, g, and the subsequent reconceptualization of factor analysis by Thurstone (1947), and provides insight into the complex relationships between the order and factor analysis (Krus and Weiss, 1976; Krus and Kennedy, 1980).

Coefficients of homogeneity and reliability in historical perspective

The historical perspective on the coefficients of reliability and homogeneity is outlined in Table 1. The row A lists the original conceptualizations of these coefficients. The values of these coefficients frequently differ from the subsequent formulations of these coefficients, listed in rows B, C, and D.

Table 1. Coefficients of reliability and homogeneity
in historical perspective.

 

The theoretical concept of the internal consistency reliability was introduced by Spearman and Brown in 1910. Spearman’s manuscript arrived at the editorial office of the British Journal of Psychology by the morning mail; Brown’s manuscript arrived the same day by the afternoon mail, thus this theoretical conceptualization of the internal consistency reliability is abbreviated as the S-B formula. The K-R-20 formula is defined for binary (dichotomous) data; Cronbach’s alpha is defined for both the dichotomous and for the continuous data. The Hoyt-Jackson coefficient of the internal consistency reliability was formulated within the framework of the analysis of variance. The coefficient of homogeneity was conceptualized by Louis Guttman and formalized by Jane Loevinger. Interest in this index was revived during the closing decades of the last century by Norman Cliff within the context of the ordinal test theory.

The algebraic formulae for computation of coefficients of reliability and homogeneity in the row D were formulated within the framework of the analysis of variance and thus MS stands for the mean square, R for the row component of variance, and I for the variance due to interactions among the off-row, off-columns elements of the data matrix analyzed. The max superscript indicates that the mean squares were obtained from the data matrix where the variance of the variables was maximized. Detailed information about the coefficients of reliability listed in Table 1 can be obtained from publications by Spearman (1910), Brown (1910), Kuder and Richardson (1937), Cronbach (1951), and Hoyt (1941). Formulations of the coefficient of homogeneity are described in publications by Guttman (1944), Loevinger (1948), Cliff (1977) and Krus and Blackman (1988).

References

Brown, W. (1910) Some experimental results in the correlation of mental abilities. British Journal of Psychology, 3, 296-322.

Hoyt, C. (1941) Test reliability estimated by analysis of variance. Psychometrika, 6, 153-160.

Cliff, N. (1977) A theory of consistency of ordering generalizable to tailored testing. Psychometrika, 42, 375-399.

Cronbach, L. J. (1951) Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-333.

Guttman, L. (1944) The Cornell technique for scale and intensity analysis. Educational and Psychological Measurement, 7, 2, 247280.

Kuder, G. & Richardson, M. (1937) The theory of estimation of test reliability. Psychometrika, 2, 151-160.

Krus, D.J., & Blackman, H.S. (1988) Test reliability and homogeneity from perspective of the ordinal test theory. Applied Measurement in Education, 1, 79-88 .

Krus, D. J. & Kennedy, P. H. (1980) Dimensionality of hierarchical and proximal data structures. Applied Psychological Measurement, 4, 313-321.

Krus, D. J. & Weiss, D. J. (1976) Empirical comparison of factor and order analysis on pre-structured and random data. Multivariate Behavioral Research, 11, pp. 95-104.

Loevinger, J. (1948) The technic of homogeneous tests compared with some aspects of scale analysis and factor analysis. Psychological Bulletin, 45, 507-529.

Spearman, C. (1910) Correlation calculated from faulty data. British Journal of Psychology, 3, 271-295.

Spearman, C. (1927) The abilities of man: their nature and measurement. New York: Macmillan.

Thurstone, L. (1947) Multiple factor analysis. Chicago: The University of Chicago Press.

See also

Elements of propositional calculus.

 

Illustration of theory of true and error scores on idealized data sets.