Cruise Scientific                Visual Statistics Studio                Visual Statistics Studio Guide

Reliability and Homogeneity

Reliability Analysis

Select (Designs, Measurement and Scaling, Reliability Analysis) to obtain data for this example.

Select (Analysis III, Reliability Analysis)

for the table of results shown below:

The obtained reliability can be optionally stored in one (1 2 3 4 5 6 7 8 9) of the cells of the Scalar module by selecting a memory cell to the right of the Transfer rxx heading..

Homogeneity Analysis

Homogeneity analysis of the same data (Analysis III, Homogeneity Analysis)

returns the index of internal consistency homogeneity equal to .838 which, again, can be stored in one of the nine memories of the Scalar module by selecting a memory cell to the right of the Transfer hxx heading.

Theory of True and Error Scores

Basic postulates of the theory of true and error scores are

that obtained (XO) scores consists of the true (XT) and error (XE) scores, that their means and variances are additive, that the internal consistency reliability is defined as the proportion of true variance in the obtained test scores and that can be estimated by the Spearman - Brown formula from a test split into two halves.

To demonstrate these postulates, select  (Designs, Measurement and Scaling, Hypothetical Components of the Obtained Scores) and click on the Accept command.

True Scores consists from the True Components and  Obtained Scores consists from the True and Error Components. To obtain True Scores, add two True Components (multiply the true component by 2). To obtain Obtained Scores, add the True and Error Component. Select (Operations, Multiply by a Constant), enter 2 into the Constant input box, Name the product True, mark True Comp and click the Append command. Select (Operations, Add Variables), Name the Sum Obtained, mark True Comp and Obt Comp, and click the Accept command.
 
 

Select (Graphs, Multiple Graphs), mark True and Obtained, and click on the Accept command. Select (Gallery, Cube Graphs), Click on the 3-D command, and red-color the error component.

 

At this point we could compute the internal consistency reliability (.992) as the ratio of variances of true (125) and obtained (126) scores

however the problem remains how to to compute the internal consistency reliability from real data sets. The answer to this problem was provided by Spearman and Brown who suggested that test scores should be split into two halves and that correlation between these two halves can be used for estimation of internal consistency reliability by their reliability formula. To demonstrate computation of Spearman - Brown reliability, select (Project, Parallel Project)

 and click the Open a New Project command. Select (Designs, Measurement and Scaling, Hypothetical Components of a Split-Half Test) and click the Accept command.

 

To evenly distribute the true and obtained components into each half of the split test, the true and error components have to be multiplied by the C1 and C2 coding vectors as (C1 * TComp) (C2 * OComp) and (C1 * OComp) (C2 * TComp). The (C1 * TComp) and  (C2 * OComp) have to be added to get the first (H1) half of the test scores. The (C1 * OComp) (C2 * TComp) have to be added to get the second (H2) half of the test scores.

The correlation (Analysis I, Correlation) between halves H1 and H2 of the split test is .984. Substituting this value into the Spearman - Brown reliability formula,

the internal consistency reliability can be estimated as .992.

Delete (Data, Delete Variables) all but the last two variables from the second project and compact the display. Add the H1 and H2 variables (Operations, Add Variables) to confirm that the halves of split test add to the obtained scores

with mean (25) equal to sum of the means of each half (12.5 + 12.5) and variance (126) equal to sum of the variances of each half (31.75 + 31.75 = 63.5) plus two times the covariance between the H1 and H2 ( 2 * 31.25 = 62.5).

Delete (Data, Delete Variables) all but the last two variables from the second project and compact the display. Select (Analysis I, Regression Analysis) and associate the Predictor Variable with True scores and the Criterion Variable with the Obtained scores. Click the Accept command.

to observe that the Regression Analysis can split the Obtained scores into its True (Obtained ') and Error (Obtained ^ ) components.

Click on (Descriptive Statistics, Standardize Variance, Select the Standardizing Variable Obtained) and compare the coefficients of Determination (.992) and Alienation (.008)

 

with the standard variance components

 

of the predicted (.992) and error (.008) variables. Optionally, select (Designs, Measurement and Scaling, Hypothetical Model of Theory of True and Error Scores)

to summarize the basic postulates of the theory of true and error scores.