Variance and differences between values of a variable

Cruise Scientific Visual Statistics Studio Measurement and Scaling

Krus, D. J . (2006) Variance and the differences between values of a variable. Journal of Visual Statistics at VisualStatistics.net (August 3, 2006).

Variance and the differences between values of a variable
David J. Krus
Arizona State University

Variance of a variable X,
can be computed by subtracting the mean M from the values of the variable X, and squaring and averaging these values, as

where n is the number of values of the variable X. For instance, for variable X [1 2 3 4 5]

the variance equals 1.25.

Matrices of differences

Using matrix algebra, variance can be measured by computing all possible differences between elements of a variable. A major difference of a variable X with its transpose results in a skew-symmetric matrix. Its elements describe all possible differences between values of the variable X.

Skew symmetric matrices are redundant, as the negative values can be guessed from the symmetric positive values. Removing this redundancy, a skew-asymmetric matrix can be defined as

Variance can be obtained from the sum of the squared elements of a skew-asymmetric matrix divided by the square of its order.

For the example (1² + 2² +1² + 3² + 2² + 1² ) / 4² = 1.25.

Differences between data elements and their mean

Before the computer era, computing a skew-asymmetric matrix of differences was an arduous task. To simplify it, let us consider what happens if instead of computing a major difference of a variable from its transpose we compute the major difference of a variable from its mean.

In the above matrix, the X - M difference is repeated n times in every row. This repetition is redundant and we can write

The result of the above operation is a matrix of deviation scores x, since

Thus

Since the sum of the deviation scores is always zero, a meaningful index can be obtained by the averaging the sum of its squares, for the example 5 / 4 which equals to 1.25, a value the same as obtained above. Squaring the scores is congruent with the criterion of least squares, and thus these two concepts form the basis of the general linear model of statistics.

Variance and Information

Transformation of the variable X into its adjacent binary matrix, for our example of the variable X as

and subtraction of the transpose of this matrix from itself

results in the same skew symmetric matrix as that of the major difference of the transpose of variable X from itself. Triangulation of the above matrix from its skew symmetric to its skew asymmetric form,

provides information about the number of 0-1 changes (bits) contained within the data.

Retrospect

The concept of least squares was developed by Laplace (1749-1827) in his work explaining the differences in motion of Jupiter and Saturn. The concept of variance in terms of all possible differences between values of a variable was introduced by von Andrae (1872) and Helmert (1876) in a series of articles to Astronomische Nachtrichten. The convention to use the Greek lowercase character σ for the standard deviation was coined by Karl Pearson in a series of articles published in Philosophical Transactions and Biometrika between 1896 and 1906.

Using all possible differences between values of a variable as a foundation of statistical theory was contemplated by Kendall (1943, p. 47) who defined a coefficient, called here u, as

$\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} (x-y)^2\, dF(x) dF(y)$

For the discontinuous infinite case, the above equation can be written as

$u^2 =\frac{\sum_{i=-\infty}^{\infty} \sum_{i=-\infty}^{\infty} (x_i - x_j)^2 f(x_i) f(x_j)}{n^2}$

and for the finite case as

$u^2 =\frac{\sum_{i=1}^{n^2} x^2_{\Delta_i}}{n^2}$

where the summed term in the above equation is a vector of all possible differences between elements of variable x. Pointing out that the value of the u coefficient is dependent on the spread of the variate-values among themselves and not on the deviations from some central value, Kendall (1943, p.47) shows that u = 2σ, concludes that the initial defining formula is nothing but twice the variance, and abandons the idea. One can only wonder which direction statistics could have taken if Kendall would have realized that matrices of differences between all values of a variable are not just another way to compute variance, but reflect also the information content of the variable and the hierarchical relationships between its elements.

References

Andrae, von (1872). Über die Bestimmung des wahrscheinlichen Fehlers durch die gegebenen Differenzen vom gleich genauen Beobachtungen einer Unbekannten. Astronomische Nachrichten, vol. 84.

Helmert, F.R. (1876). Die Berechnung des wahrscheinlichen Beobachtungsfehlers aus den ersten Potenzen der Differenzen gleichgenauer directer Beobachtungen. Astronomische Nachrichten, vol. 88.

Kendall, M. (1943). The Advanced Theory of Statistics. In Stuart, A., & Ord, J.K. (1987) Kendall’s Advanced Theory of Statistics, 5th Ed. London: Griffin.
Press, W. H., Teukolsky, S.A., Vetterling, W.T., & Flannery, B.P. (1992, 2^nd Ed.). Numerical Recipes. Cambridge, MA: Cambridge University Press.

Krus, D.J., & Ceurvorst, R.W. (1979) Dominance, information, and hierarchical scaling of variance space. Applied Psychological Measurement, 3, 515-527.

Shannon. C. E., & Weaver, W. (1949) The mathematical theory of communication. Urbana: University of Illinois Press

Note

The unbiased variance can be expressed in the matrix algebra notation as

and in the algebraic notation as