Variance and differences between values of a variable

Cruise Scientific Visual Statistics Studio Measurement and Scaling

Krus, D. J . (2006) Variance and the differences between values of a variable. Journal of Visual Statistics at VisualStatistics.net (August 3, 2006).

Variance and the differences between values of a variable
David J. Krus
Arizona State University

Variance of a variable
can be computed by subtracting the mean M from the values of the variable X, and squaring and averaging these values, as

where n is the number of values of the variable X. For instance of a variable X [1 2 3 4 5]

the variance equals 1.25.

Matrices of differences

Using matrix algebra, variance can be measured by computing all possible differences between elements of a population. Consider that a major difference of a vector results in a skew-symmetric matrix with elements describing all possible differences between its values.

Skew symmetric matrices are redundant, as the negative values can be guessed from the symmetric positive values. Removing this redundancy, a skew-asymmetric matrix can be defined as

Variance can be obtained from the sum of the squared elements of skew-asymmetric matrices divided by the square of their order. Thus

and for the example (1² + 2² +1² + 3² + 2² + 1² ) / 4² = 1.25.

Differences between data elements and their mean

The above definition of variance in terms of differences contained by the data does not involve the arithmetic mean. It seems plausible to assume that the information contained in the above matrix could have been also obtained from a matrix of all possible differences between the data elements and a scalar vector M

which might be as well written as

since the variance can be computed either as

(for the example as equal to 1.25) or as

For the example

Thus, using the matrix algebra notation, variance can be computed as an index of differences between elements of a variable

or, by using algebraic notation, as an average of the squared differences between values of a variable and their mean

These observations facilitate the understanding the definition of variance as an index of the magnitude of differences between the values of a variable.

Variance and Information

Transformation of the vector x into its adjacent implicatic matrix as

and subtraction of the transpose of this matrix from itself

results in the same skew symmetric matrix as that of the major difference of the vector x.

Transformation of the above matrix from its skew symmetric to the skew asymmetric form,

can provide information about the number of bits contained within the columns of the data vector, as defined within the mathematical theory of information.

Retrospect

The concept of variance in terms of all possible differences between values of a variable was introduced by von Andrae (1872) and Helmert (1876) in a series of articles to Astronomische Nachtrichten and the convention to use the Greek lowercase character σ for the standard deviation was coined by Karl Pearson in a series of articles published in Philosophical Transactions and Biometrika between 1896 and 1906. Ronald Fisher can be credited with popularization of degrees of freedom and using the expression variance in lieu of that of variability, but not with the analytical conceptualization of variante eventum. That was done nearly a century earlier within the framework of astronomical observations.

Using all possible differences between values of a variable as a foundation of statistical theory was contemplated by Kendall (1943, p. 47) who defined a coefficient, called here u, as

$\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} (x-y)^2\, dF(x) dF(y)$

For the discontinuous infinite case, the above equation can be written as

$u^2 =\frac{\sum_{i=-\infty}^{\infty} \sum_{i=-\infty}^{\infty} (x_i - x_j)^2 f(x_i) f(x_j)}{n^2}$

and for the finite case as

$u^2 =\frac{\sum_{i=1}^{n^2} x^2_{\Delta_i}}{n^2}$

where the summed term in the above equation is a vector of all possible differences between elements of variable x. Pointing out that the value of the u coefficient is dependent on the spread of the variate-values among themselves and not on the deviations from some central value, Kendall (1943, p.47) shows that u = 2σ, concludes that the initial defining formula is nothing but twice the variance, and abandons the idea. One can only wonder which direction statistics could have taken if Kendall would have realized that matrices of differences between all values of a variable are not just another way to compute variance, but reflect also the information content of the variable and the hierarchical relationships between its elements.

References

Andrae, von (1872). Über die Bestimmung des wahrscheinlichen Fehlers durch die gegebenen Differenzen vom gleich genauen Beobachtungen einer Unbekannten. Astronomische Nachrichten, vol. 84.

Helmert, F.R. (1876). Die Berechnung des wahrscheinlichen Beobachtungsfehlers aus den ersten Potenzen der Differenzen gleichgenauer directer Beobachtungen. Astronomische Nachrichten, vol. 88.

Kendall, M. (1943). The Advanced Theory of Statistics. In Stuart, A., & Ord, J.K. (1987) Kendall’s Advanced Theory of Statistics, 5th Ed. London: Griffin.
Press, W. H., Teukolsky, S.A., Vetterling, W.T., & Flannery, B.P. (1992, 2^nd Ed.). Numerical Recipes. Cambridge, MA: Cambridge University Press.

Krus, D.J., & Ceurvorst, R.W. (1979) Dominance, information, and hierarchical scaling of variance space. Applied Psychological Measurement, 3, 515-527.

Shannon. C. E., & Weaver, W. (1949) The mathematical theory of communication. Urbana: University of Illinois Press

Note

The unbiased variance can be expressed in the matrix algebra notation as

and in the algebraic notation as