## The Standard Error of Regressions

Journal of Economic Literature

Vol. XXXIV (March 1996), pp. 97114

The Standard Error of Regressions

By DEIRDRE N. MCCLOSKEY

and

STEPHEN T. ZILIAK

University of Iowa*Below is a small part of the first part of the article*

THE IDEA OF statistical significance is

old, as old as Cicero writing on forecasts

(Cicero, De Divinatione, I. xiii. 23).

In 1773 Laplace used it to test whether

comets came from outside the solar system

(Elizabeth Scott 1953, p. 20). The

first use of the very word significance

in a statistical context seems to be John

Venns, in 1888, speaking of differences

expressed in units of probable error:

They inform us which of the differences in

the above tables are permanent and significant,

in the sense that we may be tolerably

confident that if we took another similar

batch we should find a similar difference; and

which are merely transient and insignificant,

in the sense that another similar batch is

about as likely as not to reverse the conclusion

we have obtained. (Venn, quoted in Lancelot

Hogben 1968, p. 325).

Statistical significance has been much

used since Venn, and especially since

Ronald Fisher.

The problem, and our main point, is

that a difference can be permanent (as

Venn put it) without being significant

in other senses, such as for science or

policy. And a difference can be significant for science or policy and yet be insignificant

statistically, ignored by the

less thoughtful researchers

.

In the 1930s Jerzy Neyman and Egon

S. Pearson, and then more explicitly

Abraham Wald, argued that actual investigations

should depend on substantive

not merely statistical significance. In

1933 Neyman and Pearson wrote of type

I and type II errors:

Is it more serious to convict an innocent man

or to acquit a guilty? That will depend on the

consequences of the error; is the punishment

death or fine; what is the danger to the community

of released criminals; what are the

current ethical views on punishment? From

the point of view of mathematical theory all

that we can do is to show how the risk of

errors may be controlled and minimised. The

use of these statistical tools in any given case,

in determining just how the balance should

be struck, must be left to the investigator.

(Neyman and Pearson 1933, p. 296; italics

supplied)

Wald went further:

The question as to how the form of the

weight [that is, loss] function . . . should be

determined, is not a mathematical or statistical

one. The statistician who wants to test certain hypotheses must first determine the relative importance of all possible errors, *which will
depend on the special purposes of his investigation*

…………………………………….

70 percent of the empirical papers in the *American Economic Review* papers

did not distinguish statistical significance from economic, policy, or scientific significance.

…………………………………….

69 percent did not report descriptive statistics — the means of the regression variables, for example —

that would allow the reader to make a judgment about the economic significance of the results.

**{The authors also note a pattern of multiple author papers being a bit sloppier than single author papers}**

*summarized by Sherman Hanna*