The Standard Error of Regressions

The Standard Error of Regressions

The Standard Error of Regressions

Journal of Economic Literature
Vol. XXXIV (March 1996), pp. 97–114
The Standard Error of Regressions
By DEIRDRE N. MCCLOSKEY
and
STEPHEN T. ZILIAK
University of Iowa
Below is a small part of the first part of the article

THE IDEA OF statistical significance is
old, as old as Cicero writing on forecasts
(Cicero, De Divinatione, I. xiii. 23).
In 1773 Laplace used it to test whether
comets came from outside the solar system
(Elizabeth Scott 1953, p. 20). The
first use of the very word “significance”
in a statistical context seems to be John
Venn’s, in 1888, speaking of differences
expressed in units of probable error:
They inform us which of the differences in
the above tables are permanent and significant,
in the sense that we may be tolerably
confident that if we took another similar
batch we should find a similar difference; and
which are merely transient and insignificant,
in the sense that another similar batch is
about as likely as not to reverse the conclusion
we have obtained. (Venn, quoted in Lancelot
Hogben 1968, p. 325).
Statistical significance has been much
used since Venn, and especially since
Ronald Fisher.

The problem, and our main point, is
that a difference can be permanent (as
Venn put it) without being “significant”
in other senses, such as for science or
policy. And a difference can be significant for science or policy and yet be insignificant
statistically, ignored by the
less thoughtful researchers

.
In the 1930s Jerzy Neyman and Egon
S. Pearson, and then more explicitly
Abraham Wald, argued that actual investigations
should depend on substantive
not merely statistical significance. In
1933 Neyman and Pearson wrote of type
I and type II errors:

Is it more serious to convict an innocent man
or to acquit a guilty? That will depend on the
consequences of the error; is the punishment
death or fine; what is the danger to the community
of released criminals; what are the
current ethical views on punishment? From
the point of view of mathematical theory all
that we can do is to show how the risk of
errors may be controlled and minimised. The
use of these statistical tools in any given case,
in determining just how the balance should
be struck, must be left to the investigator.
(Neyman and Pearson 1933, p. 296; italics
supplied)
Wald went further:
The question as to how the form of the
weight [that is, loss] function . . . should be
determined, is not a mathematical or statistical
one. The statistician who wants to test certain hypotheses must first determine the relative importance of all possible errors, which will
depend on the special purposes of his investigation

…………………………………….

70 percent of the empirical papers in the American Economic Review papers
did not distinguish statistical significance from economic, policy, or scientific significance.

…………………………………….

69 percent did not report descriptive statistics — the means of the regression variables, for example —
that would allow the reader to make a judgment about the economic significance of the results.

{The authors also note a pattern of multiple author papers being a bit sloppier than single author papers}


summarized by Sherman Hanna


Return to SCF Data Section Home Page

Return to SCF Research Group Home Page