The science or alchemy debate

After 1950 econometrics became a mature field and the Cowles Commission (henceforth, CC) approach was the dominant practice. But after two decades, 1950s and 1960s, of high expectations of econometrics as producer of reliable predictions and policy advice, in the 1970s these expectations were increasingly doubted.

Hendry labelled Keynes’s list of concerns as “problems of the linear regression model”, which, according to him, consisted of using an incomplete set of determining factors (omitted variables bias); building models with unobservable variables (such as expectations), estimated from badly measured data based on index numbers; obtaining spurious correlations from the use of proxy variables and simultaneity, being unable to separate the distinct effects of multicollinear variables; assuming linear functional forms not knowing the appropriate dimensions of the regressors; misspecifying the dynamic reactions and lag lengths; incorrectly pre-filtering the data; invalidly inferring causes from correlations; predicting inaccurately (non-constant parameters); confusing statistical with economic significance of results; and failing to relate economic theory to econometrics. To Keynes’s list of problems, he added stochastic misspecification, incor­rect exogeneity assumptions, inadequate sample size, aggregation, lack of structural identification, and an inability to refer back uniquely from observed empirical results to any given initial theory.

Hendry admitted that “it is difficult to provide a convincing case for the defence against Keynes’s accusation almost 40 years ago that econometrics is statistical alchemy since many of his criticisms remain apposite” (1980: 402, original emphasis).

Hendry referred to Keynes’s use of the term alchemy to discuss the scientific nature of econometrics. Another way to denote this discussion is to see how much economet­rics differs from “economic tricks” (or “econo-mystics” or “icon-ometrics”; see Hendry 1980: 388). This was the kind of denotation Edward Leamer (1983) used to contribute to the debate about the scientific character of econometrics under the title “Let’s take the con out of econometrics”.

Leamer’s paper is very much about the “myth” of science that empirical research is (randomized controlled) experimentation and scientific inference is objective and free of personal prejudice. The problem of nonexperimental settings (usually the case in eco­nomics) compared with experimental settings (“routinely done” in science) is that “the misspecification uncertainty in many experimental settings may be so small that it is well approximated by zero. This can very rarely be said in nonexperimental settings” (Leamer 1983: 33). Protecting the image that econometrics is like agricultural experimentation (randomized controlled experimentation) is “grossly misleading” (ibid.: 31).

In addition, “the false idol of objectivity has done great damage to economic science” (Leamer 1983: 36). If we want to make progress, according to Leamer, “the first step we must take is to discard the counterproductive goal of objective inference” (ibid.: 37). Inference is a logical conclusion based on facts, but because “the sampling distribution and the prior distribution are actually opinions and not facts, a statistical inference is and must forever remain an opinion” (ibid., original emphasis). Moreover, Leamer considers a fact as “merely an opinion held by all, or at least held by a set of people you regard to be a close approximation to all” (ibid.).

The problem with using opinion, however, is its “whimsical nature”. An inference is not “believable” if it is fragile, if it can be reversed by a minor change in assumptions. It is thus the task of the econometrician to withhold belief until an inference is shown “to be adequately insensitive to the choice of assumptions” (Leamer 1983: 43).