The measurement-without-theory debate

Another of the early econometric debates started with a lengthy book review written in 1947 for The Review of Economic Statistics by Tjalling C. Koopmans (1910-1985). In 1949, under the general heading “Methodological issues in quantitative economics”, a number of publications advancing the debate appeared in the journal: a response from Rutledge Vining (1949a) was followed by a “Reply” from Koopmans (1949), and finally a “Rejoinder” from Vining (1949b).

The subject of Koopmans’s review was Measuring Business Cycles, by Arthur F. Burns and Wesley C. Mitchell (1946), and published by the National Bureau of Economic Research (NBER), of which Mitchell was the director between 1920 and 1945. At this time Koopmans was a senior research figure, and midway through the debate (1948) he became director of the Cowles Commission. So, his original article was more than a book review. Koopmans was mounting a full critique of the NBER empirical approach while simultaneously defending the Cowles Commission’s econometric approach.

Koopmans’s critique was based on Haavelmo’s “Probability approach” (1944). He accused Burns and Mitchell of trying to measure economic cycles in the absence of any economic theory about the workings of such cycles: “The toolkit of the theoretical economist is deliberately spurned” (Koopmans 1947: 163).

Koopmans put forward three arguments to explain the implications and limitations of the NBER’s “empiricist position”. His first argument is that for the purposes of system­atic and large-scale observation of a many sided phenomenon such as the business cycle, “theoretical preconceptions about its nature cannot be dispensed with, and the authors do so only to the detriment of the analysis” (Koopmans 1947: 163). He compared this empiricist position with Kepler’s discovery of the more superficial empirical regularities of planetary motion, which fell short of the deeper “fundamental laws” later discovered by Newton.

However, Koopmans believed that economists are unable to perform experiments on an economic system as a whole, and that it is therefore impossible for many economic problems to separate causes and effects by varying causes one by one, and studying the separate effects of each cause. According to Koopmans, instead of experiments, econo­mists possess “more elaborate and better established theories of economic behavior than the theories of motion of material bodies known to Kepler” (1947: 166), because the evi­dence for these theories is based on introspection, on interviews, and on inferences from the observed behaviour of individuals.

In general, economic variables are determined by the simultaneous validity of a large number of structural equations describing behaviour and technology. Any observed empirical regularity between a number of variables may be the result of the working of several simultaneous structural relations. Because so many empirical relations are valid simultaneously, it may be difficult - or even impossible - to uncover the more funda­mental structural relationships. In the absence of experimentation, the identification of these structural relations is possible only if the set of variables involved in each equa­tion, and perhaps also the manner in which they are combined, is specified by economic theory.

Koopmans’s second argument against the NBER’s empiricist position was that it offered no evidence for the assumption that the empirical relations found would be invariant across time. As long as the dynamics of economic variables are not based on structural relations of behaviour and technology, it was difficult to know how reliable they would be for the purposes of prediction or as a guide for economic policy.

Koopmans’s third argument against a purely empiricist approach is that statistical analysis of the data requires additional assumptions about their probabilistic character­istics that cannot be subject to statistical testing from the same data.

In a defence of “empiricism as a fundamental part of scientific procedure”, Vining replied by offering three points that challenged Koopmans’s arguments. His first point is that he doubted whether “the method of Koopmans’s group” would lead to the uncovering of the fundamental invariant relationships:

Is it not something of a mighty jump to imply that the postulated preference function of an individual is in some sense analogous to the general laws of thermodynamics, the dynamics of friction, etc., etc.? Is the Walrasian conception not in fact a pretty skinny fellow of untested capacity upon which to load the burden of a general theory accounting for the events in space and time which take place within the spatial boundary of an economic system? (Vining 1949a: 82)

He asserted that the theory about the behaviour of economic agents had not been given in sufficient detail. Vining stated that the Cowles model was therefore a “pretty skinny fellow” upon which to base so much high-powered statistical estimation.

He questioned the position that empirical research should be evaluated from the point of view of social usefulness. However, Vining did not offer any further discussion of this point.

The third point was that Cowles’ version of statistical economics, if it includes only the estimation of postulated relations, had little or no role to play in the discovery of economic hypotheses. According to Vining, statistical theory should play a similar role in economic research to that played by microscopy in biology.