Monday, October 03, 2005

The killer application

I do modeling of the economy (micro and macro) for a living. I've been asked, how good are computer models? After all, a model, any model, requires simplifying assumptions. Lots of factors are omitted so that some clean equations can be written down in a way that a computer can process. Forecasts generated are prone to uncertainty (and, literally speaking, are always wrong!)

Outside of economics, let me present the following:

Exhibit A. Climate forecasting

The prediction of global warming could only come from simulations of computer models of the climate. Virtually a scientific consensus on the future trends of global tempertures has been reached, on the strength of computer models. How so given model uncertainty? A simple explanation is given by an economist article on climate change:

Individual models have their individual faults, of course. But unless all contain some huge, false underlying assumption that is invisible to the world's climatologists, the fact that all of them trend in the same direction reinforces the idea that it is the data which are spurious rather than the models' predictions.

Exhibit B. War forecasting

The techniques used in model-building cannot differ across disciplines. In the technique of "backcasting" we use the following: Hypothesize a theoretical relationship between variables; build a database; identify the numerical relationships using data; check if the data is replicated by the model; apply the model to forecast future trends. That is precisely what the state-of-the art commercial war forecasting software does - though with the advantage of an enormous database.

Some anecdotal evidence:
IN DECEMBER 1990, 35 days before the outbreak of the Gulf war, an unassuming retired colonel appeared before the Armed Services Committee of America's House of Representatives and made a startling prediction. The Pentagon's casualty projections—that 20,000 to 30,000 coalition soldiers would be killed in the first two weeks of combat against the Iraqi army—were, he declared, completely wrong. Casualties would, he said, still be less than 6,000 after a month of hostilities. Military officials had also projected that the war would take at least six months, including several months of fighting on the ground. That estimate was also wide of the mark, said the former colonel. The conflict would last less than two months, with the ground war taking just 10 to 14 days.
Operation Desert Storm began on January 17th with an aerial bombardment. President George Bush senior declared victory 43 days later. Fewer than 1,400 coalition troops had been killed or wounded, and the ground-war phase had lasted five days. The forecaster, a military historian called Trevor Dupuy, had been strikingly accurate.

Would that economists had such a killer app.

5 comments:

Anonymous said...

One other issue with data models is that your conclusions can dramatically change depending on the scope of data you used. In the climate change example, take a look at this web site: http://muller.lbl.gov/pages/IceAgeBook/history_of_climate.html

Notice the change in temperature over a 500,000 year period, and think about the conclusions one can draw from it (which is contrary to the current consensus on what is driving climate change).

AT

Roehlano said...

I think climatologists understand very well that long term cyclical swings swamp short term fluctuations. However the short term the global warming school is referring to is in the order of centuries (one century, in fact). Surely more relevant to us than an ice age in say, 5,000 years hence!

Amadeo said...

I'm not sure exactly why the two models (climate and war)were explained in relation to economics.

I hope it is just because of the sophistication and technical advances of the processes being applied by the former two.

And not because the three areas, or call them disciplines, are similar enough to be lumped into one basket.

Climate, for example, is essentially of nature, and not man-made. Of course, we grant that man collectively over time does have an effect on it. But quite negligible. Current knowledge tells us that all the collective damage to the environment caused by man's machines and factories is nothing compared to the destructive effects of a good-sized volcanic eruption.

And war, well maybe, with regard to deployment of human resources, logistics, and maybe even the time frame. But projection of casualties? Might not the last one simply be one of rolling the dice? For the current war, initial projections in media were as high as tens of thousands of casualties. And nobody came out to rebut them with any "official" projection.

My thinking is simply that modelling may work better for certain disciplines and not for others.

The more man in a free society is involved in the equation the less accurate the created model will be.

The reason maybe why Economics is derisively called the dismal science, and its study still littered with a lot of theories?

Just my thoughts.

Anonymous said...

Econblogger: my point was that if we look at data over the past century, we only see a rise, which corelates with increased industrial activity. Hence, we start to assign causation. however, if we were to look at climate patterns beyond the recent century, we discover this rise has been in the works for a time, and could simply be part of a natural cycle. While I am for controlling greenhouse gas emmissions, I think we need to put things in perspective. The wrong decision would be to bankrupt ourselves controlling emmissions, but have temperatures in the next 100 years continue to rise.

Amadeo, I think the point is not that the three are similar, but they use the same statistical tools.

AT

Roehlano said...

Amadeo,

My original question was: How good are those forecasting models? Economics, climatology, and now war modeling seem to be sharing a common toolbox (as observed by anonymous).

Anonymous,

I share your reluctance to endorse draconian greenhouse gas emissions protocols. I suggest a regime of tradable carbon quotas, where the quota is not set too high. (Don't ask me for a definition of that!) This regime will spur innovations towards reductions of greenhouse gases.