The unknown unknowns

Nassim Taleb has written a new book, The Black Swan: The Impact of the Highly Improbable. (See the description in my book list on the sidebar). His main point is that most of the highly trained and highly paid experts in government and industry and academia and the media who produce endless learned projections about future stock and commodity prices, oil prices, sales volumes, interest rate movements, government spending, projects costs and the like simply don’t know what they are doing. And in fact, a little simple research supports this observation. Look at the past projections in fields like these and the success rate is not much different from that of an untrained person simply predicting a continuation of whatever the current trend happens to be.

The experts will defend their analyses by pointing out that when they were wrong (which is more often than not) it was because unusual circumstances intervened – wars, unanticipated scientific breakthroughs, natural disasters, unusual weather, terrorist actions, the collapse of a government or an empire, an unexpected shift in fashions, labor unrest, or any of an endless litany of other “unusual”, “one-off” events that were extremely unlikely, and therefore not factored into their predictions.

Taleb’s point is that although each these events may be highly improbable and unlikely in themselves, for future projections over any significant period (say more than a few weeks) it is nevertheless highly likely that one or more highly improbable and unpredictable events will occur during the periods being projected, and that these improbable events will tend to have massive consequences, rendering the projection worthless.

Despite the impressive mathematical models and learned theories that the practitioners believe in, and use to impress the gullible public and clients and justify their high status and salaries, most of these projections are worthless for spans of more than a few weeks or perhaps a few months, because they have no way to factor in the (inevitable) unexpected and unanticipated.

The lesson here is to know what one does not know. Donald Rumsfeld took a lot of ridicule from the Plain English Campaign and many journalists for his statement many months ago that it was the “unknown unknowns” that were the biggest problem. But he was right (for once), and those who ridiculed him were just exposing their own ignorance. There are, in fact, “known unknowns” and “unknown unknowns”, as he correctly stated.

In fact, we each know a good bit less than we think we know, and we would be wise to always remember that. Weather forecasters know that they can’t predict accurately for more than a few days (actually significantly fewer days than most weather channels and newspapers report); they know what they don’t know. Would that other public pundits could learn the same humility.