Why Progress Cannot Be Planned, or Even Forecast

Few modern intellectuals gave more serious thought to forecasting the future than Herman Kahn. He wrote several books and essays imagining what the future might look like. But he was also a profoundly humble man who understood the limits of forecasting the future. On that point, I am reminded of my favorite Herman Kahn quote:

History is likely to write scenarios that most observers would find implausible not only prospectively but sometimes, even in retrospect. Many sequences of events seem plausible now only because they have actually occurred; a man who knew no history might not believe any. Future events may not be drawn from the restricted list of those we have learned are possible; we should expect to go on being surprised.^[1]

I have always loved that phrase, “a man who knew no history might not believe any.” Indeed, sometimes the truth (how history actually unfolds) really is stranger than fiction (or the hypothetical forecasts that came before it.)

This insight has profound ramifications for public policy and efforts to “plan progress,” something that typically ends badly.  No two scholars nailed that point better in their work than Karl Popper and F.A. Hayek, two of the preeminent philosophers of science and politics of the 20^th century. Popper cogently argued that the problem with “the Utopian programme” is that “we do not possess the experimental knowledge needed for such an undertaking.”^[2] “Progress by its very nature cannot be planned,” Hayek taught us, and the wiser man “is very much aware that we do not know all the answers and that he is not sure that the answers he has are certainly the right ones or even that we can find all the answers.”^[3] One hundred years prior to Hayek making that insight, social philosopher Herbert Spencer explained how humans would only be truly wise once they fathomed the limits of their own knowledge:

In all directions his investigations eventually bring him face to face with the unknowable; and he ever more clearly perceives it to be the unknowable. He learns at once the greatness and the littleness of human intellect—its power in dealing with all that comes within the range of experience; its impotence in dealing with all that transcends experience. He feels more vividly than any others can feel, the utter incomprehensibleness of the simplest fact, considered in itself. He alone truly sees that absolute knowledge is impossible. He alone knows that under all things there lies an impenetrable mystery.^[4]

Unfortunately, most humans suffer from what Nassim Nicholas Taleb calls “epistemic ignorance” or hubris concerning the limits of our knowledge. “We are demonstrably arrogant about what we think we know,” he says.^[5] “We overestimate what we know, and underestimate uncertainty.”^[6] “There are no crystal balls, and no style of thinking, no technique, no model will ever eliminate uncertainty,” argues Dan Gardner, author of Future Babble: Why Expert Predictions Are Next to Worthless. “The future will forever be shrouded in darkness. Only if we accept and embrace this fundamental fact can we hope to be prepared for the inevitable surprises that lie ahead,” Gardner notes.^[7]

This is why attempts to forecast the future so often end in folly. The great Austrian school economist Israel Kirzner spoke of “the shortsightedness of those who, not recognizing the open-ended character of entrepreneurial discovery, repeatedly fall into the trap of forecasting the future against the background of today’s expectations rather than against the unknowable background of tomorrow’s discoveries.”^[8] This is especially true as it pertains to technological change and change in information markets. “Anyone who predicts the technological future is sure soon to seem foolish,” noted technology scholar George Gilder. “It springs from human creativity and thus inevitably consists of surprise.”^[9]

Knowledge of history and historical trends can help inform our decisions and predictions, yet they are insufficient to accurately forecast all that may come our way. “The past seldom obliges by revealing to us when wildness will break out in the future,” observes Peter L. Bernstein, author of Against the Gods: The Remarkable Story of Risk.^[10]

We can relate these lessons to Internet policy and digital economics. A largely unfettered cyberspace has left digital denizens better off in terms of the information they can access as well as the goods and services from which they can choose. In true Schumpeterian fashion, “information empires” have come and gone in increasingly rapid progression.^[11] There are countless “tech titans” that, for a time, seemed to rule their respective sectors only to experience a precipitous fall.^[12] Indeed, if you blink your eyes in the information age, you can miss revolutions.^[13]

“The challenge of disruptive innovation,” observes technology lawyer Glenn Manishin, is that “it forces market participants to rethink their premises and reimagine the business they are in. Those who get it wrong will be lost in the dustbin (or buggy whip rack) of history. Those who get it right typically enjoy a window of success until the next inflection point arrives.”^[14] And the cycle Manishin describes just keeps repeating faster and faster throughout modern information sectors.

This explain why, just as planning and forecasting often fail in a macro sense, they also fail in the micro sense as industries and analysts repeatedly fail to accurately identify future trends and marketplace developments. “Markets that don’t exist can’t be analyzed,” observes Clayton M. Christensen, author of The Innovator’s Dilemma. “In dealing with disruptive technologies leading to new markets,” he finds, “researchers and business planners have consistently dismal records.”^[15]  Simply put, as Yogi Berra once famously quipped: “It’s tough to make predictions, especially about the future.”

The ramifications for public policy are clear. Patience and humility are key virtues since, as economist Daniel F. Spulber correctly writes, “Governments are notoriously inept at picking technology winners or steering fast-moving markets in superior directions. Understanding technology requires extensive scientific and technical knowledge. Government agencies cannot expect to replicate or improve upon private sector knowledge. Technological innovation is uncertain by its very nature because it is based on scientific discoveries. The benefits of new technologies and the returns to commercial development also are uncertain.”^[16]

Policymakers would be wise to heed all this advice before trying to “plan progress,” especially in highly-dynamic, rapidly-evolving technology sectors. As the old saying goes, the best-laid plans of mice and men often go awry.

SHARE: 0 comments