IT has always been a risk-filled enterprise. That's because software development is a first-of-a-kind undertaking. Programmers write programs to make software and hardware systems function in new ways. Often, the novelty of the development effort is small, requiring minor adjustments to existing programs. In this case, development risk is generally low. To the extent that the programming entails exploring truly new territory, then development risk goes up.
Programmers and systems analysts have been aware of this aspect of development risk for decades. However, in an intriguing 2007 book, titled The Black Swans, Nassim Nicholas Taleb suggests that with our single-minded focus on the traditional perspective on development risk, we are ignoring a category of risk whose consequences can be devastating -- what he calls the black swans.
The idea of black swans arose among English philosophers in their discussion of David Hume's problem of induction. The induction problem holds that even if you observe 1,000 white swans, you cannot say with certainty that the next swan you encounter will be white. (As a matter of fact, black swans were unknown until their discovery in Australia in 1790.)
Taleb uses black swans as a metaphor for totally unanticipated high-impact events. DoD managers know them as unk-unks, i.e., unknown-unknowns. He points out that when you look at problems individuals, organizations, and societies face from a historical perspective, those with greatest consequences are the total surprises. The 9/11 attacks and the global financial crisis of 2008 are recent examples.
When applied to IT undertakings, we find that in certain respects, IT efforts are well-prepared to deal with black swan events, while in others they are not. They are well-prepared in the sense that all capable IT organizations have established disaster-recovery plans to deal with catastrophic challenges to their operations. By having cool, warm, and/or hot back-up sites, IT enterprises are able to handle disasters that bring down their data processing and communication capabilities. Thus they are able to handle the consequences of power outages, earthquakes, fires, hurricanes, and other natural disasters. As the 9/11 attacks showed, they can even withstand horrific terrorist onslaughts.
However, in the day-to-day pursuit of IT projects, IT organizations do not generally equip themselves to deal with black swan events. Their risk management efforts are largely directed at catching problems through testing and resolving them by fixing bugs. Seldom do they explicitly take into account broader forces that can jeopardize IT projects, e.g., funding cutbacks, changes of management, technology changes that render their solutions obsolete, and other things of this ilk. These are the forces that are likely to produce black swans in IT. The fact that most IT projects gain funding support only after project champions promise to do ten months of work in six months, and promise to deliver more functionality than is possible to deliver, increases the negative impacts that black swan events can have on IT enterprises and their projects.
These are uncertain times. For the first time since the turmoil of the Great Depression, Americans recognize that when bad things happen, things do not always turn out for the best. It behooves all organizations -- including IT shops -- to recognize that even though black swans are rare-occurrence events that are not predictable, they need to be taken seriously. What may have been viewed as recoverable setbacks one or two years ago, may yield catastrophic consequences today.
COMMENTS
On major programs, we have addressed black swans a long time, only we call them unk-unks. While we have a general idea of the possibility of untoward risk events (e.g., the attack by terrorists on an American icon), we are in the dark about the specifics (e.g., World Trade Center/Pentagon, three hijacked aircraft, 19 terrorists, September 11). So when a specific negative risk event comes to fruition, it comes as a surprise -- not in general terms, but in terms of the specifics.
At DoD and DoE, unk-unks are largely handled with management reserves. While we don't know the details of what bad things will happen, we establish resources and strategies to deal with them in general terms. Computer disaster recovery processes are another way to handle unk-unks, i.e., black swans.
In dealing with known-unknowns (which are not black swans), we establish contingency reserves -- plans and resources set aside to deal with specific, known events. Contingent upon X happening, we implement response Y.
Ronald Solis 05/19/09 01:19 pm ET
While I agree with the idea of Black Swan events, I do not agree with the characterization of these being total surprises. It is more poor risk management. Neither 9/11 nor the recent financial crises were total surprises. The potential for 9/11 style attacks were discussed in the popular media, but were characterized as media hype. I expect there were some in the intelligence community who were also predicting this type of event, but government tends to minimize events with a low or long term probability. The financial meltdown was even more obvious with probablities higher, dare I say 100%. The question was when. People get caught up with the euphoria of easy credit and a fast buck.
There are a number of issues re risk management. First is the list of potential outcomes and their probablity of occurrence. Oftentimes low probability events are ignored, such as 9/11. Another issue is the potential impacts of the occurrence of an event. While many expected a pull back in the financial markets, many did not foresee the cascading effects of multiple failures likely when the financial system was so out of balance. The third issue is risk management, i.e. mitigating the risk of an event or mitigating the impacts of an occurrence of an event. The government did little to mitigate the risk of 9/11. They should have been sharing information between the various agencies, but that would have meant overcoming longstanding institutional barriers. The government actually increased the risk of the financial meltdown. Capitalism is self-correcting, but the ups and downs can be extreme. Regulation tends to reduce the extremes. Unfortunately the government reduced regulations, enabling the extremes.
John Broderick 05/19/09 08:35 am ET
Ho Hum. Such profundity. I'm waiting for your blog on why bad things happen to good people.
Phil Kiviat 05/19/09 07:21 am ET