First, let’s start with a fun exercise courtesy of Wilmott Magazine. Let’s look at damage estimates of earthquakes in California from 1970 to 1993 in the table below. Can you make an educated calculation of losses due to earthquakes in 1994?

Taking a look at the distribution of data, notice the low end is “0″ and at high end, the most damage caused was “129″. So what’s your guess?

If you’re like me, you probably guessed wrong. Using the above numbers as an “anchor“, most people would reasonably assume that 1994’s earthquake was either an average of the above numbers or perhaps a bit higher than 129. Maybe you even threw out “129″ as an outlier in the dataset. To be honest, I guessed around “200″.

The correct answer is “2217.2″! FEMA estimates that every year earthquake losses in the United States add up to $4.4 billion a year. But then, some extreme outliers can really skew that number, especially years like 1994 where just the Northridge Earthquake in California alone tallied $20B in damage!

Let’s get back to talking turkeys via a parable from Nassim Taleb, author of the “Black Swan“. Dr. Taleb reminds us that fat, dumb and happy is probably the best way to describe the life of a turkey. They’re fed and nurtured for three years straight. Day after day, they expect the same thing. But then, one fateful day arrives and the “life” of a turkey ends quite abruptly.

Can we accurately predict the future based on reviewing and analyzing historical data? Sometimes, but we have to make assumptions of similar conditions, a normal distribution, and event independence. Complex systems will have none of these characteristics. Dr. Taleb says as much; “Real life isn’t a casino.”

Indeed, the parable of the turkey and the earthquake loss estimation exercise show us that predicting the future in complex systems can be a futile exercise because there are so many unknowns, changing conditions, and inter-connecting relationships. Extreme events that carry a huge impact happen, and some would argue they’re happening a whole lot more often as interlocking financial markets and globalization become commonplace.

Should prediction exercises be avoided? Nassim Taleb would argue otherwise; “We need to start thinking of the inconceivable,” he says. And while we cannot determine the exact probability of tomorrow’s events, we can “get a general idea about the possibility of their occurrence.”

And that’s where scenario planning comes into play. Bill Ziemba, author of the aforementioned Wilmott Magazine article says, “Getting all the scenarios and their probabilities right is impossible and doesn’t matter much anyway. What is important is to cover the board of possible occurrences. Then you will make sound decisions with risk under control.”

The fact is, like the turkey, we just don’t know what tomorrow will bring. So, plan for the five to seven most likely occurrences and then develop contingencies based on those scenarios. French microbiologist Louis Pasteur says it best, “In the fields of observation chance favors only the prepared mind.”

For a turkey, today may appear like any other “normal” day. However, tomorrow could be the chopping block.

Questions:

• Nassim Taleb says, “It is only in very rare circumstances that probability (by itself) is a guide to decision making.” Does this mean that historical data analysis isn’t worth the effort?
• If chance favors the prepared mind, what’s the “next unexpected twist” that marketers should be looking for?

It’s that wonderful time of the year where leaves turn bright amber, apples hang from trees, tempting smells waft from the kitchen and families across the United States gather for the Thanksgiving holiday. To be sure, of all holiday staples for Thanksgiving, most in-home chefs won’t forget the turkey.

However, this particular Thanksgiving, I’d like to bring up the parable of the turkey as told by Nassim Nicholas Taleb as a constant reminder that all business executives should always plan for the unexpected …. no matter how remote the possibility.

As detailed in “The Black Swan“, Taleb asks us to consider the life of a turkey. You see, the lifecycle of most turkeys is well known to the outside observer. A turkey is born, and then fed generously everyday. For 1,000 days straight the turkey expects its morning shovel of grain, grass, acorns or shoots.
A monumental event happens to that turkey on day 1001–and we all know what that event is. Until that point of course, everything is considered “normal” by the turkey. In fact most days deviate only slightly from the norm, with maybe a few minor exceptions.

Getting back to day 1001, Taleb points out, “A turkey cannot figure out what is in store for it tomorrow based on the events of today.” And Taleb notes that this is one of the key challenges with assuming tomorrow or even next year will be much like today.

Author Peter L. Bernstein often asks, “What is ‘normal’ anyway?” Good question.

Quantitative financial analysts, attempting to model the stock market, took heavy losses in 2008 as they assumed the past would be a relevant predictor of the future. Automobile manufacturers forecasted that demand would be similar to last year (maybe a bit less), and now many are frantically leasing space outside of key US ports to store unsold cars.

This year has seen the DOW index range between 13,000 and 8,000. And Mr. Greenspan, former Federal Reserve Chairman, said in testimony to the United States Congress in October 2008, “This crisis–has turned out to be much broader than anything I could have imagined.”

What’s a business executive to do?
Don’t throw out the models–just yet. I have previously mentioned that modeling can be a very valuable tool for companies assessing future scenarios, determining cause and effect, and allocating scarce resources. But use caution; models are support tools that should be combined with good judgment, experience, and the input of others to effectively drive decisions.

Another avenue is to pay attention to outliers via analysis. Alberto Roldan, mentions in a post that, “(Outlier analysis) uses concepts like average, standard deviation, and Z-scores to determine whether a determined data point is abnormal in the same classification or category.” Examine the outliers in your market. Are they occurring more frequently than your models predict?

A final recommendation from Taleb, “We need to start thinking of the inconceivable.” Can a business executive plan and resource for every outcome? No, but Taleb reminds us while it is very difficult to discern the exact probability of significant events, “it is easy, however to get a general idea about the possibility of their occurrence.”

Complacency is not your friend. There are good chances that tomorrow will vary significantly from today. Don’t forget the parable of the turkey!

Questions:
* Are the global financial events of 2008 simply outliers? Should we assume that things will return to a level of “normalcy”?
* Should the “extremes” be considered as you budget and forecast for the next one to three years? How are you taking the possibility of “significant events” into account?
* As we close out the end of the year, most marketers have already turned in preliminary budget requests for their next fiscal year. What key assumptions are you using to plan for next year?
* Are global markets getting more risky or less? In an unpredictable world, how are you dealing with uncertainty?

Data, by itself is of little value. The real value lies in the capturing, cleansing, management and analysis of data thereby making it more useful for decision making.

One of the ways companies analyze data is to build or “program” models to simulate, test, learn and predict outcomes. Indeed, models built upon statistical techniques are helping companies identify fraud, predict next best offers, determine customer churn, and assess credit risk among other valuable applications.

But modeling isn’t a panacea, and a recent New York Times article, “How Wall Street Lied to Its Computers“, shows us how companies can get it wrong.

Writer Saul Hansell notes that Wall Street traders have long had very sophisticated models on market behavior–devised by quantitative analysts–that were supposed to help them hedge their positions and allow them to essentially manage their risks (or enable them to take bigger risks).

However a key challenge emerged when, “The people who ran the financial firms chose to program their risk management systems with overly optimistic assumptions and feed them oversimplified data.”
Even worse, many of the products (read: derivatives and derivatives of derivatives) weren’t understood by the creators of the products and thus it was near impossible to accurately assess the risk of these products with a mathematical model.

Modeling isn’t just for risk management, and can be a very valuable tool for companies assessing future scenarios, determining cause and effect, and allocating scarce resources. However the New York Times article highlights a great case study of pitfalls and key challenges when attempting to model a system, phenomenon or behavior.

First, understand that a model will only be as good as your assumptions. For example, in many risk management systems, models are designed to assume rational decision makers, a stable and relatively volatile-free marketplace, and that outliers generally have a limited effect on the entire population. Anyone who’s invested in a 401K and tracked their stock portfolio recently knows the futility of these assumptions (Nassim Nicholas Taleb explains why here).

Second, mathematical modeling is only as good as your data. Hansell’s article points out that it was in the best interest of traders to ensure the models didn’t warn them of impending danger, so they took efforts to smooth the data and manipulate the amount of historical data their risk management systems could analyze so as to take more aggressive trading positions.

Third, modeling is only as good as the design and designer of the model. “There was a willful designing of the systems to measure the risks in a certain way that would not necessarily pick up all the right risks,” says Gregg Berman of software company Risk Metrics. The design of an model should be checked for accuracy–not only of the accuracy statistical concepts used, but also that the model is not “tweaked” to produce desired results.

Last point: a model might be based on fair and accurate assumptions, sourcing clean and legitimate data, and designed properly–however it is of little use of politics stands in the way of recognizing and acting upon the output. All the analytical systems in the world are of little use if corporate politics dictate an outcome that is different than what the model prescribes.

No mathematical model is perfect–a model is just that–a model and not a silver bullet. Also such models are support tools that should be combined with good judgment, experience, and the input of others to effectively drive decisions.

That said, the time, energy, and investment dollars spent on mathematical modeling is close to worthless when poor assumptions, faulty/dirty data, bad design and corporate politics get in the way of good decision making.

Questions:
* Mathematical models are used by companies for customer segmentation, risk management, propensity to buy, loyalty management etc. Are you using models to help you make better decisions? If so, how?
* Do you think we often try to model things that are too complex–things that can’t be modeled effectively? What might be the ramifications when we get it wrong?
* In the business world, do you think the use of mathematics sometimes overrides “common sense”?
* Mathematical modeling–done right–can be a powerful business tool. How can we teach future generations of business leaders to use these tools ethically?