First, let’s start with a fun exercise courtesy of Wilmott Magazine. Let’s look at damage estimates of earthquakes in California from 1970 to 1993 in the table below. Can you make an educated calculation of losses due to earthquakes in 1994?

Taking a look at the distribution of data, notice the low end is “0″ and at high end, the most damage caused was “129″. So what’s your guess?

If you’re like me, you probably guessed wrong. Using the above numbers as an “anchor“, most people would reasonably assume that 1994’s earthquake was either an average of the above numbers or perhaps a bit higher than 129. Maybe you even threw out “129″ as an outlier in the dataset. To be honest, I guessed around “200″.

The correct answer is “2217.2″! FEMA estimates that every year earthquake losses in the United States add up to $4.4 billion a year. But then, some extreme outliers can really skew that number, especially years like 1994 where just the Northridge Earthquake in California alone tallied $20B in damage!

Let’s get back to talking turkeys via a parable from Nassim Taleb, author of the “Black Swan“. Dr. Taleb reminds us that fat, dumb and happy is probably the best way to describe the life of a turkey. They’re fed and nurtured for three years straight. Day after day, they expect the same thing. But then, one fateful day arrives and the “life” of a turkey ends quite abruptly.

Can we accurately predict the future based on reviewing and analyzing historical data? Sometimes, but we have to make assumptions of similar conditions, a normal distribution, and event independence. Complex systems will have none of these characteristics. Dr. Taleb says as much; “Real life isn’t a casino.”

Indeed, the parable of the turkey and the earthquake loss estimation exercise show us that predicting the future in complex systems can be a futile exercise because there are so many unknowns, changing conditions, and inter-connecting relationships. Extreme events that carry a huge impact happen, and some would argue they’re happening a whole lot more often as interlocking financial markets and globalization become commonplace.

Should prediction exercises be avoided? Nassim Taleb would argue otherwise; “We need to start thinking of the inconceivable,” he says. And while we cannot determine the exact probability of tomorrow’s events, we can “get a general idea about the possibility of their occurrence.”

And that’s where scenario planning comes into play. Bill Ziemba, author of the aforementioned Wilmott Magazine article says, “Getting all the scenarios and their probabilities right is impossible and doesn’t matter much anyway. What is important is to cover the board of possible occurrences. Then you will make sound decisions with risk under control.”

The fact is, like the turkey, we just don’t know what tomorrow will bring. So, plan for the five to seven most likely occurrences and then develop contingencies based on those scenarios. French microbiologist Louis Pasteur says it best, “In the fields of observation chance favors only the prepared mind.”

For a turkey, today may appear like any other “normal” day. However, tomorrow could be the chopping block.

Questions:

• Nassim Taleb says, “It is only in very rare circumstances that probability (by itself) is a guide to decision making.” Does this mean that historical data analysis isn’t worth the effort?
• If chance favors the prepared mind, what’s the “next unexpected twist” that marketers should be looking for?

Forecasting sales and inventory levels is probably one of the most difficult jobs of a product and/or supply chain manager as companies need to marry demand signals with supply. Adding more complexity to the mix is global supply chains that span weeks, multiple countries and sometimes oceans. Lots of hand-offs, tons of data to track, and lots of points for things to go wrong.

For many product managers (and the marketing/brand managers that support them) inventory management is a critical task. By not carrying enough inventory, companies can not only lose out on sales but also suffer reputation damage by not meeting customer needs.

Nonetheless, with companies hoarding cash–it seems the last thing companies need is to be stuck with unsold finished goods or piecemeal parts.

Apple’s Chief Operating Officer Tim Cook agrees. In a recent Fortune article, Cook says inventory is “fundamentally evil.” And Cook should know, as he’s in the very fickle consumer electronics business. “You kind of want to manage it like you are in the dairy business,” he says. “If it gets past its fresh date you have a problem.”

Here’s the rub, however. Forces of globalization and tight coupling are magnifying the complexity, impact and frequency of events. Once steady suppliers are going bankrupt, some suppliers cannot get loans in the credit crunch, and disruptions in the supply chain are becoming more commonplace. Your product launch date doesn’t matter much if your suppliers cannot deliver.

But can’t analytical modeling save us? After all, most companies are using advanced planning applications to predict future trends and behaviors, right?

While statistical forecasting techniques can help extrapolate future trends, these methods rely on building models based on historical data. And some executives say in volatile times, historical data can no longer be trusted to accurately model and predict the future.

So what’s the solution? Should we build more redundancy into our supply chains to better manage the risk of suppliers, or stay the course with the trend towards information management and just-in time supply chains that are well optimized and thin?

Better communication is a potential answer says Camille Schuster, President of Global Collaborations. What is needed, she says is, “Proactive contact with suppliers on a regular basis to determine how supplies are doing, what issues are coming up, whether any shortages are foreseen, whether there is any softness in any product area, what changes and/or rumors are floating about.”

For many companies, effective inventory management is a critical component of financial health. With “cash” at a premium in this global financial pandemic, inventory decisions can literally make or break your company.

When it comes to inventory, what level of risk are you comfortable with?

Questions for DailyFix readers:

* Is a little inventory cushion warranted as risks (environmental, political, criminal, financial, reputation, terrorism etc) seem to be increasing in intensity, complexity and frequency?
* In volatile times, should forecasting and inventory management be more focused on “gut” decision making rather than mathematical models?
* Stockouts leave “money on the table” and ultimately reduce customer satisfaction. What is your marketing advice to supply chain, operations and/or engineering executives in these volatile times? Hedge their bets with a little more inventory, or continue to operate “thin”?
* Is inventory still evil? Should it be avoided at all costs?

Data, by itself is of little value. The real value lies in the capturing, cleansing, management and analysis of data thereby making it more useful for decision making.

One of the ways companies analyze data is to build or “program” models to simulate, test, learn and predict outcomes. Indeed, models built upon statistical techniques are helping companies identify fraud, predict next best offers, determine customer churn, and assess credit risk among other valuable applications.

But modeling isn’t a panacea, and a recent New York Times article, “How Wall Street Lied to Its Computers“, shows us how companies can get it wrong.

Writer Saul Hansell notes that Wall Street traders have long had very sophisticated models on market behavior–devised by quantitative analysts–that were supposed to help them hedge their positions and allow them to essentially manage their risks (or enable them to take bigger risks).

However a key challenge emerged when, “The people who ran the financial firms chose to program their risk management systems with overly optimistic assumptions and feed them oversimplified data.”
Even worse, many of the products (read: derivatives and derivatives of derivatives) weren’t understood by the creators of the products and thus it was near impossible to accurately assess the risk of these products with a mathematical model.

Modeling isn’t just for risk management, and can be a very valuable tool for companies assessing future scenarios, determining cause and effect, and allocating scarce resources. However the New York Times article highlights a great case study of pitfalls and key challenges when attempting to model a system, phenomenon or behavior.

First, understand that a model will only be as good as your assumptions. For example, in many risk management systems, models are designed to assume rational decision makers, a stable and relatively volatile-free marketplace, and that outliers generally have a limited effect on the entire population. Anyone who’s invested in a 401K and tracked their stock portfolio recently knows the futility of these assumptions (Nassim Nicholas Taleb explains why here).

Second, mathematical modeling is only as good as your data. Hansell’s article points out that it was in the best interest of traders to ensure the models didn’t warn them of impending danger, so they took efforts to smooth the data and manipulate the amount of historical data their risk management systems could analyze so as to take more aggressive trading positions.

Third, modeling is only as good as the design and designer of the model. “There was a willful designing of the systems to measure the risks in a certain way that would not necessarily pick up all the right risks,” says Gregg Berman of software company Risk Metrics. The design of an model should be checked for accuracy–not only of the accuracy statistical concepts used, but also that the model is not “tweaked” to produce desired results.

Last point: a model might be based on fair and accurate assumptions, sourcing clean and legitimate data, and designed properly–however it is of little use of politics stands in the way of recognizing and acting upon the output. All the analytical systems in the world are of little use if corporate politics dictate an outcome that is different than what the model prescribes.

No mathematical model is perfect–a model is just that–a model and not a silver bullet. Also such models are support tools that should be combined with good judgment, experience, and the input of others to effectively drive decisions.

That said, the time, energy, and investment dollars spent on mathematical modeling is close to worthless when poor assumptions, faulty/dirty data, bad design and corporate politics get in the way of good decision making.

Questions:
* Mathematical models are used by companies for customer segmentation, risk management, propensity to buy, loyalty management etc. Are you using models to help you make better decisions? If so, how?
* Do you think we often try to model things that are too complex–things that can’t be modeled effectively? What might be the ramifications when we get it wrong?
* In the business world, do you think the use of mathematics sometimes overrides “common sense”?
* Mathematical modeling–done right–can be a powerful business tool. How can we teach future generations of business leaders to use these tools ethically?

Chris Anderson, editor of Wired Magazine and author of “The Long Tail” penned a provocative article in the July 2008 issue titled, “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete.”

Mr. Anderson makes the claim that in “the Petabyte Age”, it’s more important for companies (and the marketers within them) to identify and act upon correlation first, and worry about context later. For instance he writes, “Google’s founding philosophy is that we don’t know why this page is better than that one: if the statistics of incoming links say it is, that’s good enough. No semantic or causal analysis is required.”

And dismissing many of the sciences that attempt to bring us understanding of the world around us, Mr. Anderson notes, “Who knows why people do what they do? The point is that they do it and we can track and measure it with unprecedented fidelity. With enough data, the numbers speak for themselves.”

As companies collect more data about their customers, competitors and macro/micro environments, Mr. Anderson makes the claim that our approach to science (hypothesize, model, test) is becoming obsolete.

Science looks for causation. Scientists hypothesize as to why something works/reacts/behaves like it does and then attempt to build a model to represent reality. The goal is to use the model to test and learn, thereby gaining an understanding of a particular phenomenon.

Modeling is not only confined to the realm of physicists and quants. In the field of marketing, managers often work alongside statisticians, to build models that help predict customer proclivities such as items they might like to buy, or identifying customers who might churn to a competitor. Models can be tested and refined and with enough tweaking, models generally get more accurate over time. The output of those models can be used to piece together more complete customer profiles.

However, in the Petabyte Age, Mr. Anderson claims, “There is now a better way. Petabytes allow us to say correlation is enough. We can stop looking at the models.”

Let’s suppose Mr. Anderson is on to something. Mr. Anderson makes the claim that since modeling is often a poor representation of reality and tends to over-simplify things, we’re much better off with less explanation and more action based on the identification of correlation.
In a sense, he says, in an age of massive data, we’re better off with fewer discoveries of knowledge and understanding.

I’m not sure I agree.

Peter Atkins, author of “Galileo’s Finger“, says it much better than I can: “With the rise of the computer and its ability to handle huge numerical calculations of the greatest intensity, we are seeing a shift from analysis to numerical computation. (This is dangerous because) resorting to numerical solution can distance us from understanding.”

It is true that building models is an imperfect science, and cannot in all instances be 100% accurate–it is after all just a model!

That said, models help us test our assumptions and verify our forecasts. They help us test what we think we know, with the ultimate goal of improving our decision making in real-life situations where marketing budgets and return on investment are on the line. Modeling helps us piece the world together, and look beyond the patterns towards discovering “why” things happen as they do. And models help us transform reams of raw data into intelligence thereby helping us predict outcomes with greater accuracy.

While focusing on correlation can help us make better decisions to a degree, philosophically I am concerned with marketers potentially losing our interest in truly understanding what makes our customers, prospects and partners tick. For me at least, I want to know more than “my customers do this, or they do that”; I want to know “why“!

In all fairness to Mr. Anderson, every coin has two sides. There is a fine line between too much pontification and too little action. Indeed, there are many instances where it doesn’t make sense to dig deeper in understanding–where it doesn’t matter “why”, only that a given solution produces results.

I’d like to open this discussion up to the MPDailyFix community.

* As a marketer, do you want to understand and be able to explain the causes of why customers/companies/competitors do what they do, or is observing and acting on the behaviors good enough?
* Building accurate models requires tons of data collection, the proper analytical technologies and applications and of course the know-how. It’s not easy work. Do you agree with Mr. Anderson that for most decision making opportunities in the Petabyte age, correlation is enough?
* In a complex and busy world, where a marketer’s time is at a premium, will brainstorming, learning, and piecing the world together, become passé?