Blame for the global financial crisis has been cast upon government leaders, bankers, home owners, and quantitative analysts. In fact, mathematical models and formulas such as Black Scholes, Gaussian Copula, and VAR have received particular attention as culprits in this financial mess.
And while these financial models had flaws, a more egregious error was the blind faith and lack of critical thinking in the adoption of these models. Executives are realizing that when it comes to modeling complexity–there are no magic beans, there are no magic formulas. But have we learned these lessons too late?
From the dawn of time, humans have wrestled with, and tried to explain the unknown. From myths of how the sun and moon were formed, to why it rains some days and not others, ancients have tried to pin a “cause” on every “effect”. But sometimes, complexity defies simplification.
Case in point, a recent Wired Magazine article describes the rise and fall of David X. Li, a senior Wall Street quantitative analyst who tried to simplify a very complex issue (probability of default on a pool of mortgages) through mathematical modeling.
Li’s formula was known as a Gaussian Copula function and it worked like this. Based on historical data of people similar to me, it might be pretty easy for a banker to discern the probability that I will default on my home loan (assuming a normal distribution and independence). Now let’s add my neighbor to the mix. What are the probabilities that our fortunes are somehow tied together? Are we correlated at all? If I default on my mortgage, will he default on his?
Adding more complexity to the mix, Wall Street often bundled packages of mortgages together, 150-200 at a time, and sold the pool of mortgages to investors. Obviously, a method was needed to identify the overall “risk” of a particular package so that the pool could be priced and sold. With hundreds of mortgages in a particular pool, correlation was extremely difficult–after all, there are nearly an “infinite amount of relationships between various loans that make up a pool.” Yikes!
Now without getting into a messy discussion about how Li modeled correlation on the prices of credit default swaps–instead of historical data on actual mortgage defaults–I’ll sum it up this way: Li’s formula put a pretty bow on a very complex problem.
Armed with Li’s formula, Wall Street was able to build an entire industry of packaged mortgage pools (known as collateralized debt obligations, or CDOs). Using this simple formula, all kinds of bonds and loans could be packaged together (securitized) and priced for investors. It was a nice, neat solution to a messy challenge.
Wall Street bankers ate up the output of the Gaussian Copula formula. They had their magic numbers for assessing risk. That’s all they needed to get a trillion dollar machine moving. Few asked questions, most just wanted their magic numbers. And critical thinking went out the window.
Complex systems often defy a simple mathematical explanation, but what does this have to do with marketing management?
Mathematical modeling isn’t just for financial companies. In fact, analytically savvy companies are now using analytical modeling to discern customer value, reduce customer churn, examine the effect of price changes on demand, and divine the right mix of marketing investments. So modeling–in marketing–is a valuable function. But we can certainly apply some lessons learned from this debacle on modeling default correlation.
First, we cannot turn off our brains. It’s tempting to think we’ve found the magic beans–the single answer that solves an intractable problem. David X. Li and the Wall Street bankers thought they found it. Magic beans cannot replace thinking, questioning, and challenging assumptions.
Second, complex systems are hard to model. Markets, weather–or any system where hundreds, thousands, if not millions of interactions take place at any one time is very difficult to accurately model. If you think you’ve found the magic beans, be careful.
Third, don’t mistake the model for the system. A model is simply a representation of the system. Sometimes a model will predict the behavior of a system correctly and maybe 99% of the time it will get it right. But watch out for the impact of the 1% outliers. They often pack a knockout punch.
For most complex challenges, there are no magic beans, there are no magic formulas. If you think you’ve finally discovered those magic beans–think of David X. Li, and consider going back to the drawing board.
Tags: analysis, analytic modeling, analytics, assumptions, Black Scholes, forecasting, Gaussian Copula, magic formulas, marketing lessons, marketing modeling, securitization, VaR
Paul,
This is an excellent analysis and seems to be right on target, based on my reading of Li’s system. His formula produced numbers that bankers liked, so they turned off their brains and made decisions they otherwise would have avoided.
Excellent post, Paul. My understanding, after the housing bubble burst, was that very few people even understood CDOs or derivatives at all. So my question is this: how could any bankers or other financial institutions risk buying these up without understanding the formula in the first place? Supposedly, only a small handful of people even understood Li’s formula. Even then, does it seem logical to be able to assess risk when hundreds of mortgages are lumped together in one large pool? Think about it. Hope a hard lesson has been learned here so we don’t repeat the mistake of letting the potential for big profits–and greed–get in the way of making sound decisions.
Lewis, I appreciate your comments. This article really highlighted the disconnect in many companies (not just in financial services) between the quantitative analysts and those on the business side. The mathematicians understood “the math” perfectly, but they had no way of knowing that their formulas would be taken to such an extreme and used in unpredictable ways.
And those on the business side never questioned the math, never questioned the assumptions behind the model.
There are no magic beans, there are no magic formulas.
Claire, thank you for taking the time to comment.
Is a lesson here, ‘if it’s too good to be true, it probably is’? Another lesson, which everyone knows, but we conveniently forget (especially when the $$ roll in) is the correlation/causation fallacy.
Hey Paul, amazing as always. Got me thinking back to my University degree where I’ve studied the demand prediction models. It turned out that there’s a matrix of relationships that you need to “calculate” if you want to be accurate.
And the matrix goes from legal, demographic, economic, technological and other “environments” down to the companies, suppliers, customers, shareholders,… And they all interact between them constantly. There’s a butterfly effect as in weather models.
It’s just (too?) complex to predict.
One of my favorite qute was something like this: if the environment is stable, the prediction is simple (add 10% to last year). If the environment becomes unstable, there’s no way you can calculate it.
Dusan, thank you for the compliment!
As you rightly pointed out, modeling complexity is extremely difficult. For example, I know of one financial services company whose fraud detection model has over 2,000 variables. And let’s suppose you get the variables right (and don’t miss any–which is not likely), how then do you properly weight them?
Modeling human behavior – whether it’s propensity to buy something, churn off service, file a claim, commit fraud, perform as expected etc is both science AND art–and definitely not easy.
There is so much finger pointing to do is now it is silly. To many people involved. Our government needs to put their head together and find a solution.
Hi Nick, thanks for adding your comments! There are way too many culprits in this mess to pin the blame on one person or entity, although there are some players in the game that have a larger role (see Bernanke’s mea culpa today in WSJ).
Our global financial system is tied together in very complex and sometimes hidden ways. Markets, companies, assets are all intertwined now, so it’s not just a national challenge, but a global challenge.
A good lesson in that risk lies not only inside our corporate walls (i.e. VAR and the 4:15 report), but increasingly– outside as well.
What I like (hate?) about this modelling is that 1 small variable (most probably even unknown as you pointed out), can:
1. Change the complete weighting system and
2. Change the complete interaction system (1 variable doesn’t make effect ona another anymore, even tough it did before)
So we have a complex model that actually isn’t static. Perfect formula of today doesn’t work tomorrow (as your post excellently pointed out).
Consumer behavior and databases. Art&science indeed!
What I’m proposing is a bit of a stretch, but please read through this entire post before passing judgement.
I’ve done some work in the past studying Ecosystems and the models that govern relationships between organisms in these very complex systems. I see many parallels between ecology and business. A basic obvious parallel is that all IBUs have the same mission: survival (Where organisms need food, water, shelter, space IBUs need profit margins and a defined market environment). Just like all organisms in an ecosystem. In many ways, I think of companies like species in the system. Each ecosystem has keystone species, their well being indicates the well being of the entire system. In the US economy, our keystone species are creditors and lenders.
I think the models exist for the current market situation. There are several well publicized models of ecosystems that collapsed and a few of systems that have recovered.
Try reading this article replacing “acidic inputs” with “toxic assets,” for example:
Terrestrial Ecosystem Recovery …. Modelling the Effects of Reduced Acidic Inputs and Increased Inputs of Sea-salts Induced by Global Change
http://ambio.allenpress.com/perlserv/?request=get-abstract&doi=10.1639%2F0044-7447(2003)032%5B0275%3ATERMTE%5D2.0.CO%3B2
This is just a hypothesis. In times of change (evolution), we fall back on what we know.
What do you think?
Hi Motkya, if this is a test of my mental acuity, I’m destined to fail. That said, let me try and answer your question.
There are many parallels between business and ecology. In fact, the whole field of behavioral finance tries to tie together human psychology with economics to determine why people make the decisions they do.
It’s extremely difficult to determine why an individual makes the decisions he/she does, and even more complex to model the interactions of several individuals if not thousands. Add variables concerning the external environment (war, weather, economic cycles, seasons, life events etc) and it can be argued that it’s nearly impossible to accurately model such complex systems. That does not, however, stop the quants from trying.
The main gist of my post is that while a model may attempt to explain and accurately represent the behavior of a system, and in most instances accurately predict that behavior, it’s foolish to completely base your decisions on the output of that model, especially because of the impact of fat tails. That’s one of the reasons we’re in this financial crisis–the over reliance on financial models by Wall Street firms in assessing risk. And since marketers are starting to work with financial analysts, and statisticians, there are lessons we can learn from the past fourty years of financial modeling.
I think you are on to something. For more on the growing interactions, interdependencies and interconnections in the marketplace, I’ll point you towards Richard Bookstaber’s “A Demon of Our Own Design”-a fantastic read.
Good questions that have stretched my feeble brain. Thank you for commenting on this post!
Great post, Paul. What comes to mind is that it seems that Wall Street allowed models to override critical thinking and common sense.
One thing that really bothers me was the removal of direct risk of providing a loan. These loans were going to be bundled and sold and so their was not incentive not to go nuts on the loan side and the crazy thing is people who bought these CDOs thought there was very little risk, too.
This model started a chain reaction of irrational thinking and suspension of common sense.
There were a number of economists warning of the dangers of all this several years ago but nobody would listen to them. They were derided and even mocked. I have seen video on youtube of an economist being laughed at on some sort of financial show for saying the thing was going to come crashing down.
Motyka, while I am not sure about the specific study you reference, there have been books written comparing ecological systems to economic systems and the parallels are absolutely fascinating.
The number of dependencies and variables are incredibly complex in both cases. I do not think anybody truly understands either.
Ecologist can gain some understanding of a limited ecosystem and economists of of micro-economics. But when it gets to the macro level, understanding seems to be especially limited in both spheres.
Neil, thanks for adding your comments. Securitization is a valuable function in the financial marketplace, but it was the magic formulas that were supposedly to assign risk levels to the tranches within the pools. The magic beans (and those that took them as gospel) played a pretty large role in the subprime debacle.
You have a good point though, when quasi-federal agencies (Fred, Fan) buy a loan no matter how poorly documented (or downright fraudulent), you have a recipe for disaster.
Paul,
Good post and food for thought…your words jumped out “blind faith and lack of critical thinking”
Somewhere, prior to Slumdog Millionaire: “It is written”… “They are blind guides…leading the blind”
Do these words reflect the times and answer the question: How did we get here?
“Critical thinking” is so lacking in our culture of everything instant…this “magic” mentality is pervasive and destructive.
To learn from our history we must all develop the skill of “critical thinking” and teach it to our children.
The things that will destroy America are prosperity-at-any-price, peace-at-any-price, safety-first instead of duty-first, the love of soft living, and the get-rich-quick theory of life. ~Theodore Roosevelt
Fay, I appreciate your comments on the need for critical thinking. For this particular situation (accepting the output of mathematical models as gospel), I would have been happy with any thinking at all!
Critical thinking in the ability to capture, synthesize, analyze, apply and learn from our mistakes is something that will never go out of style.