Comments on: No Magic Beans, No Magic Formulas

By: Paul Barsch

Paul Barsch — Fri, 13 Mar 2009 21:45:17 +0000

Fay, I appreciate your comments on the need for critical thinking. For this particular situation (accepting the output of mathematical models as gospel), I would have been happy with any thinking at all!
Critical thinking in the ability to capture, synthesize, analyze, apply and learn from our mistakes is something that will never go out of style.

By: Fay

Fay — Fri, 13 Mar 2009 20:17:20 +0000

Paul,
Good post and food for thought…your words jumped out “blind faith and lack of critical thinking”
Somewhere, prior to Slumdog Millionaire: “It is written”… “They are blind guides…leading the blind”
Do these words reflect the times and answer the question: How did we get here?
“Critical thinking” is so lacking in our culture of everything instant…this “magic” mentality is pervasive and destructive.
To learn from our history we must all develop the skill of “critical thinking” and teach it to our children.
The things that will destroy America are prosperity-at-any-price, peace-at-any-price, safety-first instead of duty-first, the love of soft living, and the get-rich-quick theory of life. ~Theodore Roosevelt

By: Paul Barsch

Paul Barsch — Fri, 13 Mar 2009 14:07:00 +0000

Neil, thanks for adding your comments. Securitization is a valuable function in the financial marketplace, but it was the magic formulas that were supposedly to assign risk levels to the tranches within the pools. The magic beans (and those that took them as gospel) played a pretty large role in the subprime debacle.
You have a good point though, when quasi-federal agencies (Fred, Fan) buy a loan no matter how poorly documented (or downright fraudulent), you have a recipe for disaster.

By: Neil Anuskiewicz

Neil Anuskiewicz — Fri, 13 Mar 2009 00:40:13 +0000

Motyka, while I am not sure about the specific study you reference, there have been books written comparing ecological systems to economic systems and the parallels are absolutely fascinating.
The number of dependencies and variables are incredibly complex in both cases. I do not think anybody truly understands either.
Ecologist can gain some understanding of a limited ecosystem and economists of of micro-economics. But when it gets to the macro level, understanding seems to be especially limited in both spheres.

By: Neil Anuskiewicz

Neil Anuskiewicz — Fri, 13 Mar 2009 00:29:20 +0000

Great post, Paul. What comes to mind is that it seems that Wall Street allowed models to override critical thinking and common sense.
One thing that really bothers me was the removal of direct risk of providing a loan. These loans were going to be bundled and sold and so their was not incentive not to go nuts on the loan side and the crazy thing is people who bought these CDOs thought there was very little risk, too.
This model started a chain reaction of irrational thinking and suspension of common sense.
There were a number of economists warning of the dangers of all this several years ago but nobody would listen to them. They were derided and even mocked. I have seen video on youtube of an economist being laughed at on some sort of financial show for saying the thing was going to come crashing down.

By: Paul Barsch

Paul Barsch — Thu, 12 Mar 2009 20:32:27 +0000

Hi Motkya, if this is a test of my mental acuity, I’m destined to fail. That said, let me try and answer your question.
There are many parallels between business and ecology. In fact, the whole field of behavioral finance tries to tie together human psychology with economics to determine why people make the decisions they do.
It’s extremely difficult to determine why an individual makes the decisions he/she does, and even more complex to model the interactions of several individuals if not thousands. Add variables concerning the external environment (war, weather, economic cycles, seasons, life events etc) and it can be argued that it’s nearly impossible to accurately model such complex systems. That does not, however, stop the quants from trying.
The main gist of my post is that while a model may attempt to explain and accurately represent the behavior of a system, and in most instances accurately predict that behavior, it’s foolish to completely base your decisions on the output of that model, especially because of the impact of fat tails. That’s one of the reasons we’re in this financial crisis–the over reliance on financial models by Wall Street firms in assessing risk. And since marketers are starting to work with financial analysts, and statisticians, there are lessons we can learn from the past fourty years of financial modeling.
I think you are on to something. For more on the growing interactions, interdependencies and interconnections in the marketplace, I’ll point you towards Richard Bookstaber’s “A Demon of Our Own Design”-a fantastic read.
Good questions that have stretched my feeble brain. Thank you for commenting on this post!

By: Motyka Johnson

Motyka Johnson — Thu, 12 Mar 2009 19:52:41 +0000

What I’m proposing is a bit of a stretch, but please read through this entire post before passing judgement.
I’ve done some work in the past studying Ecosystems and the models that govern relationships between organisms in these very complex systems. I see many parallels between ecology and business. A basic obvious parallel is that all IBUs have the same mission: survival (Where organisms need food, water, shelter, space IBUs need profit margins and a defined market environment). Just like all organisms in an ecosystem. In many ways, I think of companies like species in the system. Each ecosystem has keystone species, their well being indicates the well being of the entire system. In the US economy, our keystone species are creditors and lenders.
I think the models exist for the current market situation. There are several well publicized models of ecosystems that collapsed and a few of systems that have recovered.
Try reading this article replacing “acidic inputs” with “toxic assets,” for example:
Terrestrial Ecosystem Recovery …. Modelling the Effects of Reduced Acidic Inputs and Increased Inputs of Sea-salts Induced by Global Change
http://ambio.allenpress.com/perlserv/?request=get-abstract&doi=10.1639%2F0044-7447(2003)032%5B0275%3ATERMTE%5D2.0.CO%3B2
This is just a hypothesis. In times of change (evolution), we fall back on what we know.
What do you think?

By: Dusan Vrban

Dusan Vrban — Tue, 10 Mar 2009 19:13:27 +0000

What I like (hate?) about this modelling is that 1 small variable (most probably even unknown as you pointed out), can:
1. Change the complete weighting system and
2. Change the complete interaction system (1 variable doesn’t make effect ona another anymore, even tough it did before)
So we have a complex model that actually isn’t static. Perfect formula of today doesn’t work tomorrow (as your post excellently pointed out).
Consumer behavior and databases. Art&science indeed!

By: Paul Barsch

Paul Barsch — Tue, 10 Mar 2009 18:57:04 +0000

Hi Nick, thanks for adding your comments! There are way too many culprits in this mess to pin the blame on one person or entity, although there are some players in the game that have a larger role (see Bernanke’s mea culpa today in WSJ).
Our global financial system is tied together in very complex and sometimes hidden ways. Markets, companies, assets are all intertwined now, so it’s not just a national challenge, but a global challenge.
A good lesson in that risk lies not only inside our corporate walls (i.e. VAR and the 4:15 report), but increasingly– outside as well.

By: Nick Stamoulis

Nick Stamoulis — Tue, 10 Mar 2009 17:58:29 +0000

There is so much finger pointing to do is now it is silly. To many people involved. Our government needs to put their head together and find a solution.

By: Paul Barsch

Paul Barsch — Tue, 10 Mar 2009 16:54:53 +0000

Dusan, thank you for the compliment!
As you rightly pointed out, modeling complexity is extremely difficult. For example, I know of one financial services company whose fraud detection model has over 2,000 variables. And let’s suppose you get the variables right (and don’t miss any–which is not likely), how then do you properly weight them?
Modeling human behavior – whether it’s propensity to buy something, churn off service, file a claim, commit fraud, perform as expected etc is both science AND art–and definitely not easy.

By: Dusan Vrban

Dusan Vrban — Tue, 10 Mar 2009 16:35:49 +0000

Hey Paul, amazing as always. Got me thinking back to my University degree where I’ve studied the demand prediction models. It turned out that there’s a matrix of relationships that you need to “calculate” if you want to be accurate.
And the matrix goes from legal, demographic, economic, technological and other “environments” down to the companies, suppliers, customers, shareholders,… And they all interact between them constantly. There’s a butterfly effect as in weather models.
It’s just (too?) complex to predict.
One of my favorite qute was something like this: if the environment is stable, the prediction is simple (add 10% to last year). If the environment becomes unstable, there’s no way you can calculate it.

By: Paul Barsch

Paul Barsch — Tue, 10 Mar 2009 14:11:52 +0000

Claire, thank you for taking the time to comment.
Is a lesson here, ‘if it’s too good to be true, it probably is’? Another lesson, which everyone knows, but we conveniently forget (especially when the $$ roll in) is the correlation/causation fallacy.

By: Paul Barsch

Paul Barsch — Tue, 10 Mar 2009 14:03:51 +0000

Lewis, I appreciate your comments. This article really highlighted the disconnect in many companies (not just in financial services) between the quantitative analysts and those on the business side. The mathematicians understood “the math” perfectly, but they had no way of knowing that their formulas would be taken to such an extreme and used in unpredictable ways.
And those on the business side never questioned the math, never questioned the assumptions behind the model.
There are no magic beans, there are no magic formulas.

By: Claire Ratushny

Claire Ratushny — Tue, 10 Mar 2009 14:01:19 +0000

Excellent post, Paul. My understanding, after the housing bubble burst, was that very few people even understood CDOs or derivatives at all. So my question is this: how could any bankers or other financial institutions risk buying these up without understanding the formula in the first place? Supposedly, only a small handful of people even understood Li’s formula. Even then, does it seem logical to be able to assess risk when hundreds of mortgages are lumped together in one large pool? Think about it. Hope a hard lesson has been learned here so we don’t repeat the mistake of letting the potential for big profits–and greed–get in the way of making sound decisions.

By: Lewis Green

Lewis Green — Tue, 10 Mar 2009 13:47:16 +0000

Paul,
This is an excellent analysis and seems to be right on target, based on my reading of Li’s system. His formula produced numbers that bankers liked, so they turned off their brains and made decisions they otherwise would have avoided.