Despite the logic of using data to complement or drive decision making, the business and mainstream press continue to glorify intuition and “gut” decision making by managers of all stripes. Where does this leave “data-driven” approaches?
A recent article in Fast Company titled, “Going for the Gut,” details how even though we like our “heroes to crunch the numbers, we (also) like them to play their hunches.” Author Rob Walker laments that books like Malcolm Gladwell’s Blink, Jack Welch’s Straight from the Gut and others, give gut decision making first billing over a “careful, rational, empirical” approach.
Walker asks, “Are the narratives of popular culture dominated by super rational heroes triumphing over seat of the pants, gut-trusting bad guys? Actually, it’s the opposite: from Captain Kirk to Indiana Jones to Rambo to Tony Soprano–we’re drawn to the character who follows the hunch and wins.”
And business press and mainstream media largely agree. After all, wouldn’t you rather read about the business executive who had the right hunch and made millions, as opposed to the quant-jock who crunched the numbers and came up with the winning combination?
Gut decision making is “in” and for lack of a better word–cool.
Some senior executives have alluded there’s a mystique to gut decision making–those who have it have it, and those who don’t will never ascend the ivory tower of business success.
Case in point, Ralph Larsen, former CEO of a large consumer product goods company, states in a Harvard Business Review article, “When to Trust Your Gut,” that “very often people will do a brilliant job up through the middle of management levels, where it’s very heavy quantitative in terms of decision making. But then they reach senior management, where the problems get more complex and ambiguous, and we discover that their judgment or intuition is not what it should be.”
I do agree with Mr. Larsen that data driven decision making works best when there is in fact “data” to analyze. Sometimes, senior level decisions can be challenging because situations might be in uncharted territory–and there’s no past data, or too small a data set for analysis or prediction. That said, I’m not convinced that good judgment and intuition is the sole purview of senior management.
There is hope, however, that a data-driven approach can work just as good as gut thinking, and in most instances–compliment it.
Recent books, “Super Crunchers” by Ian Ayres and “Smart Enough Systems” by Neil Raden and James Taylor show case study after case study where companies and government entities of all sizes are using data analysis, experimentation and analytical applications to make smarter and faster decisions.
And the business press also seems to be noticing. In the same Fast Company issue (November 2007) executive Matt Carey cites how he’s attempting to create a “culture of analytics” at eBay, where experimentation, data and testing rule. “I want to eliminate feelings,” he says, “and get down to true math.”
If 2007 was “The Year of the Gut”, perhaps 2008 will be the “The Rise of The Quant Jock.”
Well, “quant jock” might not be the best terminology, and “analytics” might not be the sexiest noun, however running the company and making decisions based on the hard facts and numbers is something that I believe will never go out of style.
* Have you seen other examples of the “glorification of the gut”?
* How is decision making approached in your company (and in your marketing department)–by the numbers or by the gut? Do you see a trend one way or another?
Related posts:
- Decisioning in Volatile Times–Probability, Intuition or Inaction?
- The Moneyball-itzation of Marketing
- The Perils Of Intuition
- Can Mathematical Modeling Be Trusted?
- Social Network Analysis: Hype or Help?
Tags: analytics, data driven, decision making, empirical, evidence, experimentation, fact based approach, gut thinking, intuition, Paul Barsch, quant jock, rational
Hmmm. Interesting question. It is one I faced in the late 70s when the business I started was growing fast. I needed to hire so I thought that I should read some business books (I was on the creative side) to see what the experts said. What they said at that time was never trust your gut. So, I stuck to the facts on my next two hires. They were the worse hires in my career.
After that, I started getting the facts but following my gut. The reality is that most often gut feelings are based on facts. The facts you obtain through your life experience, similar situations and from what you read, hear and see. We are given this trait so we can survive.
I say get the facts and follow your intuition. It is interesting that the examples of “heroes” you mention Captain Kirk , Indiana Jones, Rambo and Tony Soprano all are extremely experienced people who are well trained in their craft. So their “gut” feels are more than decisions based on undisciplined over-reaction.
Next time my spaceship is attacked by hostiles from Centuria Prime I want Kirk and his “gut” feeling running the show. There just might not be enough time to run a data analysis of the situation before we are vaporized.
I understand where you’re coming from on your post, Paul. In business, as in society, we seem to be increasingly making decisions on emotion, rather than considering the facts or data. Businesses need to quash that as new people enter their companies and teach them to study the available data, provided it’s good information, of course, and make decisions based on that data. Having said that, as people move through corporate echelons, they intuit much from experience, as Harry pointed out. For my part, I believe that all of the data in the universe has to be balanced by our gut in our decision-making processes–once we really understand our businesses. Rarely is anything totally black and white. Thanks for a thoughtful post, Paul.
Interestingly enough, Major league baseball front offices used to rely on scouts and thier gut feelings on baseball prospects. But what has happened lately is that they are relying more on statistical information in making baseball decisions because, the data don’t lie. No matter what potential they might see.
Harry and Claire, thank you for commenting. Following your gut (intuition and experience) is a valid decision making process, I’m just for complementing the process with a data-driven approach.
There are some instances where a data driven approach will invalidate our gut thinking, and othertimes there may not be enough time or data to validate our gut thinking.
In general, for the business and popular press, I’d like to see more glorification (case studies and articles) of “thinking by the numbers”.
Direct response marketer, you make a good point. I almost forgot to include a mention of Money Ball, by Michael Lewis which details how the Oakland A’s turned around their small market franchise by focusing on a data driven approach to prospecting and signing baseball talent.
Not to mention what Theo Epstein has done with the Red Sox in 4+ years…
I should have added one other thing, Paul. The closer the marketer is to the customers, and having a direct conduit to them and speaking with them is invaluable, the better and more reliable your gut intuition becomes.
And as you just articulated, Paul: “In general, for the business and popular press, I’d like to see more glorification (case studies and articles) of “thinking by the numbers”. I can’t argue with that statement at all.
Claire, in the B2B space it’s much easier (as you point out) to do account based marketing and get a better pulse on customer needs.
Data driven approaches work well in B2B world, but really take off in the B2C world (volume based businesses).
. . .”Not to mention what Theo Epstein has done with the Red Sox in 4+ years…”
Thanks for the mention of this, Paul. We’re not breathing in New England right now–not until the Red Sox win the World Series. Here’s hoping with all of our might. . .and while I’m at it: here’s hoping our beloved Pats win the Super Bowl, too.
Paul,
Thought-provoking post. I am in Harry’s camp on this; however, we also look at the data, when appropriate. At the end of the day, however, if my experiences and accrued business knowledge add up to something different than what the data says, I’m going with my gut every time.
Lewis, thank you for commenting.
Decision making is both art and science. I argue that in the business world, too often decision making is based on experience and intuition, as opposed to as Jim Collins would say, “letting the data speak.”
From my previous post about the advance of technology (specifically decision support) and the fact that data volumes are increasing in some cases exponentially, I believe “thinking by the numbers” will take a larger piece of the decisioning pie.
That said, there will always be a place for “gut thinking.”
Good post, Paul. I actually think you end up making an eloquent case for acting with both factual data analysis and gut thinking. Yes, business should give more weight to using solid data when making decisions, but as everyone seems to have concluded, there are definitely times to go with one’s gut instincts. However, as you point out, shooting from the hip and going with one’s instincts without understanding what the data is telling us, can lead to disastrous results. Thanks for reminding us about that.
I think even when people say “gut” feeling they do refer to educated intuition and some form on prior knowledge base to use to make a decision. I have always found that “gut” decisions are effective but always know that there is a mountain of knowledge or experience behind it.
Nowadays there are so many data points and so much data around that it can drag you and your decisions to painful lengths.
People often get bogged down with data and sometimes lose the big picture.
Having said that, data is becoming more plentiful and more targeted and more to the point – therefore ignoring data is not necessarily the best strategy.
I truly believe that the question should not be whether to use data instead of gut or visa versa, the real question is how do you integrate both into good decision making process.
Aline and Ted, thank you for commenting.
Aline, gut decision making can be based on experience, but in some instances it can also be based on superstition, “it just feels right”, or a physiological reaction like back spasms or headaches. I’ll even throw in divine guidance for those who believe in it!
You do, however hit the nail on the head that the best approach is a complimentary joining of intuition/experience and data-based decisioning. They’re not mutually exclusive.
Paul, the advice we give to our clients is “collect data, but trust in your gut”.
I believe that data alone is never enough. As we become better and better at collecting and analyzing data, the data becomes more and more valuable. However, the “gut” we talk about is really the unconscious mind. The mind that can see, feel and analyze information that we can’t collect in a spreadsheet or database.
We will never be able to collect enough data to make data-only decisions, so why think we can make data-only decisions. We should make decisions based on good data, combined with information from our unconscious mind.
If you’d like to see more of my thoughts on this, you can visit this link to my post on “Avoid Statistical Significance”
http://www.achievemarketleadership.com/?p=60
Glenn, thank you for commenting.
I agree with you that in most instances, a combination approach of intuition and data-based decision works best. There are, however, many instances where computers can find hidden patterns and analyze data much better than the human mind. Specifically I am thinking about fraud detection (combing through millions of financial transactions), scanning call detail records (CDRs) to find patterns, point of sale data for product affinities etc…).
Very thoughtful and provocative post, Paul. And great discussion.
Considering the decisions most marketers make and the battles we fight INSIDE our organizations, I believe we have to be transparent and systematic in our decision making processes. This requires facts and data.
The reason Finance has more prestige and influence in organizations than Marketing is that the analysts put numbers in a model and the computer spits out the answers.
Yes, financial models are full of assumptions, which the “gut” produces, but ultimately there is a repeatable and quantiative process that leads to repeatable decision making.
We marketers need more, not less, of this.
Of course, much in business is intuitive and many of the great business leaders make decisions from the gut. We also tend to motivate others with emotion and images. So, when we communicate or argue for the support we need, we often need to leave the data behind and move to metaphors and images that resonate with our target market.
The Heath brothers in Made to Stick have great lessons on this.
Still, the bottom line is most marketers searching for power, influence and business impact today need facts and data to make convincing arguments that will convince others we are right.
Roy, thank you for weighing in on this subject. Like you, I believe marketers would have more power and influence in the organization if we could back up our gut feelings and intuition with hard facts/numbers. It’s important to get away from, “I think this will work”, or “it just feels right,” to… “I’ve reviewed the data, ran the numbers and here’s my take…”
I think there’s room for both, and different approaches work for different people. It’s important to have the numbers to back up your decisions, but it’s equally important, I think, to pay attention to that inner voice that tells you that something is right or wrong with what you’re thinking of doing.
In my business, I do a lot of intuitive thinking – what direction feels best, where should I go next, should I attend that networking event? But I also follow and keep track of what works, and use that to inform my decisionmaking.
Interesting discussion; for the past few years my partners and I have bought a great deal of commercial real estate. He has arrived at his decisions through “back of envelope” and gut instincts. I am very quantitative and built very sophisticated analytics tools to help with investment decision making.
Success? We were most successful on those projects when we effectively complemented each other’s styles.
I’d be nowhere without his “gut” and, I suspect, he’d tell you his decision making would be substantially less confident without my decision support tools.