|
When a private company publicizes the results of its own research study, how do we know it's for real and not just self-promotion...?
As a contributing writer to The Business Journal, my editor recently asked me to write an article based on the results of a study conducted by a private training company. She had received a news release about the topic in question and I contacted the media contact - a publicist - who then turned me onto the company's principal.
The study's results were intriguing and would certainly have been excellent fodder for the meat and potatoes of the article. When I asked for a copy of the study, the principal sent me links to pages on the company Web site that showed the study's brief summary results.
When I asked for a copy of the full study (so I could extrapolate findings of my choice and to verify the existence of the study), I never heard back. Responses to my inquiry, up to that point, had been fairly swift, but once I asked for the full study, that was the end of communication.
Now, in all fairness, it's possible that something happened. Maybe the principal got kidnapped by a cannibal tribe on a business trip to a third-world country, or maybe his laptop got run over by a bus. Hey, I'm open to any number of simple explanations why I never heard from him again.
But, I admit to feeling duped. This study had apparently been mentioned in countless articles in major national business publications, so I thought it had to be bona fide. Maybe I'm being cynical, but what if it wasn't? What if it was a sham - a way for this company to promote itself with data many business people would find fascinating? What if the study was non-existent, or conducted with fewer participants than would typically be acceptable? I wonder, with all the studies and news releases that flourish at major publications, do reporters always fact-check?
In the end, I didn't use the study as part of the article. I couldn't ethically include the information without having some sort of confirmation that it was real. I did interview local experts on the subject and used other published statistics.
I'm curious to know what you think about this. Do you think I overreacted or was I right in passing on the study's results?
|
Comments
As a former journalist, I say without hesitation that you were exactly right. I would have taken it one step further and added the information about the company refusing to respond to your request for the study. That is news and the public deserves to hear it.
Posted by: Lewis Green | 11.03.06
As a skeptical consumer, I have always wondered about these types of studies. A company could set up the parameters in a way to sway the results. If you ran it against 10 other brands and you beat 6 of them it could always be published that your product beat 6 national brands for lets say germ-killing affect in a recent study. This I true; but the other 3 could have out preformed their product dramatically, but as a consumer you would never know.
So being a 40 year old beginning marketing student my question is how ethical it this.
Posted by: Jeff Lawrence | 11.03.06
Oh yeah forgot this. I am not a journalist but if love to stir things up and I would love to have seen the fact that they would not respond to your request published for all to see. Even though I would have felt bad if the whole cannibal thing did turn out to be true.
This is great stuff; I wish I had started blogging years ago.
Posted by: Jeff Lawrence | 11.03.06
It happens more often than we all probably realize. I heartily recommend the old classic, "How to Lie with Statistics" by Darrell Huff.
Posted by: Cam Beck | 11.03.06
Elaine was the study for sale? Maybe they didn't want to give it away?
Posted by: Mack Collier | 11.03.06
Hi Elaine. I did quite a lot of research for different companies and I can assure you that I have seen some pretty bad work until now.
Well, if I even put it further, most probably in over 50% of cases people just did some kind of a questionnaire based on few questions they have been asking themselves. Methodology, sampling and interpretation is often something they don't want to "waste time" about. So mainly they just want to get the "correct" findings.
From my point of view, as being a researcher for a while, I don't believe ANY research where I don't know the methodology and the way it was done. And even then I try to interpret the results my way, not the way the researcher interpreted them.
So yes - you did the right thing. :-)
Posted by: Dusan Vrban | 11.05.06
Thank you all for your comments. I am convinced now that I did the right thing.
Lewis, I'd be reluctant to blow the whistle on this company for two reasons: one, I never attempted to contact the principal a second time, and two, I'd need to understand the legal ramifications.
Mark, as far as I know, the study wasn't for sale. I called in my role as a business journalist for The Business Journal, after the company's publicist sent a news release. They were obviously looking for PR coverage and the study was the catalyst.
Dusan, I'm not a specialist in market research, but what you said makes total sense. I, too, am dubious of private studies that are self-serving. Can we really believe the hype that milk or butter is good for us when the stats are based on internal studies done by the dairy board? How about pharmaceutical studies? That in itself, is a whole different ballgame.
I think journalists need to be extra cautious when it comes to these types of private studies, otherwise, the public can be lead astray on any number of issues. Unfortunately, we are all susceptible to believing what we see in print.
It also raises the question, too, of consumer goods and internal studies. If a product is that good, I'd like to see an independent study that shows its value or benefit. Now, that would be a lot more convincing.
Thanks again for your feedback.
Posted by: Elaine Fogel | 11.05.06