Oy vey! What's even worse is when a client shoves the results of misconceived research under your nose and insists it's a meaningful guide for marketing decisions. I once had a research "professional" show me a program in in which interview subjects were asked to choose the most appealing positioning statement among three hopelessly meaningless options. Each was as vague as the next, built on abstractions such as "quality," "value" and "expertise." Naturally, the results were a dead heat: 33 percent, 33 percent and 34 percent. The coup de grace? The researcher insisted that the 34 percent vote represented a "statistically meaningful distinction" and that we should build our marketing on that position!

Great post, Allen. The other problem with surveys occurs when questions are phrased in such a way that they elicit the "desired" responses that the interviewer is seeking.(No personal influencing there, of course) How meaningful are those surveys when shared with marketers?

Allen - you are absolutely correct. The problem is that too many marketers like to use these higher-order words to try to convey a broad, all-encompassing meaning. BUT, those higher-order words cannot be meaningfully measured in a survey questionnaire because they mean different things to different people. Even the word "reliability" is not meaningful - to an engineer it means "mean time to failure", but to a social scientist it means that results can be consistently replicated. As one of those who try to design these types of surveys to be meaningful, I'm constantly fighting corporate execs who want to "fuzzy" it up. After 33+ years, I've come to the conclusion that most marketers either can not, or do not, want to deal with the specifics of product performance. When you try to pin them down, they get seriously frustrated - Okay everybody, it's agreed - we shoot the consultant!

All cliched words. Leftovers from the dinosaur era.

Every now and then when I see surveys like that I wonder if the intended purpose is to simply make the consumer think that they actually care about those broadly-defined words, as opposed to actually gauging consumer response.

P.S. One of these days I'm going to actually press "post" before I realize that I've put my email password where the URL goes.

Funny thing is that many times these "traditional" surveys. (I am talking about the ones they send you by mail) get filled out by the little kids in the household.

If you want real data, give them a survey at the end of the shopping process. (where applicable)

Thanks for the heads up. My boss just put me in charge of creting a survey to send out to new customers, and this will definitely help me out. Our company always says it has great quality, but your right, what does that mean to the average person. I'll have to put more tangable aspects to the questions then just "did you think the finished product was of high quality." I'm realy going to have to think about this project before giving it to the website guy to post.

Thanks all. I've never liked the words value or quality since they are so broad and meaningless. Anyway, to make matters worse, this survey asked me to pick 2 charatertistics to describe each product. Imagine how hard that would be. I gave up on the survey and eventually started answering in a random fashion...something you don't want your respondents to do.

I guess what Mr. Twain said about the difference between the right word and the almost right word being the difference between lightning and a lightning bug, holds true for surveys as well.

Rant on! I am totally with you on this!

Funny, Mike! Great minds think alike, I guess...when I read Allen's post earlier I was reminded of that quote, too (which is famous in editorial circles, for obvious reasons).

The other thing I was reminded of was advice my first journalism teacher (Charlie Ball, at Simmons College in Boston) gave his class: Never use a dollar word when a ten-cent word will do. Inherent in that lesson was that fancy-pants words tend to be open to interpretation. In daily journalism (and in surveys), simpler is better.

Marketing speak. And people are fed up with it. It *IS* meaningless and has almost no relevance to the average person.

And this is why the Ad/Marketing industry is ripe for a major shake-up.

I'll add my two cents of agreement on this one. Use language that average people recognize and understand easily. Plus, I think this practice permeates into other areas - like marketing copy. Hey, good subject for my next blog post!

For downloadabale questionnaires you might want to check out the Survey Questionniare Archive at:
www.cadsr.udel.edu/sqa

The Survey Questionnaire Archive is an open collection of survey questionnaires used in social sciences and public policy making.

The archive is created, hosted and maintained by the Center for Applied Demography & Survey Research at the University of Delaware.

This digital collection allows users to browse, search, store and share survey instruments over the web.

The goal of the archive is to collect, capture, disseminate and preserve a wide variety of survey instruments.