Questions? Feedback? powered by Olark live chat software

5 Tips for Writing Survey Questions that Don’t Yield Statistical Garbage

Thursday is guest post day here at Duct Tape Marketing and today’s guest is from Josh Pigford – Enjoy!

Survey Question GarbageWhile the end result of a survey might make the conclusions look cut and dry, there are many ways that data can be manipulated or misrepresented to change the truth. However, the sloppiest (and probably most common) method of fumbling the truth is when data is simply misunderstood.

Even if your survey questions sound great, if they’re not credible they won’t produce valid results. In case you don’t know the difference between discriminant and regression analyses (really, who does?), this post will cover five tips to make sure that your sweet survey doesn’t turn into a statistical bummer.

1. Think First, Ask Second

Think broadly about why you’re creating a survey. What are you really trying to figure out? When you have a clear idea, make a list detailing the kind of information that you’re setting out to look for. Now you can begin to write your questions, always keeping in mind that they must match your original informational targets.

If you realize after collecting all your responses that your questions are actually asking something rather different from what you originally intended, then your data will also be telling something rather different from what you intended. You’ll have to settle for either “different,” or a big lie.

2. Ratings vs. Rankings

Depending on the kind of information you’re looking for, you can either ask your respondents to rate or rank several items in a list. However, it’s important to realize the difference between these two types of questions.

A ranking will only tell you which items are more or less preferred relative to each other, but you won’t actually know from a ranking if a respondent likes or dislikes any items. For this latter purpose, you must use a rating question.

3. The Multiple Choice Golden Rule

Most surveys depend heavily on multiple choice questions since prepping for standardized tests have wiped out this country’s ability to formulate an original answer. Or we’re too lazy. Either way, the possible responses to any multiple choice question must be mutually exclusive. This means that no two answers could equally serve as appropriate responses.

Not only do non-exclusive answers annoy people (you’ve made them think too hard!), but they’ll make accurately analyzing your data nearly impossible. If someone could choose one of two answers and feel good about either response, you won’t be able to determine the respondent’s actual preference.

4. Surveys Are Not Like Airplane Exit Rows

More “legroom” in your question doesn’t make it better. You wrote your survey with a specific purpose, so make sure your questions are direct without giving your respondents too much leeway in answering.

If you want to know how to make your company’s logo look more cutting edge, make sure you specify your desire for responses regarding the logo’s impact, not simply ask about the company in general.

5. Don’t Get Too Excited in One Question

Surveys are awesome, we know, but that’s no excuse for asking more than one question at a time. Each question you ask needs to be aimed at collecting one unique point of information, or else you’ll end up skewing your data by mixing results that should be separated.

It’s even possible to get so excited that in a fit of survey-exuberance you accidentally put two contradictory questions together. That’s not only pretty weird, but it will also void the results for that part of the survey.

Statistics and the wild field of data analysis include another laundry list of Dos and Don’ts, but hopefully these five tips can get you started on creating surveys that also produce credible results.

If you’re ever in doubt about how a question comes off, ask some friends to test it out. If you don’t have any friends, well then you’ve got bigger problems than that tricky survey question.

Josh PigfordJosh Pigford is co-founder + CEO of  PopSurvey, where they’re building online survey software to try and make the survey industry a little less coma-inducing. They’ve got a huge collection of survey templates to help you get started with the click of a button!

Stumped for content?

Yahoo AnswersContent is the commerce of social media. But, relevant content, the stuff your customers and prospects actually want to know is the gold. Constantly cranking out this gold can sometimes present creative challenges.

So how do you know what to write about, how do you keep the ideas flowing, how do you keep the content relevant and popular.

There a number of ways to address these important questions, not the least of which is simply paying attention to what your prospects and customers ask on the daily basis. There’s another tool that not many small business know much about, but is a wealth of great content ideas.

The service is called Yahoo Answers. Millions of people go to Yahoo Answers to get answers to questions on just about every topic known to man. Yahoo members are free to answer these questions and then even earn points for participation and developing a reputation for good answers. I’m not sure working your tail off answering all kinds of questions is the best use of your time (although you may actually come across opportunities to interact with people looking to buy what you sell) but, I have found that the questions posed on your particular subject of expertise can provide some tremendous research for blog topics. If a number of people are asking the same questions, and they are, then maybe the market is ripe for your blog post addressing the question.

The questions are broken down into many searchable categories. In addition, if you find the occasional question that you have already thoroughly addressed on your blog you can step up and provide an answer on the Yahoo Answer page and cite your blog post as the resource for your answer. If your answer is considered particularly helpful, this can send some additional traffic (no link juice though) your way.

What’s the key strategic behavior of your ideal customer?

Marketing behaviorLast week at this time I sat down to write a post about targeting and it turned into quite a research project all by itself. It started innocently enough by me asking if what your ordered at Starbucks or the local pub said something about you that a marketer could use. (The comments are way better than my post)

The simple answer is, of course it does. And if you could look inside a prospect’s car (of course just knowing the make and model would reveal something) and see that clutter you would know even more, if you could visit your prospect’s house and have them tell you about their favorite outfit or shoes, you would know even more, if you could spend a few minutes reading what’s on your prospect’s refrigerator, you would know even more.

The point is that your prospects and customers do things, act in certain ways, that can help you identify them as ideal prospects and customers. The trick is to pay attention closely enough to see the behavior that I call their key strategic behavior. Discover it and it’s like getting them to raise their hand to be called on.

Now, in all likelyhood, if were able to do the kind of research mentioned above you might actually learn more than you want to know, but it’s also completely impractical to think that you could ever gain that kind access to your customer’s behavior.

But, the good news is that there are very public things that your prospect’s and customers might do that can be great indicators if you start to analyze them.

    For instance:

  • I discovered long ago that business owners who participated at an officer level in local and national organizations related to their industry are more likely to have an interested in long term solutions rather than quick fixes.
  • I discovered that business owners who actively participated in groups like Rotary are more likely to embrace the idea of referrals and networking.
  • I discovered that business owners who sought out professional, but perhaps a bit more entrepreneurial, service providers such as CPAs, attorneys and bankers, were more willing to explore innovative approaches.

I know these are pretty broad generalizations, but they are fairly accurate markers of behavior. They helped me identify a prospect I knew I could work successfully with in a matter of minutes. So, take a look at your current ideal customers with an eye on identifying your key strategic behavior, something you can know about them, that is also a pretty good indicator of what type of customer they might be. If you can get this step right it can change who your target and what you say in all of your communications.

Dig deep in this exercise and get a bit creative, think club membership, community participation, hobbies, reading likes, Starbucks drink, even pet ownership. I once had a customer that came to the conclusion that all of her ideal customers owned big dogs – it’s not really that far fetched of an idea! (puns are always intended here.)

So, what behavior have identified as telling from a marketing perspective?

What would you Google if?

Finding the right mix between marketing focused messages and customer focused messages is a real art and, in my opinion, at the heart of creating a powerful marketing strategy.

Interviewing your existing customers is one of the best ways to uncover this mix and find out what your customers really value, really want, and in the case of online search – what they really look for when trying to find someone that does what you do.

Google Local MapsThe customer interview process, interviewing our customers’ customers, is a system used by the Duct Tape Marketing coaches as part of the strategy planning process and Australian coach Joe Bowers and Canadian coach Elizabeth Walker took this a step further by actually inserting the interview question – “What would you Google if you were looking for this kind of services/products?”

Try it sometime with your customers and I think you will discover some potentially powerful keyword phrases that you may not be optimizing your web pages for.

What does your drink say about you part 2

Tullamore DewI wrote a post a few days ago that sparked an uncanny number of replies. Essentially, I made a brief point about whether or not your coffee drink order from a Starbucks would give a marketer clues about who you are.

Well, more than one reader pointed out the fact that they had another kind of drink in mind altogether when they read this post and wondered what might your choice of adult beverage say about you. I think the question is a valid one and so I pondered it over an Irish Whiskey myself just last night. By the way I am writing part 3 as we speak, no it’s not what kind of gravy are you, it’s my original point about determining behavior that allows you to identify your ideal customer – stay tuned on that.

But for now, I must confess I’m a Tullamore Dew and water (a nod to my Irish ancestors I guess.)

So, what drink are you?

What does your drink say about you?


StarbucksMarketers spend a lot of time dissecting the various demographics and psychographics of target markets and I’m wondering if you could do a study of what all your customers order at a Starbucks, or if they even frequent such a place, and come up with appropriate marketing messages.

For the record, I’m a grande coffee with 2 shots of espresso added – the fact that people call this drink names like a depth charge or red eye ought to tell you something. (More on this when I’m back at the office)

So, what are you?


Frequency matters

The term frequency is often related in marketing terms to the number of times you run an ad or send out a mailing to a certain list, but I think it has another, perhaps more important, meaning for the small business.

Frequency, in quantum physics circles, is something like the vibration rate of a wave or particle. Okay, I’m not going to win any Nobel Prizes for the sciences, but the idea further goes to suggest that atoms, particles, and stuffs in general are attracted to those with like frequency.

Have you ever entered a room and noticed someone you’ve never met, but instantly connected with – it may be looks, but more often it’s just something that you can’t really put your finger on. There are those that would call this a connection based energy or like frequency.

I think this phenomenon, regardless of how deep you want to dive into this, is true for your marketing. Every business, every person, every marketing contact emits a frequency that attracts whatever you have in your business.

Think about it, marketing is all about attraction. So, the question is, who have you attracted to your business to date? Are they the right customers, ideal, or are they all wrong? Do you have any idea what actually makes, and therefor what does not make, the perfect customer for your business. Why aren’t you attracting enough of them.

My bet is that if you are not attracting the ideal customer, then you are sending the wrong message, the wrong energy, either intentionally or accidentally – your frequency is jambed. Your vibe is out of tune.

I know that can sound a little goofy, so let’s use a phrase that, to me, is the same thing – your brand is out of alignment.

Muddy brands happen because small business owners either don’t know or won’t commit to narrowly defining their ideal prospect and setting their business up to communicate authentically with that very narrow target. If you can take this essential step you will find this whole “law of attraction thing” that’s been getting a lot of attention of late is grounded in practical business application.

Let Your Customers Build Your Business

I read somewhere that Intuit, the makers of QuickBooks and Quicken, have a practice they call “follow me homes” that allow them to go into an actual customer’s home and watch them install their software and get it set-up. They find that often, the way they designed a screen or instruction may not be the way a user actually interacts with it.

It’s pretty easy to see how this kind of user testing could be applied to software, but what about your situation?

This kind of testing could be applied to many aspects of a small business but I’m going to coin the term “Build a Biz” for this approach.

    Here are some examples of how you might use this:

  • Getting a new logo? Let your best customers have a say
  • Writing a new brochure? Same thing
  • Web site? Watch over your customer’s shoulder as they surf around
  • Product/Service design? Before you role something new out get your client’s feedback
  • Customer service processes? Let your customer help design them

This could make for a very cool annual or semi-annual community building event. Bring a group of your most passionate customers together for an entire day of “build a biz” and have them rotate from workstation to workstation completing a series of tasks. You could build in networking, learning, and socializing throughout.

Every participant will go away feeling great about being a part of the game, you will have learned a tremendous amount about your business, and, if your best clients just happen to be business owners too, you just might want to take your new game of “build a biz” to them and show them how to get better too. Nothing adds value to your business relationships like this kind of community and value building tool.

How would you bring your clients into this game?