The Perils of Using the Internet for Surveys

As one who is involved in trademark infringement litigation surveys, I have discovered the Internet as a means of probing the marketplace to learn if consumers are confused about various products or services.

There are a number of ways to find this information. The most often used ways – other than the Internet – are mall intercept surveys and telephone surveys. Both have advantages and disadvantages.

Mall intercept surveys work best for consumer package goods and other branded products. This method allows for highly comprehensive screening. Costs are manageable and depending on the length of the survey, respondents can be recruited for a incentive fee as low as $5.00 per interview. Consumers can be shown actual photos, photos of actual products and a shopping environment can be simulated.

Mall intercept research takes place in certain shopping malls where research centers are located. Respondents are screened and recruited from the mall and brought into the research center where they are asked the survey questions.

Telephone surveys require no incentive. Depending on the screening mechanism, the incidence rate – the number of interviews per attempt – can be low and this raises the costs. Telephone surveys do not allow for any visual cues and thus trade dress (i.e. appearance of the product) is virtually impossible to ascertain via telephone. Since the telephone book is they key source for names and numbers, the possibilities are virtually unlimited.

The Internet, in theory, combines the best of both worlds. Internet surveys not only permit the asking of verbal questions and recording verbatim answers, they also permit transmission of visual images such as products, labels, logos and packaging. Internet technology also permits sound transmission. Transmission costs are minimal with an e-mail blast of 5,000 names costing about $800 or $160 per thousand. (Typical mall costs are $30-$40 per interview). Unfortunately, there is no telephone book for e-mail addresses, and in order to use this medium you have to hook into a vendor that has large opt-in consumer panel data bases. By using opt-in panels, you will bypass all the SPAM filers and anti-SPAM on-line watchdogs. Moreover, you have an instant, real-time tabulation process and, if you offer an incentive, you can request the names, addresses and phone numbers of respondents thus greatly facilitating the validation process. (Validation is a procedure when an independent research organization calls a sample of those who completed the interviews and verifies that they recall taking the survey.)

However, Internet surveys are fraught with peril. For one thing, response rates are abysmal. InfoUSA, a leading consumer panel name provider, reports the average consumer Internet survey produces a click-through rate of 0.5 to 1 percent and your average business-to-business survey produces a click-through rate of 0.5 to 2%. Also incentives are required. (Click through is the percentage of people who get the survey, open it and complete it.) Getting a sufficient number of responses is a pure “numbers game” based on the your response rate. If your response rate is 2 percent and you need 200 completed interviews, you will have to send out 10,000 e-mails out to obtain a valid number of responses. If your click-through rate is 1 % it will require 20,000 e-mails.

Bouce-back rates are another problem. Consumers are fickle, they move and change e-mail address. Typical bounce rates for consumer e-mail surveys is 10 to 35 percent and typical bounce-back rates for Business-to-Business e-mail surveys are 2-8 percent.

A far bigger problem is the respondents versus no non-respondents dilemma. This requires some kind of back-up or verification to make sure the non-respondents feel the same way as the respondents. Mall intercept is probably the most effective way to back-up a consumer study. Telephone won’t work if there is a visual element.

As any researcher will attest, there is no such thing as either the perfect survey or a totally useless one. Each method has its pluses and minuses. The key is to know and understand the limitations before you embark on the research effort.

So, there are pluses and minuses to this and every other survey instrument, the key is to understand limitations before one embarks on any market research effort.