Some Things to Consider
When Testing Mailing Lists
One of the primary things to test is lists. Different segments of the same list. Active names versus those on an inactive file. Those that responded most recentlysometimes called "hot line names"versus others. Frequency of activity, level of activity, type of product or service purchasedeach will help you reach your best audience.
Here is a list of questions to answer as you consider and plan a mailing list test:
- Where did this list come fromwhat is its source?
- Is the sample a truly random sample?
- Is the Nth* name selection representative of the entire list? Or of a special selection from within the whole?
*(Nth is a term which means a random selection from the total. Such as every 5th or 10th or 22nd name available from the total list.)
- Is the survey sample large enough to give you meaningful and measurable results?
- Could the list have been "preheated"? That is, were there any outside factors that might affect your results?
- Did the list owner/broker have any special reason for the selection method used?
- Did all the lists included in the test belong in the test?
- When we measured the results, were all the responses reported and counted?
- Did we use percents because the raw numbers looked bad?
- Are the results consistent with previous results?
- When we evaluated all the results, on what basis did the winner win?
Testing mailing lists and offers together is also important for both consumer and business program. So you can determine which lists pull best against which offer.
Exhibit 2 is another chart idea that shows how to test lists and offers together. In this sample you have 2 offers and 4 mailing lists. By testing all possible combinations, you can see what will pull best.
In addition to testing four different lists against two different offers to determine which combination pulls best, a further offer test was made. Under list #4 a further splittesting a premium offer vs. a non-premium offerwas tried. This smaller number allows for evaluation before going to big numbers and maybe making a major mistake.
What do you gain from even simple tests such as these? You gain knowledge about your marketplace. What stops them to consider and/or order from you rather than from the competition.
You learn what mailing list works best. You learn how much to invest in your direct mailand if it will bring quality and profitable results.
You learn bottom line if direct response is a tool you can use! It is not the answer to every prayer. It wont solve every problem. For a relatively small investment, you can test and learn what your next step should be.
With the best offer you can makeyou can test to see if it works. Remembering all the time that there are no failuresonly lessons.