Five Mistakes to Avoid When Analyzing
Loyalty Card Data
Loyalty programs can provide big payoffs, but they need to be planned,
executed and evaluated professionally. Most importantly, the data gathered
from these card-based programs need to be analyzed correctly.
Michael Schiff, managing partner, Partners in Loyalty Marketing in Chicago,
lists five common and potentially harmful mistakes when analyzing loyalty
program data. They can be grouped into two types: data errors and goal-
related miscues.
“In talking to retailers and manufacturers, most of the mistakes to avoid can
be overcome if you put as much time into planning for the evaluation as you
do into planning for the execution,” he advises.
Mistake 1: Failing Basic Math
The first big data mistake in analyzing frequent shopper data is basic calculation or mathematical errors.
While some errors may be intentional, as people try to put their program in the best possible light, many of the unintended mistakes can be avoided by setting up a four-way comparison for checking the data. That means having lists for pre- and post-test checking, as well as test and control group lists.
As companies prepare to analyze their data, they should set aside some consumers as a control group who were not part of the program, but whose sales are being tracked. Additionally, sales should be tracked for similar periods before and after the data that is being analyzed.
It’s important to compare the right groups to ensure apples-to-apples data evaluation. For example, one client saw a 210% sales lift compared to pre-promotion numbers, but when the data was scrutinized, they were comparing consumers in a 25-week promotional period to a 5-week pre-test group. The real sales increase was closer to 5%.
Mistake 2: Losing Control of the Control Group
A control group of people that is not part of the actual analysis is an essential part of getting the data right. But a corrupted control group can undermine all the time, work and money the retailer or manufacturer puts into collecting and understanding the data.
For example, there may be 10,000 people who qualify for a program, but 1,000 should be tracked separately. They should be in the same stores and exposed to the same materials as the other 9,000, with the exception of whatever program is being analyzed.
The key to maintaining a valid control group, especially for ongoing programs, is to refresh it periodically by enrolling people who have not already participated in the program in any way. This can be done quarterly, semi-annually or annually. If previous participants do get in, then the control group could be corrupted, as those people might skew the results; they might have a different mindset about the promotion than an average consumer.
To determine if the program has a “halo” effect after it’s over, a control group can be set up comprised of consumers who were previously tracked as part of the promotion data analysis. Manufacturers and retailers have an interest in doing this if they can learn how long a promotion needs to run to maintain the sales lift. They want to know at what point they “own” the consumer - that is, at what point does the behavior become ingrained - so they know when they can stop continuously investing in new programs. Manufacturers may be more concerned about who is getting mailings so they can learn from the results, while some retailers do not care about the analysis and want to send the mailing to everyone on their list.
A final consideration with control groups is to be aware of the other promotional materials they may be receiving. If a control group is getting a lot more or a lot less of the other materials than the group receiving the promotion, then that’s probably going to influence how they behave. This is not easy to track, but it’s something companies want to at least be aware of, and to account for as much as they can.
Mistake 3: Not Knowing Objectives
Not knowing the objective of the program is the first of the goal-related mistakes outlined here.
Not all loyalty programs are the same, and their purposes fall into three broad categories:
- To increase retention of existing customers, or sell more to them.
For example, if the objective is trial, companies should understand that the program is an investment-spend. It needs to reach a lot of people to engender trial among a few. Then the analytic period needs to be extended to determine if the program succeeded in driving trial, as well as repeat purchases.
When the objective is retention, or increasing sales of an existing user, the methods are different. For example, it requires a much shorter period of time to determine if the program is getting the consumer
to stay with a brand, or buy two units instead of one. But the length of time must be adequate to determine if the promotion had its desired effect. It’s a matter of aligning measurements of the program with the objective.
Mistake 4: Not Targeting the Consumer
The biggest bang for the buck comes from targeting a company’s own consumer. But retailers, more so than manufacturers, are lax about how deep they go in the targeting of these customers.
Both retailers and manufacturers look at their universe of existing and potential customers and they want to mail - or contact by some other means, like email - to many people as they can. When companies target “everybody,” they get an average result. It’s hard to determine what elements of the program, or what consumer targets, are driving the purchasing behavior. Companies that want to reach as many people as they can should consider segmenting their consumers and tracking them by different targets. For example, a company may find that a program does well against its core target. It decides to expand the target once, and then expand it again and again. Then it’s not a targeted program anymore; it’s more of a mass promotion. If it then decides to stop tracking the results, the company might reach the conclusion that the program is not working anymore.
The best results of frequent shopper promotions come when they take the best shoppers of Manufacturer A, and also the top customers of Retailer B. Those two elements combine to form a very responsive target. But retailers and manufacturers don’t target collaboratively as often as they should.
Manufacturers and retailers shouldn’t assume that targeting stops within their own databases. When they are combined, a strong and responsive target is created.
Mistake 5: Not Planning on External Influences
Frequent shopper promotions don’t exist in a vacuum.
Although it requires a lot of energy to put these programs together, the consumers are impervious to that effort. They don’t know about it, they don’t really care about it, and there’s a lot of other media and environmental influences that are hitting them. Companies need to be aware of this competition for their customers’ attention, and account for it in their control group.
So companies need to step back, look at what else might be going on at the retailer level, from the manufacturers, or in the competitive overall social and economic environment that might help explain some of the results that they are seeing.