Key Marketing Lessons from the MarketingExperiments Copywriting Contest. Last month, we asked you to write the most effective copy for a Consumer Reports email in a way that could also test which value factors were most appealing to Consumer Reports donors. We find marketers at a nonprofit for two reasons: we want a real audience to test with, and we want to use our collective ability as a community of marketers to create effective messaging for a greater good. We then use what we learn from this test to run a followup test, crafted by the MarketingSherpa Summit audience. This “live test” is launched and results are reported during Summit. What You Can Learn from This Experiment There are two levels of lessons you can gain from this experiment. Asking for a donation around this theme still works better than messaging around empowerment or our impact work in particular (to the broad audience),” Dawn said. The control and treatment 3 focused on the quality of research, increased clarity about Consumer Reports testing by using specificity surrounding the testing process, and how Consumer Reports would use donations. “Since Treatment 3 themed around ‘quality of research’ was second to the control in terms of clickthroughs and donations, I learned that this theme resonates and is one we haven’t emphasized as much in past communications and perhaps could emphasize more in our future messages, along with product testing,” Dawn said. Congratulations again to Ahna, and thanks to all of the marketers and copywriters who took the time to share their wisdom and provide insights Consumer Reports can use to better serve donors and consumers.
The results are in. Last month,
we asked you to write the most effective copy for a Consumer Reports
email in a way that could also test which value factors were most
appealing to Consumer Reports donors.
We’ll get to the results, and the big winner of the
MarketingSherpa Summit package, in just a moment. But first, a few
lessons for you and a little more background.
Why This Contest?
Every year, we run a test like this with the
MarketingExperiments blog audience and the audience of another
marketing blog—in this case, Convince & Convert.
We find marketers at a nonprofit for two reasons: we
want a real audience to test with, and we want to use our
collective ability as a community of marketers to create effective
messaging for a greater good. We work with the nonprofit
for a few months, diving into the data, getting an understanding of
previous tests, and coming up with hypotheses.
Then we go out to you, tell you what we’ve learned about the
nonprofit, and give you a chance to win a package for
MarketingSherpa Summit (produced by MarketingExperiments sister
publishing brand). This year, the prize was a ticket to MarketingSherpa
Summit 2016 and a stay at the Bellagio. We award the prize
based not on what we imagine will be most effective, but rather on
the actual results with real customers. We then use what we learn
from this test to run a followup test, crafted by the
MarketingSherpa Summit audience. This “live test” is launched and
results are reported during Summit.
The goal of this public experiment is to create a tangible
example of customer-first marketing to help you improve your own
marketing efforts while putting the customer first.
How We Designed the Consumer Reports Value Proposition
Test
Objective: Increase clickthrough to
donation landing page
Problem Statement: The generic value
communicated in the renew emails does not match the motivation of
why a past donor would want to contribute to Consumer Reports.
Hypothesis: By highlighting specific value
claims that communicate the appeal and
exclusivity of donating to Consumer Reports, we will increase
the overall clarity of value of donating while better matching
customer
motivations, leading to higher clickthrough to the donation
landing page.
Primary Research Question: Which value claim
best matches past donors’ motivation to donate again?
Primary Metric: Clickthrough to Donation
Landing Page
“Testing four panels [treatments] plus a control is something we
haven’t done, as we tend to do one to two test panels at a time.
Investing in a test like this, where we took the time to come up
with four different themes and testing them head-to-head in one
test, is very valuable to have and has inspired me to challenge
ourselves to consider doing more robust testing in one round,
rather than straight A/B testing in multiple rounds, to get
learnings sooner,” said Dawn Nelson, Director, Fundraising,
Consumer Reports.
The Copy Treatments
Control: Mixed value proposition
Treatment #1: Honest and unbiased reporting
Treatment #2: Personal impact
Treatment #3: Quality of research
Treatment #4: Consumer empowerment
Results
The control outperformed all of the treatments. The control had
the largest impact when measured against treatment 4 (29.2%
relative decrease at 99% level of confidence), while it did not
validate when compared to treatment 3 (6.7% relative decrease at
85% LoC).
What You Can Learn from This Experiment
There are two levels of lessons you can gain from this
experiment. At the base level, there is the analysis you must
conduct with any test: why did one treatment perform better than
the other? Or in this case, why did the control win?
“This testing reaffirmed that our core donor still responds best
to messages around product testing, and that is their main
connection with Consumer Reports. Asking for a donation around this
theme still works better than messaging around empowerment or our
impact work in particular (to the broad audience),” Dawn said.
“The big takeaway for me on our customer (and something we have
suspected and need to test further) is we definitely can talk in a
broad sense to our donors and see good results. But if we can do
segmentation and talk…
COMMENTS