Over the last 4 years I’ve conducted more than 250 A/B tests focusing on copywriting, and how copy impacts conversions. I’ve experimented with everything from headlines and body copy, over to form/button copy and everything in between.
Here are the 5 main lessons I’ve learned from this extensive research.
Lesson 1. Your copy has direct and measurable impact on conversions
98% of the copy tests I conducted had a direct, measurable impact on conversions. The impact wasn’t always positive (far from it, I’ve often made copy changes that hurt conversions), but the fact remains that almost every test affected the decisions and actions of the prospects – regardless of whether the action revolved around a sign-up, download, purchase or even just a click through to another page.
Let’s start out with a few examples from the real world
I’m going to walk you through 2 different case studies – both consisting of 2 individual tests – where the initial test generated an impressive lift, and the follow-up experiment did the opposite and generated a significant drop in conversions.
Case study 1 – Test 1:
The first case study revolves around the sign-up form on a betting community. At first glance, the original form copy is pretty decent and does an okay job of clarifying the purpose of the form. The header says, “Join BettingExpert”, and the button copy says, “Sign Up+”
However, the form copy doesn’t convey any value whatsoever and does nothing to answer the question, “Why should I fill out this form and give you my email?”
When I created the treatment, I focused on increasing the relevance and value communicated by the header and button copy. In other words – I focused on answering the question, “Why should I fill out this form?”
The main – and most tangible – benefit of becoming a member of BettingExpert.com is that you can get free betting tips from top tipsters on a daily basis.
Based on experience from similar tests, I hypothesized that I could accelerate the decision-making process of the prospects and increase signups by focusing the form copy on the main benefit.
In my treatment the header says, “Get FREE Betting Tips” and the button copy says, “Sign Up & Get the Best Daily Tips” In order to find out whether my hypothesis would hold water and increase conversions, I set up a simple A/B test with the Control Variant (A) and my Treatment (B). The Treatment increased signups by 31.54%. Read the full case study here >>
Case Study 1 – Test 2:
Being the test fanatic that I am, I went ahead and did some more tests on the form copy. For the second experiment I decided to try a different benefit/value prop – the fact that BettingExpert.com helps you make better bets.
The treatment copy, ”Join BettingExpert & Make Better Bets – Sign Up & Start Making Better Bets”, reduced sign-ups by 12.45%.
So even though both experiments revolve around the same element and are based on the same hypothesis, the changes impacted conversions in very different ways – at both ends of the scale one might say.
Case study 2 – Test 1:
The second case study revolves around the button copy on the payment page of a popular B2C website for students.
The control CTA copy, “Create My Account” is relevant, as it addresses the conversion scenario at hand and describes what will happen when you click the button. It conveys a certain level of value. As opposed to, “SUBMIT” or “Buy Subscription”, it emphasizes the positive aspect that you’re going to create an account, which has an implied value to the potential customer.
Nevertheless, I saw room for improvement and hypothesized that a few copy tweaks would lead to a lift in sales.
I knew from customer analyses and other tests I’ve conducted on the website that prospects most often sign up to WriteWork.com when they are in a hurry to get started on their writing process. Therefore, my test hypothesis was that adding urgency and a sense of immediate satisfaction to the button copy would give prospects the last “nudge in the right direction” and increase sign-ups.
In order to find out whether my hypothesis would stick, I set up a simple A/B test with the Control Variant (A) and my Treatment (B). The Treatment increased payments by 31.03%. Read the full case study here >>
Case study 2 – Test 2:
As a follow-up experiment, I decided to test the impact of different possessive determiners. So I set up an A/B test with the control “Create My Account” and the treatment “Create Your Account”. Based on previous experiments, I hypothesized that “Your” would convert better than “My”.
I expected to see a minor increase and was taken aback when the test clearly showed that the treatment copy hurt conversions and reduced the number of payments by 24%. Again the point is to illustrate how simple copy changes to the same element can affect conversions in wildly different reactions from the prospects.
These case studies are perfect examples of how much of an impact copy has on conversions, and how little it sometimes takes to affect prospects actions significantly. In this light, it’s sad how little time and money most companies invest in testing and optimizing website copy (for other purposes than SEO).
For some reason it’s more legit to spend big budgets on design, ads, and campaigns than on the most essential part of online marketing – the copy itself.
Lesson 2. Your web copy is a means to an end – not an end in itself
If you want results, you need to start by setting goals. When your goals are set, you need to find out how to best achieve them.
I think we can all subscribe to the logic behind this statement. Well, the same logic applies to writing online copy that converts.
If you want to achieve results with your website copy, you need to start by defining the goal of the individual piece of copy. After that you need to find out how to write the best possible copy to achieve that specific goal.
If you’re writing copy for a sign-up form, you need to focus on the conversion goal of the form and the purpose of the copy: to get as many qualified leads as possible to fill out the form and sign up.
Therefore, it’s important that you can leave the ”artist” on the shelf for a while in favor of a more analytical/scientific approach that will help you focus on giving your potential customers what they need in order or make the right decision.
From an artistic or creative stance, the resulting copy may not be the most elaborate or inspired solution. But essentially that doesn’t matter – as long as the copy has the desired effect on the prospects.
In the example I showed you earlier in this article, the treatment form copy wasn’t more creative or inspired – in fact, I think some would even call it lame. But again the goal wasn’t to write sexy copy, the goal was to get more sign-ups, and a 31.54% lift speaks for itself.
I’m in no way saying that you should never write creative or sexy copy, I’m saying you should do it for a reason and if it is the solution that best supports the conversion goal.
The goal is conversions not creative awards
If your web copy becomes the goal in itself, you’re very likely to end up writing something that sounds sexy and works well linguistically, but doesn’t have any effect on your potential customers. Writing such copy might get you nominated for a creative award – but it won’t get you more conversions.
As a copywriter it can be very tempting to show off your superior linguistic skills and creative super powers. However, I have come to accept the fact that I get the best results when I think of copy as a means of achieving my conversion goals.
When I write online copy for conversion optimization purposes, I really try to remove myself – and more importantly my ego – from the equation and aim for a scenario where the prospects don’t even realize that they are reading a piece of sales copy but rather just take it in as a natural part of their decision-making process. I don’t get my satisfaction from people telling me that my copy sounds awesome - I get my satisfaction from seeing conversion rates go up…
Lesson 3: It’s all about optimizing decision-making processes – not web pages
Essentially, the purpose of online marketing is to make it as easy and attractive as possible for your potential customers to accept what you are offering them. And no matter what you want them to accept, the scenario is going to be more or less the same:
You offer your prospects something that represents a value to them. In order to get it, they have to give you something in return that represents a value to you.
This scenario will invariably start a process in the minds of your prospects where they have to assess and decide whether your offer is worth accepting. And there are really only two possible outcomes:
1. They can say, “Yes”
2. They can say, “No”
There will always be a number of factors that tip the decision towards “No”, and it is the value of your offer that has to tip the decision back towards “Yes”.
In order to get a conversion, the process has to end with your potential customers agreeing that what they will get in return is worth more than they have to part with. The decision-making process takes place in the minds of your potential customers, and optimizing this process should be your primary focus.
Essentially, the most effective way of optimizing this process is to answer the question, “What’s in it for me?”
Here’s an example from the real world
I recently ran an A/B test on the newsletter sign-up form here on ContentVerve.com. The control featured a generic sign-up form that more or less just stated the obvious: that you can get fresh updates.
The treatment on the other hand promises a specific value in return by exemplifying what updates from ContentVerve.com consist of – thus answering the question, “What’s in it for me?”
I’m sure that on more popular and well-known blogs simply putting a sign-up form on the page is enough to get tons of sign-ups. But ContentVerve.com is still new and not particularly well-known. Therefore clarifying what you get in return for giving me your e-mail is an essential step in the conversion funnel.
Lesson 4: Clarity and relevance are king
Your potential customers will often go through the entire decision-making process in a few seconds, and lack of clarity and relevance is a major conversion killer.
The more time your potential customers have to spend in order to figure out what your offer is all about, the more likely they are to leave your website and move on to one of your competitors. The more clearly you express the value of your offer and why it’s relevant to your prospects, the more likely they’ll be to choose it.
So don’t waste their time with hype, fluff, and over-creative marketese. Tell them clearly how they will benefit from accepting your offer, and give them a good reason to say “Yes!”
Here’s an example from the real world
In this example, adding clarity and relevance generated a 99.4% lift in conversions on a PPC landing page for Denmark-based Saxo Bank. The landing page pitched a trial account for a Forex trading platform, and the conversion goal was to increase the number of trial account sign-ups.
If you compare the two versions, you’ll see the treatment is super focused on conveying the value of the offer, while the control is much more vague. Where the control asks the question “Why trade Forex with Forextrading.com?”, the treatment actually answers that question by giving prospects relevant information and solid, credible arguments why they should sign up for the trial.
What is more, the control copy didn’t follow up on the value promised in the PPC ads that actually did a great job of emphasizing the selling points and value of the Forex Trading demo: free, no risk, no obligations, $100,000 demo account.
In the treatment, I made sure to follow up on all the selling points, benefits and features mentioned in the PPC ads. For a full run-down of the copy changes in this case study, read this post >>
One more example from the real world
A great way of adding clarity and relevance to your copy from the get-go is by working on your headlines.
Here’s what happened on BettingExpert.com when I optimized both the form copy and the home page headline (as opposed to just the form copy) – we got an extra 3.65% increase in sign-ups.
Over the course of two treatments we went from something quite vague, ”Passionate about Betting? We are too – Join BettingExpert.com – Sign Up+” to something much more clear and relevant, ”Get Free Daily Betting Tips from Top Tipster – Get FREE Betting Tips – Sign Up & Get the Best Daily Tips”
Again, the control headline may sound better and more sexy than the treatment, but the treatment still got more potential members to sign up.
Lesson 5. There is no “secret formula” or “one-size-fits-all” – you need to test!
Let’s start with another case study. Oli from Unbounce.com and I recently ran a split test on a PPC landing page that pitches a free 30-day trial of the Unbounce.com landing page platform.
The only thing we did was to tweak 1 word in the copy – we changed the possessive determiner “You” to “My”. After running the test for 3 weeks, the treatment button copy, “Start my free 30 day trial” had increased the number of trial sign-ups by…. hold on now…. 90%.
There’s no way we could have predicted that such a small tweak would have such a significant impact on the decisions of the prospects. We only became aware of this major low-hanging fruit because we performed an A/B test. I’ve conducted similar experiments on other sites and gained significant lifts – but I’ve also seen it backfire.
Life would be easier if the one-size-fits-all, cookie-cutter approach worked – so much easier! However, in my experience there is no such thing as a global solution that works every time. All products, offers, websites, and companies are different, just like the motivations of your potential customers will be different. You need to find out what works on your specific target audience – and the only way to do this is through rigorous research and testing.
If you want to learn more about how to get into A/B testing – read this article >>
Join the conversation – ask questions or share your experience
If you have any copy/conversion-related questions, you’d like to ask me, or experience you’d like to share, please feel free to join the conversation on Google+ or simply leave a comment right here on ContentVerve.com.
Thank you, Michael. Real Life examples and real numbers are just so much more fun to read about. Keep it up!
Morten Rosenvold Villadsen recently posted..Uvjagt i Nordnorge – helleflynder og havkat
Thanks Morten – I’m glad you liked the article!
- Michael
Michael, great work once again.. I particularly liked this bit..
As a copywriter it can be very tempting to show off your superior linguistic skills and creative super powers. However, I have come to accept the fact that I get the best results when I think of copy as a means achieving my conversion goals.
Thanks Kris!
Yes in my experience, being able to do that is one of the most important aspects of CRO Copywriting.
- Michael
Great info Michael. Understanding the type of copywriting that converts has always intrigued me. Unfortunately if you don’t get lots of web traffic or conversions its difficult to get conclusive results. Do you have any ideas on other ways to test content?
Thanks again for the great article!
Nick Johansson recently posted..Why a Regular Desktop Website Doesn’t Work on a Mobile Device
Hi Nick – Thanks for the comment!
Traffic is definitely an issue when ist comes to testing, and low traffic is a validity threat.
One way of performing tests if you have low traffic is to simply let the test run for longer til you have a large enough sample to yield valid results.
Also, with small amounts of traffic it can be helpful to test radically different variations of e.g. your copy. Typically, the the more radical the change, the more radical the results will be.
MarketingExperiments recently posted this article on testing with small samples: http://www.marketingexperiments.com/blog/analytics-testing/testing-small-sample-sizes.html
- Michael
Thank you Michael, really enjoying your posts on CRO. Keep up the good work.
Thank you very much Simon!
Thanks for reading
- Michael
I really appreciate reading your posts Michael
You really teach us important matters when it comes to converting visitors. Even more importantly you inspire (at least me) to do a lot of testing myself!
Godt arbejde! Good job!
Christian Møller recently posted..Musclepharm Assault anmeldelse
Thank you very much Christian – It means a lot to me
- Michael
Thank you very much for the insights you provide in this (and other) posts.
It’s really inspiring to read your thoughts.
Keep it up,
- Kent
Thank you very much Kent! Thanks for following my work.
- Michael
Very interesting post, useful to see how some relatively minor changes have a drastic effect on the number of conversions – changes that many business/website owners would disregard. The numbers definitely speak for themselves.
Hi Michael
Very interesting post.
How do you make sure your tests are not affected by seasonality, competitors etc? And how do you calculate the statistical confidence? Since it changes from time to time?
You have been added to my RSS reader for some time now, so keep up the good work!
Hej Jesper – One of the advantages of split testing is that you are testing on the same quality of traffic over the same period of time.
You split the traffic 50/50 between the original page (control) and the new variant (treatment) and measure successes via a common conversion goal. This eliminates most basic validity threats and bias and is much, much more accurate than e.g. comparing periodes. Moreover, there are cookies involved to ensure that the data for each visitor is only recorded once, even if he or she visits the tests page several times (otherwise the data would be useless).
Regarding test validity, statistical confidence is one of the factors you need to look at – but not the only one. I made this video to give you a basic introduction to test validation and validity: http://contentverve.com/how-to-determine-statistical-significance-ab-test/
- Michael
Michael,
Fascinating data.
Looks like, generally speaking, “my” is more effective than “your.” Also, specific benefits (such as $) are more attractive than more general feel-good copy.
Still, you make an excellent point: there are no one-size-fits-all rules so it pays to keep testing.
Thanks,
-Diana
Hi Michael
Wonderful blog. We’ll be using some of your findings for launching our own blog sign-up
Cheers,
Robert
Robert Clarke recently posted..10 Tips to PPC Success and Getting Your Google Money’s Worth
Hi Robert – Thanks for the kind words and for reading my blog!
I’m really happy to hear that you could find some inspiration in my work!
- Michael
this is my first time on your blog and i’m already hooked !
I’m in the process developing a wordpress theme for a niche market and i’m starting with the sales funnel of the theme. I want to provide my prospects with a “done for you” high converting funnel. What is your advise and how would you recommend i start working on it ?
Thank you MICHEAL
Yassin
Hi Yassin – I’m glad you found my blog and thanks for reading!
You’ll have to be more specific for me to help you. What is the niche market? What will you be selling? Sales funnel – does that include payment gateway? Is it a theme that you want to sell to others?
- Michael
Great data, Michael. A lot of overlap with MarketingExperiments, but always good to get a refresher course! I’m fascinated by the “my” vs. “your” difference in conversion. I always thought “your” would be the winner.
Michael recently posted..10 Life Mantras Inspired by Doing Deadlifts
Thanks for the comment Michael!
Yes, I’ve learned a lot from Dr. Flint, and I’m a big fan of MECLABS. In fact, I’ll be speaking at the MarketingSherpa Email Summit in Las Vegas in a couple of weeks – if you have the opportunity, you should definitely go! I also spoke at the 2012 Optimization Summit in Denver. Moreover, they have borrowed a couple of of my case studies for use in their teaching material (blog posts, presentations, Web Clinics).
Yes, I have a lot of very interesting research regarding possessive determiner that I will publish at some point.
- Michael
You make a great point about having a copy vs copy duel to the death. But how would you sell this idea to a client? Would you write two pieces of copy and tell him to test it himself?
Hi Jose – Thanks for the question.
It depends on the client, the individual website, and the conversion scenario.
I usually create a treatment and test it against the original copy on the website. I don’t raelly have to sell copy optimization and testing to clients any more. They usually contact me after reading some of my case studies, and that means that they’ve already made up their mind.
But back in the day, I had to do quite a bit of “selling”. A good way of doing so is to find a piece of copy where it’s more or less a given that you can generate a lift. Use that for you r first test and show results (maybe do the test for free), after that it should be smooth sailing
- Michael
Thanks again Michael. I hope you don’t mind if I push for more details.
To create a test I can take a piece of copy off a client’s website (one that needs work), create two treatments and then tell the client “test both these treatments on your own and go with the one that boosts conversions the most.”
Thanks again for the help. This is good stuff.
Interesting! I am going to start looking at areas of my site that need attention, like the forms and landing pages. This gives me some good ideas to start with. Thanks!
Rose recently posted..How to Store Scarves for $1
Awesome Rose! Just get cracking
I’d love to hear about your results, once you get going.
- Michael
I’m constantly surprised by how much of our decision making is subconscious – the magnitude of signup increase in the “Create you Account” vs “Create my Account” test surprised me.
Side note: been trying to contact you – your Hybrid Connect plugin must be 70 versions out of date! We need to upgrade you..
Paul McCarthy recently posted..App of the Week: Better Data and What to Do With It
Ha ha – Sorry man, I don’t think I’ve received any mails from you.
I did get the version of Hybrid Connect with the split testing feature. Have you updated since?
I love the testing feature, but I do have some input for you guys on how to make it better.
- Michael
Awesome post! You dropped so many little nuggets here. I am getting more and more into split testing. However, we work with a lot of small businesses and sometimes getting enough traffic is a problem. I think we need to start paying for some traffic or paying real users at usertesting.com or using the new google survey’s to get more data back. Do you recommend any particular services where one can buy feedback from real users to split test things faster?
matthew Hunt recently posted..1 in 3 Searches are Local
Hi Matthew – Thanks for your comment and the kind words about the article!
Yes, small sample size is an issue when it comes to split testing. PPC is always a good to generate more traffic to your landing pages – and landing pages are a great place to start testing.
User testing and surveys are great ways to get data and insights that will provide you with more qualified test hypotheses – which will make large gains much more likely. But you need to test whether your hypotheses work when you put them all together on a page.
I use Quaraloo.com (formerly KISSinsights) for simple surveys and usertribe.com for user tests. Also, I recommend Kristin Zhivago’s book “Roadmap to Revenue” for learning more about how to get your customers to tell you how to sell to them: http://lp.roadmaptorevenue.com/get-the-book/
- Michael
Great article, great info and editing my web forms will be a top priority the next time I’m in front of my computer. Really great! Thanks so much for the info.
Nadine recently posted..I Received Your Marketing But…
Hi Nadine – Thank you for the kind words!
I’d love to hear about your results optimizing your forms.
- Michael
Thanks for this cracking article that really shows just how important it is to split test. I’ve book marked and subsribed. Thanks Michale.
Marcus Rockey recently posted..If they unsubscribe it’s good for business!
Thanks for the article. I love the fact that we think we have such control over every decision we make. This stuff makes you realise how much the brain is influenced by the environment around us. Loving conversion-related posts at the moment!
Quentin Aisbett recently posted..DIY: The 3 minute SEO Audit for your website
Michael, your tests are always interesting.
And make me think. Meeting your customer in person you have to talk to him directly and making the value of your offer obvious. If not you will loose the order. Thats about the same.
You can use the sentences from your split tests when asking for the order – or when you meet your customer. The customer always ask him self: whats in it for me? Whats the risk?
Exactly Bjarne – keenly observed! You have to give your potential customers just as much information and security online as in real life.
- Michael
Hi Michael.
Interesting article. Can you tell me, have you ever run a split test using identical versions for both? If so, what results did you get?
Also, do you ever look at outputs (say conversions) over time to assess the levels of variation before running a test?
Thanks.
Simon
Hi Simon – Thanks for your comment.
Yes I have run several tests with to identical variations. The results are as expected; once the sample size is large enough, and the standard error is small enough, the two variations end up having a minuscule difference in performance i.e. 0.3% or no difference at all. Running an A/A test is a good way of judging whether you split test tool is in fact reliable. However, you need to look at both standard error and sample size as well as statistical confidence. Many make he mistake of just looking at confidence level – this is never enough to establish statistical significance! Moreover, running A/A/B tests is a good way of adding an extra validity check to your test setup. Once the two A/A have an almost identical conversion rate, you have a good sign that the test is reaching a significant result. In other words you’ll compare A/A to B.
Yes, I do look at conversions over time – many things can affect your potential customer’s level of motivation – which is the primary factor in any conversion sequence. However, one of the best things about A/B testing is that it inherently eliminates a lot of bias due to the fact that you run the test on the same quality of traffic, over the same period of time, as opposed to simply comparing periodes.
- Michael
Thanks for responding Michael.
You have to measure before and after or i) you don’t know if the improvement sustains beyond the lifetime of the test and ii) you don’t know if natural variation in the process is a factor. Statistical significance (even as described by you so well in your excellent video) compares just two data points and can only tell you that the two numbers are different, not why. That has to be inferred.
I haven’t done any A/A testing (although I will now) but I have certainly found that the same A/B test run at different times can have vastly different results, meaning it is nothing more than a snapshot rather than a sustainable and repeatable improvement. The other thing I’ve found repeatedly is that very different copy variants often have almost identical (and certainly not statistically significant) results, making it impossible to infer causal links.
Hi Simon – Of course even very different variations can have almost identical results. Like Dr. Flint says “It’s not the magnitude of change on the page that impacts conversion – it is the magnitude of change in the mind of the prospect.”
If you repeatedly find yourself testing radically different variations with no results, my experience tells me that it’s because your making the wrong changes or that the motivation level of your prospects is so high that your changes aren’t affecting their decisions. If I were you, I’d go back to the drawing board and re-consider the hypotheses that you base your experiments on. Also, I’d go deeper in collecting data and doing research on my target audience. The more you know about you potential customers and their motivations/barriers, the more likely you’ll be to create treatments that get results. I’ve performed many split tests with no results, and been baffled by it, only to find that I was focusing on the wrong aspects. In those situations, I find that the best solution is to go into research mode and find out what represents the largest “change in the mid of the prospect.”
Of course A/B tests run at different times can have different results. A button variation with Santa on it will in all likelihood perform better in December than in June
A lot of people think that A/B testing is an automated process you can perform blind-folded – nothing could be farther from the truth.
- Michael
Thanks Michael.
Hi Michael,
Great post. Have you any done any experiments regarding placement of social media share buttons.?
I went to tweet your post after I finished reading it but had to go searching for the share button…all the way back to the top of your post!
- Doug
Hi Doug – Great point, thanks for the heads up. I just added buttons to the button of my posts
- Michael
Hi Michael,
Thanks for the awesome real life examples. They are helpful indeed and I will be sure to reference them in the future for clients. In my opinion there are 4 main parts; design, message, form fields, and call-to-action button. Each one has to be simple, concise, and to the point to have the ultimate impact. Obviously different businesses have different customers so it should be adjusted accordingly.
Keep up the great work
Cheers!
Internet Marketing Mentor recently posted..Search Engine Optimization Strategy: The Foe and The Friend
Hi Mike – Thank you for the kind words!
I like your approach with the 4 main parts, it makes a lot of sense – especially for LPO.
Thanks!
Way cool! Some very valid points! I appreciate you writing this
write-up and the rest of the site is extremely good.
Son recently posted..Son
Thanks Son – I’m glad you liked it!