Test Your Gut Feeling – Which call-to-action generated a 31.03% lift in sales?

Your call-to-action is critical to the decision-making process of your potential customers. And even small tweaks can have major impact on conversions. Here’s an example from a test I just concluded on a payment page, where one of the variations generated a 31.03% lift in sales. The question is: “Can you guess which variant won?” Check out the case study and leave a comment with your guesstimate. I’ll post the test results on Friday 2 November.

Background info:

I’ve been doing some conversion rate optimization and testing on WriteWork.com, a subscription-based educational website for students. Among the experiments I’ve conducted have been a number of tests on the various calls-to-action that are located across the website.

The test described here is particularly interesting because – while the other CTAs simply move potential customers down the path to conversion – clicking this CTA actually means money in the bank.

The test design was a simple A/B split and the conversion goal was a payment of $9.90 for a monthly subscription to WriteWork.com (recurring monthly bill of $14.95).

I ran the test for 10 days and concluded it at a statistical confidence level of 98%. The winning variant generated a lift of 31.03% in sales.

Your turn – Test your CRO gut feeling, which variant won?

Leave a comment at the bottom of the page and let me know which variant you think won. Also, feel free to leave a few notes with your thoughts on what you think made the difference.

I’ll post the test results on Friday 2 November – check back in and find out how accurate your CRO gut feeling was!

Now it’s time to guesstimate – Go Go Go for it!

You can find the full case study with test results here >>

Comments

  1. My guess: button B
    Why: user already typed in his credit card number, at this point she needs reassurance she could use the service/product straight away, and the longer C2A does a better job on it.
    Waiting for the second of September; this is just my gut feeling, but you can never guess with split tests!

  2. Rasmus hoeks says:

    B.

  3. Chande says:

    “Start Writing” :)

  4. My Guess is “B”
    It says what people wants – It must be “B” …. Or I’ll swim across the
    No promise – Just testing!! – B

  5. Stefan says:

    B.
    Because generally the more relevant and the more value for a click a bottom promises the better.

    Besides – Bigger is always better.

    • Michael Aagaard says:

      Hi Stefan – keen observations, but we’re actually talking about “buttons” not “bottoms”.
      I wonder if that changes your point that “Bigger is always better”? ;-)

      - Michael

  6. Hmm. I would go for the “Create account and get started” version.

    Look forward to the see real life test results.

  7. Tough one… But my guess would be Button B! Mostly due to its focus on getting started right away – at this point in the process, the decision to buy is made, and you just wanna get going (…and be reminded of the value, you’re getting) :-)

  8. I’d say “B” because it gives someone more of what they want. Creating an account is something we do on every site, just a hoop to go through while “getting started” is really why the person is on the site.

  9. I would definitely say A. The word “My” is very fitting, it looks better… And it’s what my gut tells me.

  10. Peter says:

    B – because you already see what the next step will be – you are part of the process – the only thing you have to do is click the button to make it really happen.

  11. deandawkins says:

    My head says “B”…but my gut says “A”… Why A? It’s shorter with less text – yes, having the words “create account and get started” is nice – but they have most likely already decided to get started…so “create account” is simpler, the less words the better…

  12. Mohammed IH says:

    Hi,
    Nice. I am all for button A. It is more personal and shows the website understands its visitors at that particular stage. I personally believe writework is speaking to me especially with the use of “my” in the button copy and i will feel comfortable parting with my money.

  13. Kent says:

    Button B

    Because at this point you just want to get started. And for sure it’s not your moms account;)

    But it’s deffinitely not as straight forward as it seems, I believe.

  14. David shaw says:

    My gut was B, however I’m now thinking that’s obvious and why would you post an article if it were obvious! Can I change my mind? ;)

    • Michael Aagaard says:

      Hi David – Ha ha,could be I’m messing with your mind ;-)

      Feel free to change your mind as many times as you like!

      - Michael

    • Shane says:

      This is my problem as well. B is too obvious.

      And I’ve been wrong on too many whichtestwon results…

      So, is it possible that A converts better and if it did, why? Perhaps if the service is something that people feel like they need more than they actually want it… they maybe want to “get it over with” and the purchase gives them a feeling of having done something, after which they can reward themselves with something nice.

      In this case, the B button is a reminder that just signing up won’t help, they actually also have to do something. They have to get started and that involves work. So it makes them hesitate.

      A bit of a forced, theory, I know. But someone has to go against the trend. :)

  15. I would go for button A – I guess the word “My” would be what makes the difference to me. It just gets a bit more personal.

  16. I would say A because of “my” and proberly also because, when you have chosen this case – there are surely a surprise :-) as B is in most cases the obvisely choice

  17. Chande says:

    This is why testing is important:
    A – because it’s the obvious, don’t think thing. Because button doesn’t matter if your decision is made. Because first thing to do everywhere is to create account. Because “Get started” rings “what?” in my mind (what’s the process afterwards?)
    B – because it’s bigger, because it has a process (user may know that).

    Other possible buttons to test.
    Start writing
    Get started
    Create my account & Start writing (don’t know if this is the process or the user needs to confirm the account)

  18. Carol says:

    I would say B because it stands out & the word ‘get’ can be powerful but when we run tests there are usually some surprises.

  19. B: Create my account and get started. Once people get as far as creating an account, they are most likely ready to get an article guide. I don’t think a customer would create an account unless they were ready to “GET STARTED” so I’m going to guess that test won.

  20. I would say: A

    On buttons I often see, the less words the better.

  21. Rémi says:

    Vote : A
    :-)

  22. Adam Miller says:

    I’m sure its ‘A’ but I personally like ‘B’ more. I didn’t think about the word ‘My’ in button ‘A’ until someone commented about that above. I’m going with ‘A’!

  23. Anne Stahl says:

    I’m surprised that there was such a stark difference between the two variants to produce a 31.03% lift in sales with 98% statistical conclusiveness after just 10 days. How many actual conversions did you measure? In addition, we always recommend that tests need to run for the full duration of weeks, so as to exclude noise caused by weekend/weekday differential. To be honest I’m surprised the site manages to make sales at all. In reviewing some content pages and the quality of their ‘product’ I found it to be really rather poor.

    I think the key here is the target audience. So often overlooked by marketers and designers and copy writers. Here is a site that seems to offer students essays to ‘copy’ . Although for correctness, they present themselves as providing educational tools, it’s quite apparent what students are actually coming to do here. If the test really was run properly and such a stark win was recorded, then it must be that I (as many others here) are not that familiar with the target audience. Each button had advantages, such as the personalization of the first/left button and the bigger size and more action driven copy of the second one. I would probably go with Button two (on the right), but I would have thought it would be a close call. Thus I’m curious to learn the outcome (which I do not see posted here, despite it being after Nov 2). Please do post the full test details (which tool was used, when was it run, how many users, how many actual conversions, etc…).

    Thanks a mill for an interesting post and all the commentators for their input.
    Anne

    • Michael Aagaard says:

      Hi Anne – Thank you for the comment!

      Sorry you’re late for the party ;-) I published the full case study on 2 November – you can read it here: http://contentverve.com/case-study-31-03-increase-in-sales-by-tweaking-the-call-to-action-copy-on-a-payment-page/

      The post should answer all your questions, but here’s a summary with a few extra notes:

      As you probably know from experience, clients often aren’t comfortable posting the full data for a publicly available case study. So in many cases, It’s a question of sharing a case study with 90% of the data – or not sharing at all.

      In this case, I did not get permission to disclose the exact number of visitors and conversions. However, I can tell you that we had well over 100 conversions (1 conversion = 1 sale). As mentioned the test ran for 10 days from 17 October till 27 October. We had a 98% level of statistical confidence, and a standard deviation of < 1%. Moreover, there was a clear tendency from the very beginning that the treatment outperformed the control button – in other words there were no fluctuations where the control topped the treatment (commonly known as "diamond" shapes). The website gets 30.000+ visitors a day, so 10 days of traffic provides quite a large sample size. I used Visual Website Optimizer – my preferred testing tool. Like you, I am a Certified VWO Partner (via my Danish company Online Tekstforfatter).

      I was surprised at the result as well. But I know from my experience of more than 200 split tests, that CTAs have significant impact on conversions. Moreover, my experience is that they represent the ultimate low-hanging fruit, when it comes to CRO/LPO. In fact, I will be doing a session on how to optimize landing page CTA buttons at MarketingSherpa’s 2013 Email Summit in Las Vegas in February – hope to see you there ;-)

      If you would like to read more case studies (including more CTA tests), you can find a bunch under the category “Case Studies” in the top nav bar.

      Thanks!

      - Michael

Speak Your Mind

*