Digipropolis Blog: Marketing Pros & Their Cons

You Gotta A/B Test to Make it the Best

After posting last week about the power of using URL builder to track campaigns via email and social media, I got to thinking about how to know whether a campaign is “the best” it can be. This is one of the downsides of using Google Analytics to see campaign metrics – it’s hard to judge whether results are good, bad, or so-so.

young female covering one eyeThat’s where testing, the heavy lifting of integrated marketing communications, comes in: A/B testing, also known as split testing. Why is it heavy lifting? Because coming up with multiple potential winning offers, headlines, images, etc. to include in a campaign is harder than coming up with just one great idea.

What is A/B Testing, aka Split Testing?

A/B testing is a controlled experiment in which a marketer creates a direct response piece such as an email message and develops multiple executions of a single variable, such as an e-mail subject line. The marketer then runs the campaign, simultaneously sending one execution of the direct response piece (“version A”) to a small selection of recipients, and a different execution (“version B”) to a different small selection of recipients. The results are analyzed, and the version that produces the most responses “wins.” The winning version is then sent to the remaining recipients. The aim of A/B testing is to optimize the direct response marketing campaign to produce the best-performing marketing piece.

How Does A/B Testing Work with Landing Pages?

Landing pages are the page to which you direct your traffic which comes from clicking on your email, Facebook message, Twitter link, or some such promotion. Just as with email split testing, you should consider experiments to test the performance of different headlines, buttons, colors, and images on your landing pages. Landing page design and options can determine whether visitors go on to “convert” by taking the next step to interact with your brand, whether that means clicking the “next” button, inquiring through a contact form, or making a purchase in an ecommerce store.

It’s possible to do some very cool, automated A/B testing (and even multivariate testing) of website landing pages from within Google AdWords by using a free tool, Google Website Optimizer. With this tool, you add javascript tags to the experimental section of your website (for example, the headline), which allow Google to track the progress of your experiment. Google AdWords then directs visitors who click on your ad to view alternative versions of your landing page, and tracks which version of the page receives the most conversions.

I expect to be using this for the first time next month, and I’m pretty excited to see what the results will be! Note: to use Google Website Optimizer, you must have both a Google AdWords account and a Google Analytics Account.

Tips for A/B Testing:

A-B_test_outcome_graphic● Always run split tests simultaneously (Running a different campaign every day may produce different results, but there would be many uncontrolled variables that could impact your test. Running tests at the same time helps to improve the accuracy of your test)

● Use a large enough sample to produce statistically significant results, and allow the test to run long enough to draw a conclusion

● Be careful not to show the same individual more than one offer. Carefully preserve the list segmentation, or use an automatic tool such as Google Website Optimizer to prevent repeat visitors from seeing both a test version and a final version. After all, you don’t want the same visitor to see two different prices or offers!

● Let the results of the test make the decision about which headline, image, or button is best, not your gut feeling. The results may be counter-intuitive, but the value of the test lies in trusting the statistical evidence about what produces desired responses from target audiences.

For more “do’s and don’ts” of A/B testing, click here.

Perform Tests When the Answers Really Matter

I thank Wikipedia for leading me to this fascinating article about how A/B testing is used in the web design process at Google. Lead designer Douglas Bowman left Google in 2009 after he found himself involved in split testing over decisions such as whether a border ought to be “3, 4, or 5 pixels wide.” The moral of this story? Testing efforts are time-consuming for staff, and have the potential to interfere with the creative process; employ them only when variables have the ability to change outcomes.

May designers and marketers continue to work in peace and cooperation. Amen.

More Resources About A/B Testing:

Google Website Optimizer User Guide (Flash Presentation)

The Ultimate Guide to A/B Testing (Smashing Magazine)

User Experience and Design Tips: Four Reason to A/B Test


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: