Multivariate vs. A/B Testing

In this post about Multivariate vs. A/B testing, you will find out when to use A/B and multivariate testing & the pros and cons of each method.

1. A/B Split Testing

Overview:
A/B testing compares the performance of two or more alternatives of a single page element, page or funnel.

When to use:
To test radically different ideas, e.g. layouts, different conversion paths (checkout, sign up & lead generation process) etc.


NOTE: when doing a test on a site you always start with A/B testing first, then after you find the winning page, you move on to multivariate testing to tune up the page (buttons, messages, hero shot).

2. Multivariate Testing MVT

Overview:
With multivariate testing MVT you test different content simultaneously determining the most effective combination. Democratic as your site visitors “vote” through their actions.

When to use:
For more sophisticated comparisons of multiple elements and to develop broader/deeper insight.


NOTE: Google Website Optimizer is using full factorial method, i.e. tests all combinations.
There are testing tools which provide fractional factorial method, however I recommend to make yourself aware of potential downsides of using fractional factorial method by reading this white paper by Tim Ash, an author of the book “Always be Testing”.

Watch the full webinar

Will open in new window. Registration prior viewing might be required.

Jan Petrovic
I have 9 Years of Tremendous Experience as web analyst and Senior Conversion Specialist. I have worked on optimisation project for the second-largest store retailer in the USA (the company is also a component of the S&P 500 Index), as well as for a well-known USA fashion retailer with over 380 stores nationwide. Over the course of my career I have achieved many INSANE improvements on our client’s websites, sometimes by applying best practices, another time from findings I made during conversion audits and sometimes thinking out of the box. For one of them I have even received an award in 2012 from WhichTestOne, where after testing different emails and parts of email message we increased revenue by 303.08%. What might also interest you is the fact that my average conversion improvement is 19% and my success rate is 70%. Basically, I’m VERY GOOD at what I do thanks to my Can Do Attitude to find solutions to given problems & Experience in this field.
Get 6 FREE Kickass A/B Case
Studies
Show Buttons
Hide Buttons