In this post about Multivariate vs. A/B testing, you will find out when to use A/B and multivariate testing & the pros and cons of each method.
1. A/B Split Testing
A/B testing compares the performance of two or more alternatives of a single page element, page or funnel.
When to use:
To test radically different ideas, e.g. layouts, different conversion paths (checkout, sign up & lead generation process) etc.
NOTE: when doing a test on a site you always start with A/B testing first, then after you find the winning page, you move on to multivariate testing to tune up the page (buttons, messages, hero shot).
2. Multivariate Testing MVT
With multivariate testing MVT you test different content simultaneously determining the most effective combination. Democratic as your site visitors “vote” through their actions.
When to use:
For more sophisticated comparisons of multiple elements and to develop broader/deeper insight.
NOTE: Google Website Optimizer is using full factorial method, i.e. tests all combinations.
There are testing tools which provide fractional factorial method, however I recommend to make yourself aware of potential downsides of using fractional factorial method by reading this white paper by Tim Ash, an author of the book “Always be Testing”.
Watch the full webinar