Tel: 020 7617 7848
or 
Multivariate vs. A/B Testing
In this post about Multivariate vs. A/B testing, you will find out when to use A/B and multivariate testing & the pros and cons of each method.
1. A/B Split Testing
Overview:
A/B testing compares the performance of two or more alternatives of a single page element, page or funnel.
When to use:
To test radically different ideas, e.g. layouts, different conversion paths (checkout, sign up & lead generation process) etc.
NOTE: when doing a test on a site you always start with A/B testing first, then after you find the winning page, you move on to multivariate testing to tune up the page (buttons, messages, hero shot).
2. Multivariate Testing MVT
Overview:
With multivariate testing MVT you test different content simultaneously determining the most effective combination. Democratic as your site visitors “vote” through their actions.
When to use:
For more sophisticated comparisons of multiple elements and to develop broader/deeper insight.
NOTE: Google Website Optimizer is using full factorial method, i.e. tests all combinations.
There are testing tools which provide fractional factorial method, however I recommend to make yourself aware of potential downsides of using fractional factorial method by reading this white paper by Tim Ash, an author of the book “Always be Testing”.
Watch the full webinar
Will open in new window. Registration prior viewing might be required.
Questions or comments?
For queries regarding web design, landing page optimization, and for more information on this article please contact Jan.
jan@proimpact7.com
[...] This post was mentioned on Twitter by Jan Petrovic, Katie Beswick, Suzanne Taylor, eShopping Cart and others. eShopping Cart said: Conversion/Cart tip: Multivariate vs. A/B Testing http://bit.ly/dpp5EG by @ProImpact7 [...]
@Jan
It always surprises me when I hear from WhichTestWon readers who not only don’t know the difference between A/B testing and multivariate testing, but also don’t know the difference between a controlled experiment (A/B Test) and before and after testing.
Just last week, at least two of our Top 7 Lead Gen Form A/B Tests webinar attendees asked us to explain the difference between an A/B test and a Before and After test. Just goes to show, more marketers need the kind of info you’ve provided here.
It’s a great way to see the very clear difference between two types of testing … love the full factorial vs. fractional factorial explanation as well.
Natalie Myers
Senior Reporter
WhichTestWon.com
Natalie, I know exactly what you mean. We had a client who approached us. He said that he did some A/B testing. When I asked him how he did it and what was the result he said: “1 week we run version A and then the next week version B”, which as we both know isn’t the way to do, because your test results are basically invalid. So there is still a lot of confusion out there about how to run A/B tests. Jan
Thanx for this article explaining the different MVT tests. we at http://www.reedge.com use full fractional but are indeed thinking of adding others because not everyone has lots of traffic. Giving the clients an option is good.