The most common question I get from clients new to site optimization is one that’s taken a while to clearly articulate. It’s easy to visualize and provide examples of different types of experiments and it may even be easy to point out the difference between the two types, but how do you express this in words for a coworker or boss that has finally understood that going off a gut instinct just won’t work anymore?

AB experiments are those that contain only one variable. Imagine there’s one thing you’d like to change on a product detail page like the color of the product headline, the product image or the product description. If you change just one thing, you’re working with an AB experiment. Now, what if you change everything about the product display area? It’s possible that this could be a multivariable experiment but in some situations, it’s not feasible to set it up this way. More often than not, these will also be AB experiments because you’re still changing only one thing. Sometimes, these are referred to as “concept tests.” Whether you have one challenger version or even 2, 3 or more (!), this is considered an AB experiment.

AB webpage experiment (A)

AB webpage experiment (B)

Note: I like to visualize compartments that changes will go into. If you can’t easily compartmentalize something, group it together with something else. If your grouping ends up one compartment, it’s an AB experiment.

Using the same example as before, now if you’d like to change the product headline, image and description in the same experiment, you’re on your way to implementing a multivariable experiment, where each of those three items is considered a variable. Now that you’re wandering down this path, someone may say:

1. Increase the headline size
2. Try making the headline green

Those are both called “values” that belong to the headline variable. You may find that you want to test 3 different product images and maybe 4 product descriptions. You have the freedom to do this and multivariable testing supports it.

Multivariable webpage experiment

You’ll quickly find that once you start testing your site, you won’t want to stop. Leverage your analytics implementation (e.g. – Google Analytics, Omniture, Coremetrics, etc.) to find the underperforming pages on your site in your conversion funnel. No matter what you do online, you’re trying to achieve something. Look at the pages leading up to that “something” to optimize visitor experience. One blogger memorably wrote, “Test or die” to bring home the point that the marketplace is changing and to stay ahead of the game, you need an optimization strategy. If you haven’t started yet, you may want to start looking at some options.