What is Multi-Variate Testing?

Narrative IndustriesGlossary of Web Terms, Strategy, Thoughts


The film Groundhog Day came up in a pub conversation the other day. In the 1993 comedy feature film, Groundhog Day, Bill Murray plays Phil Connors, an arrogant, egocentric TV reporter assigned to cover the annual Groundhog Day event in a small American town. Phil wakes to find himself caught in a one-day time loop; every morning he wakes-up and is forced to repeat the exact same events again and again. At first he is rude, obnoxious, steals and even commits suicide, because he knows he’ll wake-up to relive the same day in the morning, reset exactly as it was before. Eventually he realises that he can use the opportunity of reliving the same day over-and-over to learn something. He starts to play musical instruments, studies foreign languages & 19th century French poetry while helping those people around him that he had previously ignored or abused.

Imagine if you could keep trying different approaches to the same problem until you found the one that worked best… well, you can; Groundhog Day is a film about multi-variate testing.

With Multi-variate testing you get to try different approaches and see which ones your customers prefer, which ones provide a higher conversion rate, which designs generate more sales or which email subject lines get higher open rates. You can test all kind of things. Over time, using a programme of testing, evaluation & assessment in your digital strategy can highlight the best way to optimise your channels, ensuring your website, email marketing and other customer facing communications are optimised for conversion. You can’t change your customers but you can change how your business communications treat & react to them and you can find the most effective way to do that by a process of multi-variate testing.

Groundhog Day is a perfect illustration of how doing the the same thing over and over and expecting something to change will just keep you trapped in the same recurring loop. There’s no point spending money on paid search to drive traffic to your website or email marketing campaigns or eCommerce systems if you don’t make it easy for your visitors to buy your product or service when they  arrive. And if you’re trying to optimise your website for conversion, how do you know what works best?

At the start of the film, the protagonist’s reaction to his situation benefits no-one, least of all himself, because he’s focussing on himself and not the results. Once the protagonist starts to pay attention to his situation, he quickly starts to learn from the mistakes he has made and adjusts accordingly. He realises that he can’t stop the recurring events from starting anymore than you can change your customers, but he can affect the outcomes by changing how he deals with the events. Some of these changes are big and some are very, very small but by a process of continually trying different approaches, assessing the outcomes & refining his approach for the next time, he “optimises” his situation, becomes a better person, gets the girl and finally escapes the recurring time-loop nightmare.

While there is a lot of best practice when it comes to conversions, there’s no magic panacea that will work for everyone. Even the most experienced web consultancy can only ever give you their best guess of what will work for your organisation. At Narrative (previously THBOOM!) Web Consultancy, we prefer to develop digital strategy based on proven techniques and it’s in providing that evidence where multi-variate testing comes into its own.

  • Will adding X content to my page reduce my bounce rate?
  • Will a freephone number increase calls?
  • Which call to action generates more leads?
  • Where is the best place for the “buy now” button?
  • Which subject line will increase my email marketing open rates?

All these and more can be answered with multi-variate testing, so why not find out what it can do for you?…. Or you could just keep going through the same Groundhog Day over and over again.