A/B Testing: What You Did Not Know About It

Are you allowing your marketers to throw darts in the dark by just plunging into elaborate marketing campaigns, and yet struggling with conversions? Are you running an online business and haven’t yet started with A/B testing of your web and marketing assets? Don’t know how to get started, how to gather resources, what to choose and what not?

Well, A/B testing has never been easy. However, learning from my personal experiences in marketing technology and having successfully run several tests using multiple tools – some proprietary, and others out of the box – I can assure you one thing – it is real and it’s worth it.

Why A/B Testing

Every marketing strategy could have been better – so it’s critical that you explore the “unknown” possibilities and experiment in a structured, comprehensive, streamlined, and financially lucrative way. That’s because, with A/B testing, you first know the impact of any design or elementary change that you make to your web asset on your conversion rate before you actually launch it live. So, you:

• Validate ideas before the actual deployment
• Can optimize results
• Analyze consequences before making investments and taking chances
• Can be open to ideas and experimentation

Choosing the best suitable A/B testing platform

As I have tried ample tools, I have chosen the two best available platforms for A/B Testing, that I will walk you through – Optimizely and Visual Website Optimizer.[/vc_column_text][vc_column_text]

Optimizely

What is better than a free software? You can actually start to optimize your site using the free version of Optimizely. It offers to analyze up to 50K unique visitors per month. However, this version is pretty bare boned and does not offer a lot of bells and whistles the way its elder brother does, i.e. the Enterprise edition. Nonetheless, if you are trying to do a POC or want to rally folks behind your vision of building a high-power testing team, this is a great option.

Strengths

Optimizely is designed for end users in mind. You do not have to be a software jock to start using the tool. Actually based on the way the tool works, it’s helpful for non-tech-savvy users to convert themselves into techies.

• The visual editor lets you move assets on your page, rearrange them, change their relative position, modify their visual appearance, insert new elements like HTML, video, etc. just with the click of a button.
• For little more savvy users, there is the code editor, which gives you options for more sophisticated edits.
• The ability to integrate with your existing analytics tools like GA or SiteCatalyst empowers you to make data backed business decisions. As an example, the simple integration with a heat map, and the call integration third party tool are very intuitive.
• The ease with which you can add click, page view, custom goal and other CTA goals is designed intuitively. You can simply add revenue goals to see the impact of the test on the bottom line seamlessly.
• The data segmentation and presentation is again intuitive and user-friendly. It helps you come up with designs that will resonate with users based on their demographics, income, device preference, and geolocation among other slices of data.

Limitations

Here’re some challenges that I jotted down while using Optimizely:

• The free version does not allow Multivariate test option. When a user starts building the testing roadmap with A/B testing on a single page with a few tweaks, it’s all good. But once they want to test the interaction effect of multiple changes and identify the preferred option, they need the power of the MultiVariate test, which is only available in the Enterprise Edition.
• The Enterprise Edition is fairly expensive

Summing it up

Optimizely is a great A/B testing tool for any company – whether it is an SMB, a sophisticated e-commerce platform, or any other global enterprise. Giving it a shot is a great idea when you don’t want to waste much time in hard-core programming.

Visual Website Optimizer

Similar to Optimizely, VWO also enables testing for the ones who do not have the programming background. VWO is definitely a good option if you are a small team and have limited bandwidth. If you are interested in understanding your SEO/SEM initiatives and are getting traction, you can easily set up revenue goals to make quick decisions with VWO.

Strengths

A lot of VWO’s features are standard even in the free trial version, which is primarily made for small businesses with limited traffic. One can start a standard A/B test and morph it into a more complicated Multivariate test to understand the interaction effects. Some of its strengths are:

• The tool is very simple and intuitive to use.
• One can start to play with the GUI and set up simple experiments even if they do not have much of HTML/JavaScript knowledge.
• The customer success management (CSM) team is very knowledgeable and responsive.
• Heat map comes integrated and pre-built within the tool.
• Third party heat map integration options such as ClickTale, Crazyegg, etc. are also available for deeper insights into future redesigns.

Limitations

VWO lacks a lot of sophistication that Optimizely provides. The results presented by VWO seem to be much less refined, a lot of segmentation data that is standard in the case of Optimizely doesn’t exist in VWO. Let’s have a look at some of its limitations:

• VWO follows a fixed horizon testing, wherein a user has to calculate the amount of traffic, hence test duration prior to running the test.
• It evaluates the experiment data at one point in time at a set sample size. In contrast, Optimizely uses sequential testing and is designed to evaluate experimental data as it is – enabling the user to test multiple variations without worrying about the validity of the test.

Summing it up

If you are a novice in the testing world and are less versed with programming, the interface like that of VWO can be a good choice. For that matter, even if you are well versed with coding but have a less aggressive testing plan, VWO is again a good option.

On the whole

If you are ready to plan and use an aggressive testing program to drive your conversions, Optimizely is undoubtedly a preferable choice that leverage its proprietary stats engine to offer the ability to conduct more tests with the same amount of visitor traffic.

Want to start with A/B testing right away?