Testing, Testing, 123 or ABC?

There’s only a few times in my career as a web developer that I can really say it’s exciting, and 90% of those times is when something has gone terribly wrong. But probably one of the other times I get excited is when I’m running site tests. I love testing, I love seeing what and why a variation might be winning and I love to theorize and continue to build out tests. I know that might not be normal, but whatever.

Small Improvements

What I love about A/B, split-tests and multivariate tests are the ability to make small changes and see if, and how they impact your users. You don’t have to completely redesign your whole site just see it was a bad idea (Yes there are such things as bad ideas). Once I have a preferred layout and style guide picked out, I like to test placement of CTA’s, Landing page copy, Banners, Headlines, etc. I usually only make a couple small changes per variation so the increase or decrease in performance can be easily attributed to those changes.

Failures are really successes

The whole point of testing is to gather data, once you gather data you can start to form some information out of it. A change that has failed to increase your goals is not a failure, it’s increased your data, it’s increased your knowledge and has shown you that the path your on probably isn’t the best one. That’s a WIN! There’s something to the saying “You never know until you try it.” That being said, it should really be “You never really know until you test it.” Once you do really know, write it down and try something else.

The key to overall success is to fail quickly and learn from your mistakes. You win the game by making small incremental improvements over time, think of…. Evolution! Testing allows you to try something without risking go extinct.

Test Methods

Now days there are many online tools for testing, some are expensive, some are free. I’ve used many and I think a few are better than others, I’ll link some below with my thoughts.

The key to getting good tests though is getting enough data to get statistically significant results, meaning they really mean something and you can rely on them and aren’t just an anomaly. If your site doesn’t get a lot of traffic then it does become more difficult to reach statistically significant results and the truth is, you might not.

In cases where you have low traffic, I recommend sticking with A/B tests so you don’t have to spread your already thin audience. If you still can’t reach it, then I’d run your test with 100% of your traffic and examine the analytics and see if your goals are increasing or decreasing, If the latter, then change it back and try something else. No that’s not the ideal testing scenario, but it will work albeit slower. Another option is to use CPC to drive some traffic to your site during testing, yeah it’ll cost you but it may save you time. What do you have more of, time or money?

If traffic isn’t your issue, then go for it! I’d be setting up multivariate tests with as many meaningful changes as I could make, then as combinations start winning, I’d start mashing them up into new combinations and see if further improvements can be gained.

What’s your favorite things to test? Goals you test for? Let me know, I’d love to see what other people are doing with split testing

Test Providers

https://www.google.com/analytics – It’s ok, it used to be WAY better. You can only do split-tests and they’re redirect tests so they have to be complete pages. But it is Free.

https://www.optimizely.com – Probably the best, does A/B, Multivariate, Redirect and On-Page. When I used them they were relatively new, but the’re platform was hands down the best I’ve used.

https://www.vwo.com – Another great choice, I haven’t used them in awhile but they also have a good interface with all the different test type options.

I’m not getting paid by any of these providers, FYI… But if you are one of them I’d gladly take your money.

Any others you recommend? Let me know in the comments section.

0 0 votes

Article Rating