Advanced A/B Testing Strategy Course

More Winning Tests

Are you tired of A/B tests that don't get a lift? Do you want more out of your split testing than small wins? In this video I will teach you the four steps you can take now to improve the outcomes you get with every ab test. By following these steps you will be able to take your conversion rate optimization program to new heights.

Optimal Number of Variations Per A/B Test?

Did you know there is an optimal number of variations to run per A/B test? Some simple math can help you identify what that sweet spot is so you get more gains with your split testing. More gains per test amounts to better visitor experiences and more conversions for your company. In this video, you will learn several things to improve your testing efforts including: • How more variations increases your success rate • The math behind choosing the optimal number of variations • The diminishing returns of adding more variations • The constraints to consider before building multiple variations

Hypothesis Strategies

What if you could get better split testing results without using a hypothesis? You can because using a hypothesis for CRO has two fundamental flaws. There is a better way to find solutions and identify problems. Using a hypothesis is only hurting your A/B testing results.

Strategies to Avoid Becoming Hypothesis Handicapped

Trying to optimize a digital experience while you are hypothesis handicapped is like trying to live with handcuffs on. This video talks about 4 symptoms of being hypothesis handicapped and 4 ways to be free of it. Optimizers that handicap their hypothesis don't get the gains they otherwise would with their testing because they limit the possibilities of things they try. In this video, we will discuss the makeup of a hypothesis and you will learn what it means to be hypothesis handicapped. We will talk about the symptoms which include: • Testing only what you think will work • Testing a small set of alternatives that all "safely" revolved around your main concept • Test results show exactly what you had hoped would win so you stop testing more options • Not forming any follow-up questions or testing actions At the end, I will also give you four self-evaluation questions to free yourself from being hypothesis handicapped.

Evaluating A/B Tests with the Efficiency Equation

If you want to run good split tests and get more wins with every test, the Efficiency Equation can guide your efforts. This simple equation can help you know which tests are efficient to run and which ones you should avoid. It can also help you know where you should focus your future efforts by doing post-test comparisons. If you want more wins with your conversion rate optimization efforts, then this video is for you. In this video, you will learn several things to improve your A/B testing including: • What the Efficiency Equation is • How to use the Efficiency Equation • The value of pre-test analysis with the equation • Doing post-test comparisons with the equation • A real-life example of the efficiency equation

Ideation Framework for Generating A/B Testing Ideas

Coming up with better split testing ideas will allow you to get better results with your testing. This ideation framework will unlock the creativity you need to generate the best ideas possible. By using a framework for ideation you can get more consistent brainstorm sessions and you will be able to explore opportunities you hadn't thought of before. In this video, you will learn several things to improve your testing efforts including: • The importance of using a framework to unleash creativity • The three steps to use this ideation framework • The four categorical boundaries you should • Several examples of how to apply this in practice

Existence Testing Strategy

One of the most important A/B testing strategies that any conversion rate optimizer can employ is doing existence testing. Existence testing allows you to learn which elements matter and learn the relative value of elements on your site so you can get bigger wins sooner.

Baseline Testing Strategy

In this video A/B Testing Strategy video, you will learn how to get more conversions faster by using the Baseline Testing Strategy. Most split tests don't have a large impact because the test is designed to optimize things that are already mediocre. Use the Baseline Testing Strategy to ramp up your conversion rate optimization results faster.

Self Selection Strategy

Learn about the Self-Selection A/B testing strategy and how you can use it to increase your conversions. Your visitors are coming to your site with needs and problems they are hoping to get solved with your help. Unless you know exactly what each visitor needs you are missing out on optimizing their experience. This video will highlight an a/b testing strategy that can be used to discover your visitor needs and optimize their experience. At the end of the video, we will review a Vitamix case study that increased conversions for the target audience by over 40%. In this video, you will learn several things to improve your testing efforts including: • What the Self-Selection Strategy is • The main benefits of doing self-selection testing over other types of tests • What doing self-selection testing looks like • How doing self-selection testing can increase your conversions

Examples: Baseline and Self Selection Tests

Understanding the strategies and principles of good A/B Testing is important, but sometimes it is helpful to see that strategy in action. This video will review some test results where we used the Baseline Testing Strategy and the Self-Selection Testing Strategy.

The "Guess Who" Testing Strategy

The strategy to win at the game of Guess Who is the same strategy that can help you win at A/B Testing. By employing the Guess Who Testing strategy you will be able to get more wins more often. The strategy says that you should eliminate things that aren't good, not by trying everything out, but by asking bigger broader questions first. If you get too specific too soon, you will fail at testing just like you will lose at the game. In this video, you will learn several things to improve your testing efforts including: • What the Guess Who testing strategy is • How the Guess Who testing strategy works • The benefits of using the Guess Who testing strategy

"My Precious" Testing Strategy

Every business has its "precious" - those things about their site that they think are super important. The "My Precious" A/B testing strategy pushes us to challenge and question those elements we think are most valuable. Why would you want to destroy your precious? Why would you want to remove your best and most favorite elements and pages? In this video, we will answer those questions and more. You will learn several things to improve your testing efforts including: • What the My Precious testing strategy is • How to employ it • The value of doing it • The negative impact of not doing this strategy

Zoom Zoom Testing Strategy

The Zoom-Zoom A/B testing strategy will help you get better testing results by helping you focus on the things that matter most. Too many split testers test things that are so granular that they don't even matter. This results in tests that have low lifts or no lift at all. By following the Zoom-Zoom testing strategy you will be able to identify what matters to your visitors and get bigger gains faster because you will know where the value is. In this video, you will learn several things to improve your testing efforts including: • How to avoid testing things that are too granular • What it means to zoom out with your business question • What a zoom zoom sandwich is • The 4 steps to following the Zoom-Zoom Testing Strategy

Speed vs Control Testing Strategy

Every A/B test has a trade-off of doing something else. There are some testing scenarios where you should control fewer variables in order to gain more speed and go faster. The faster you go, the more you learn, and the more you impact your conversions. In this video, you will learn several things to improve your testing efforts including: • The trade-offs of speed vs control • The two scenarios when you should sacrifice control for speed • The principles you need to know to help you guide your decision of using more speed

Rapid Iteration Strategies

Rapid iteration sounds great in theory, but if it is done wrong it can be devastating for an organization and its customers. Rapidly iterating without the causal data you get from A/B Testing can also be detrimental. Iterating within a strong optimization program is the surest way to iterate rapidly in the right direction. In this video, you will learn several things to improve your testing efforts including: • The limitations of rapid design iteration • How sample size, moderator bias, and going too quickly can be detrimental • How A/B testing solves for the limitations of small group design iteration sessions • Four things to do that will increase your ability to iterate rapidly in your optimization program

CTA Stacking Strategy

Learn about the CTA Stacking Strategy and how you can use it to increase your conversions. Visitors are on a journey as they discover your content, products, and site. At key points along that journey we should be testing to offer them calls to action that help them engage more and continue down a conversion path. This video will highlight an a/b testing strategy that can be used to discover those key points. In this case study, I will show you how we were able to get a 33% lift in product requests, 28% lift in chats, and 29% lift in video views. We also learned where we should place the call to actions, how the timing of those call to actions are important, and how sequencing affects visitor engagement.

"The Impossible" Testing Strategy

As you are doing conversion rate optimization, you will get push back from stakeholders, technical people, and even team members. They will tell you the same thing, that an A/B test can't or shouldn't be done. "We can't build that test," or "So and so doesn't like that split test," or "That A/B test is too complex." This video will address 3 reasons why you should A/B test the impossible. In this video, you will learn several things to improve your testing efforts including: • The three benefits of testing things that people say you shouldn't • How to test something that is impossible • Fake it before you make it strategy • Cautions in testing the impossible

Combining Multiple Testing Strategies

The best conversion rate optimizers use multiple testing strategies to maximize the conversions and learnings with their split tests. In this video, I will show you a few of the strategies we employed to optimize our home page. A/B Testing is a dynamic process and the strategic approaches used should be dynamic as well. Knowing what strategies to use at what time is essential to maximizing impact. In this video, you will learn several things to improve your testing efforts including: • The baseline strategy • The existence testing strategy • Multivariate testing as a strategy • Isolation as a strategy • Iteration as a strategy • etc