How to A/B test checkout upsells

by

Jaap Vergote

|

November 15, 2022

To find which upsell works best in your checkout, you can now use A/B testing within UpsellPlus.

To enable A/B testing, go to the UpsellPlus app. Create an offer, choose ‘Checkout’

Once you’ve added a product and set up the design and discounts you want, you’ll see the following option.

a/b test checkout upsells

Click the button! The app will copy your current offer and create a B version. You can toggle between the two versions with the tabs at the bottom. 

Here’s an example of a simple A/B test setup that will test the callout banner that introduces the upsell offers in checkout: ‘Frequently bought together’ vs ‘73% of customers also bought’

configure your a/b test

You can also see the impressions, conversion and revenue for each version. Once you publish the offer to your checkout, 50% of shoppers will see version A and 50% of shoppers will see version B. 

The smart rules are not part of the A/B version testing as that would invalidate the 50/50% visibility for shoppers.

Some tips to run an effective A/B test:

1) Set a goal and hypothesis

To get a useful result out of an A/B test, define your hypothesis. In simple words, say what you want to test:

“Changing the discount from 20% to 30% of my socks checkout upsells will increase the conversion rate from 3% to 5%”

“The CTA button text ‘ADD TO CART’ will convert 1% higher than ‘add to cart’”

“A stack layout for my multiproduct checkout upsell will earn 20% more revenue than a carousel layout”

This will help you understand what the goal was of the test when you come back to it. You can add this hypothesis as the internal name of the upsell offer at the very top of the page.

example of a/b test

2) Only test one variable at a time

Choose 1 variable to change between the two versions and let the test run until you hit at least 30 conversions.

Ideas:

  • Discount %: 20% vs 30%
  • Callout banner text
  • Design
  • Button CTA text
  • Button color
  • Upsell product
  • Multiproduct layout: stack vs carousel
  • Upsell description
  • Upsell marketing message

3) Get a statistically significant result

The UpsellPlus app will declare a winner after a statistically significant result has been reached for your store. It will indicate a winner has been found with the ‘trending’ badge next to the winning version.

finding the successful version in an a/b test

This badge will also show in your offer overview to notify you that a winner has been determined.

a/b test analytics

Note that finding a statistically meaningful result might take a while if you’re having a low conversion rate for the upsells. Give the A/B test enough time to produce useful data.

4) Create a 'control' and a 'challenger'

When you had an upsell run without an A/B test, keep that unchanged (‘control’) and add a ‘challenger’ with one variable that’s updated to test a hypothesis.

Things to note:

  • When you choose to run an A/B test, you can no longer switch the offer from a manually chosen product to an automagically (AI) chosen product. This would invalidate the A/B test as too many variables would change.
← BACK to Blog

Join 1000+ brands already using UpsellPlus

The world's greatest Shopify Plus brands use UpsellPlus. Join us today!

Try for free