SMS
      Back to home
      1. Recart Help Center
      2. SMS

      Popup A/B Testing

      Run an experiment and display two or more different pop-ups concurrently to test which one works best with your clients.

      Table of Contents

      What is A/B Testing? 

      • A/B testing, also known as split testing, is a method widely used in digital marketing to assess the effectiveness of different variations of a webpage or elements within it.
      • When it comes to both desktop and mobile popups, A/B testing becomes a valuable tool for optimizing user engagement and conversion rates.
        • It involves presenting two or more versions of a popup to different segments of your audience and analyzing the performance of each variant.
      • This allows you to determine which popup design, content, or call-to-action resonates best with your target audience, leading to informed decisions that enhance the overall user experience and maximize desired outcomes, such as increased sign-ups, sales, or other conversions.
      • You are not limited by how many popups you’d like to test at the same time.
        • However, please note that each popup you’re testing will be taking a portion of your incoming traffic- which means, the more popups you test at the same time, the smaller volume of traffic will go into each popup’s testing data. This may ultimately impact how you determine what is statistically relevant.

      How to create a popup experiment

      You can create a popup experiment to test different versions against your current active popup.

      Desktop example

      Current Active Popup New Popup to Test Against
      • 10-second entry delay
      • 4-second entry delay
      • lightbox layout
      • fullscreen
      • no teaser step
      • teaser step
      • one-step opt-in
      • two-step opt-in
      • reply in SMS as a consent
      • in popup confirmation (with 1-time code)
      • close icon 0-second delay
      • close icon 2-second delay
      • 10% discount offer
      • 20% discount offer

        VS

        Create a new popup


        1. Go to your Opt-in Tools and click Create new

        2. Create your new popup and edit it with the differences you would like to compare

        3. In General, toggle A/B Testing to Active and click Create new


        4. Name the experiment, select the start and end date & time, and click Add

        5. The A/B Experiment will now show → Click Save as INACTIVE

          • as you already have an active popup, it is important to save this one as inactive, otherwise, you will have 2 popups showing at the same time which is a bad user experience

          • this will allow your inactive popup to be automatically activated when the experiment begins



        Update your currently active popup

        1. Go to your current active popup, toggle the A/B Testing to Active, and select the experiment you previously created.

          • Click Save as ACTIVE
          • NOTE: You will be able to see which opt-in tools are participating in the experiment



        2. Your popups will be visible in your opt-in tools. 

          • Your original active popup will still be active
          • The inactive popup will remain so until the experiment begins
          • Once the experiment begins, the inactive popup will automatically be activated by Recart



        3. Once the experiment is live and throughout the running time, you will see:

          1. all popups involved in the experiment are Active

          2. a green dot next to the experiment name under the participating popups


        ❗Important things to note❗

        • You need to create separate A/B tests for mobile and desktop
          • TEST 1  → desktop 1 vs desktop 2 
          • TEST 2 → mobile 1 vs mobile 2
        • As mentioned above, the inactive popup will automatically be activated once the experiment begins.
        • Once the experiment is complete, the losing popup will automatically be deactivated and the winning popup will remain active.
        • The determining stats will only be pulled for the period that the experiment is running.
        • You can run multiple experiments at the same time, for example, you can have two different experiments running for both mobile and desktop concurrently.
        • If you would like to test two completely new popups, you will first need to create one and activate it before proceeding with the above experiment steps.


        🎥 Watch this video for detailed instructions on how to create your test popups and experiment:

         

        View the results of the popup A/B test

        1. Once your A/B testing experiment has ended, the results will be displayed as a banner on your Opt-in Tools Page

          1. You can check the results here

            AB12

        2. You can compare the:

          1. impressions

          2. opt-in rates

          3. opt-ins

          4. new subscriptions


            NOTE: This will only reflect data collected during the running time of the experiment.

        3. Recart will calculate the popup that performed the best and the winner will automatically remain active, while the inferior popup is automatically deactivated.

          • You can still manually activate or deactivate these if you choose

            AB13

        4. You can see the results of all of your experiments at any time by using the filter function

          • Go to the filter drop-down menu

          • Under A/B testing select the experiment you would like to view

          • Click Apply

            AB14

          • The results of your experiment will be shown on this page

            AB15


        🎥 Watch this video for detailed instructions on how to view your experiment results:

        ✨Try it for yourself here with our guided video 👇🏼

         

        In case you need further assistance, feel free to contact our support team, we are happy to help. 😊