SMS
      Back to home
      1. Recart Help Center
      2. SMS

      Popup A/B Testing

      Run an experiment and display two or more different pop-ups concurrently to test which one works best with your clients.

      Table of Contents

       

      What is A/B Testing? 

      A/B testing, also known as split testing, is a method widely used in digital marketing to assess the effectiveness of different variations of a webpage or elements within it.

      When it comes to both desktop and mobile popups, A/B testing becomes a valuable tool for optimizing user engagement and conversion rates. It involves presenting two or more versions of a popup to different segments of your audience and analyzing the performance of each variant.

      This allows you to determine which popup design, content, or call-to-action resonates best with your target audience, leading to informed decisions that enhance the overall user experience and maximize desired outcomes, such as increased sign-ups, sales, or other conversions.

      You are not limited by how many popups you’d like to test at the same time. However, please note that each popup you’re testing will be taking a portion of your incoming traffic- which means, the more popups you test at the same time, the smaller volume of traffic will go into each popup’s testing data. This may ultimately impact how you determine what is statistically relevant.

      How to create a popup experiment

      You can create a popup experiment to test out different versions against your current active popup.

      1. Go to your Opt-in Tools and create a new popup
        AB Test 7

      2. Edit your popup with the differences you would like to compare
        AB PU 1

      3. In “General”, toggle “A/B Testing” on and create a new experiment.
        AB Test 6

      4. Name the experiment, select start and end “date & time”, and click “add”.
        AB Test 2

      5. Save this popup as INACTIVE

        • as you already have an active popup, it is important to save this one as inactive, otherwise, you will have 2 popups showing at the same time which is a bad user experience

        • this will allow your inactive popup to be automatically activated when the experiment begins
          AB Test 3

      6. Go to your current active popup, toggle the “A/B Testing” active, and select the experiment you have created.

        • Save this popup as ACTIVE
        • You will be able to see which opt-in tools are participating in the experiment
          AB Test 4AB Test 5

      7. Your popups will be visible in your opt-in tools. They will remain inactive until the start date and time of the experiment.

        • Your original active popup will still be active
        • The inactive popup will remain so until the experiment begins
        • Once the experiment begins, the inactive popup will automatically be activated by Recart
          AB Test 6-1

      8. Once the experiment is live and throughout the running time, you will see that all popups involved in the experiment are active, and you will see a green dot next to the experiment name under the participating popups
        AB Test 12

      ✏️ Things to note

      • As mentioned above, the inactive popup will automatically be activated once the experiment begins.
      • Once the experiment is complete, the losing popup will automatically be deactivated and the winning popup will remain active.
      • The determining stats will only be pulled for the period that the experiment is running.
      • You can run multiple experiments at the same time, for example, you can have two different experiments running for both mobile and desktop concurrently.
      • If you would like to test two completely new popups, you will first need to create one and activate it before proceeding with the above experiment steps.

      🎥 Watch this video for detailed instructions on how to create your test popups and experiment:

       

      The results of the popup A/B testing

      1. Once your A/B testing experiment has ended, the results will be displayed as a banner in your opt-in tools. You can check the results here.
        AB Test 8

      2. You can compare the impressions, opt-in rates, opt-ins, and new subscriptions of each popup here.

        • This will only reflect data collected during the running time of the experiment.

      3. Recart will calculate the popup that performed the best and the winner will automatically remain active, while the inferior popup is automatically deactivated.

        • You can still manually activate or deactivate these if you choose.
          AB Test 9

      4. You can see the results of all of your experiments at any time by using the filter function

        • Go to the filter drop-down menu

        • Under A/B testing select the experiment you would like to view

        • Click Apply
          AB Test 10

        • The results of your experiment will be shown on this page
          AB Test 11

      🎥 Watch this video for detailed instructions on how to view your experiment results:

      ✨Try it for yourself here with our guided video 👇🏼

       

      In case you need further assistance, feel free to contact our support team, we are happy to help. 😊