Popup A/B Testing

Test and compare two or more popup variations to see which version drives better results with your audience.

Table of Contents

• What is A/B Testing?

• How to Create a Popup A/B Test

  • Part1: Create New Popup
  • Part 2: Update Active Popup

• Important Notes and Best Practices

• Viewing Your A/B Test Results

• Using the “Popup A/B Tests” Page

• Configuring 3rd-Party A/B Testing

What is A/B Testing?

A/B testing, or split testing, is a marketing strategy used to compare different versions of content to see which performs better. In Recart, A/B testing allows you to display multiple popup variations to different segments of your website traffic and measure which version generates more opt-ins, revenue, or conversions.

 

You can test elements like:

• Entry timing

• Layout

• Incentives

• Consent method

• Opt-in flow

 

There is no limit to how many popups you can test at once, but keep in mind: each additional variation divides your traffic, which can affect how quickly you reach statistically significant results.

Example: Testing Two Variants

 

Current Popup	New Popup Variant
10-second delay	4-second delay
Lightbox layout	Fullscreen
One-step opt-in	Two-step opt-in
Reply in SMS	In-popup confirmation
10% offer	20% offer

 

How to Create a Popup A/B Test

To run an A/B test in Recart, you’ll need two popups: one new variant and one that’s currently active. You’ll assign both to the same experiment so Recart can automatically split traffic and compare performance. First, you’ll create a new popup with the changes you want to test, then assign your current active popup to the same experiment.

Part 1: Create New Popup

1. Go to Opt-in Tools and click Create New.

2. Customize the new popup with the elements you want to test.

3. In the popup Settings, toggle A/B Testing to “Active.”

   
   
   

4. Select or Create New Experiment and select the start and end date/time.

   

5. Click Add and then Save as Inactive.

   

It’s important to save the new popup as inactive. When the test begins, Recart will automatically activate the test popup so it can run alongside your existing popup.

 

Part 2: Update Your Current Active Popup

1. Go to the popup currently marked as Active.

2. In Settings, toggle A/B Testing to “Active.”

   

3. Select the experiment you created earlier.

4. Click Save as Active.

Both popups will now be tied to the same experiment. Once the test starts, both popups will be visible in the Opt-in Tools list and served to users based on traffic split.

 

Important Notes and Best Practices

• Create separate experiments for desktop and mobile. Example:

  • Test 1 → Desktop Popup A vs Desktop Popup B

  • Test 2 → Mobile Popup A vs Mobile Popup B

• If testing two brand new popups, you must activate one before creating the test.

• The inactive popup in a test will automatically activate when the test begins.

• At the end of the experiment:

  • The popup with stronger results will remain active.

  • The other popup will be automatically deactivated.

• You can still manually activate or deactivate popups after the test ends.

• A single A/B test can now support up to 8 popup variants.

Viewing Your A/B Test Results

Once a test ends, Recart will automatically calculate which popup performed best. You can view performance by:

• Impressions

• Opt-ins

• Opt-in rates

• Orders

• Revenue per visitor

• Average order value

• Conversion rate

You can still access test data within Opt-in Tools via filters, but for a clearer, visual experience, we recommend using the Popup A/B Tests page.

 

Using the New “Popup A/B Tests” Page

You will find the dedicated A/B testing section inside the Opt-in Tools dashboard.

Where to Find It

• Go to Opt-in Tools in the Recart app

• Click on the Popup A/B Tests tab

This tab provides:

• A list of all active, scheduled, or completed tests

• Filter and search functionality

• An analytics page for each test

  

 

Click on any test to view detailed analytics, including:

• Side-by-side popup variant performance

• Visual graphs of opt-in rates, conversions, revenue

• Opt-in funnel charts to compare user behavior

  

 

Configuring 3rd-Party A/B Testing

If you’re using external A/B testing tools like Intelligems or VWO, Recart supports third-party configuration using JavaScript variables.

Required Information

1. JavaScript Value

   • Used to determine which popup variant to show.

   • Example: window.igData.getTestGroup for Intelligems.

2. Variant ID

   • Unique ID used to identify which popup to show in a test.

3. Experiment ID (Optional)

   • Helps in scenarios with multiple overlapping experiments.

4. Default Visibility Setting

   • Choose whether to show or hide a popup if no result is received from the JS variable.

 

Example: Configuring Intelligems for Recart

To retrieve the correct settings from Intelligems, follow these steps:

1. Open the A/B Test in Intelligems

• • Click on the three dots in the upper-right corner of the test.
  • Select Show Info from the dropdown menu.

2. Retrieve the Necessary IDs

• • Locate the Experiment ID and Variant ID for the test.

3. Use These Values in Recart

• • Ensure the Variant ID matches the value expected by Recart.
  • If needed, adjust the default visibility setting based on your preference.

Where to Configure A/B Testing in Recart?

To set up A/B testing for your popups in Recart:

1. Log in to the Recart App.
2. Navigate to Opt-in Tools and select the popup you want to configure.
3. Under Settings, go to A/B Testing Configuration section.
4. Enter the JavaScript value, Variant ID, and Experiment ID (if applicable).
5. Set the Default Visibility option.
6. Click Save & Apply.

Once configured, Recart will automatically determine which popup variant to display based on the A/B testing setup. If you need help setting up third-party A/B testing, contact our support team for assistance.

 

In case you need further assistance, feel free to contact our support team, we are happy to help. 😊