> For the complete documentation index, see [llms.txt](https://help.recart.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://help.recart.com/list-growth/testing-and-scheduling/popup-a-b-testing.md).

# Popup A/B Testing

Explains how to create a popup A/B test with the A/B test wizard, choose what counts as a win, and read the results.

Test two or more popups against each other to see which one brings in more opt-ins, orders or revenue. You set up the whole test in one place — the variants, the traffic split, what counts as a win and when it runs — from the **Popup A/B Tests** page. Before you start, you'll need at least two popups to compare; they can be active, inactive or drafts.

### What you can test

You can test elements like:

* Entry timing
* Layout
* Incentives
* Consent method
* Opt-in flow

Each additional variant splits your traffic further, which can affect how quickly you reach statistically significant results.

#### Example: one change per test

Change one thing per test. If you change the delay and the offer at the same time, you can't tell which change caused the difference. Each row below is a separate test:

| What you're testing | Current popup   | New popup variant     |
| ------------------- | --------------- | --------------------- |
| Entry timing        | 10-second delay | 4-second delay        |
| Layout              | Lightbox        | Fullscreen            |
| Opt-in flow         | One-step opt-in | Two-step opt-in       |
| Consent method      | Reply in SMS    | In-popup confirmation |
| Incentive           | 10% offer       | 20% offer             |

### Create an A/B test

1. Go to [**Opt-in Tools**](https://app.recart.com/optin-tools), then the [**Popup A/B Tests**](https://app.recart.com/optin-tools/experiments) tab
2. Click **Create new** to open the A/B test wizard
3. Enter a **Name** for the test and add any **Tags**

   ![](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2FouKjCUdNzV8KeI7C5WlK%2FScreenshot%202026-09-29%20at%2014.57.42.png?alt=media\&token=7bd9b6be-19fd-47fb-975e-f1918ea5045f)
4. Under **Test participants**, click **+ Add variant** and select whether you want to test **Mobile** or **Desktop**

   ![](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2Fi7SBOjNIiHJLLQS6eju8%2FScreenshot%202026-09-29%20at%2014.57.50.png?alt=media\&token=1a8bdfad-14c6-4fd2-8e91-f9c35e91c34c)
5. Select the popups you want to compare
6. Choose the **Traffic split** — default is evenly split between variants — click on **manual** if you'd like to update this

   ![](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2F5lA7TzrIbeBbUUQw1ECq%2FScreenshot%202026-09-29%20at%2015.10.08.png?alt=media\&token=0ac126d8-0bca-448b-a27e-0d5be35bc155)
7. Choose the **Primary metric** — what counts as a win

   ![](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2FoxaBRKqdhcVaQC5N0vvI%2FScreenshot%202026-09-29%20at%2014.58.33.png?alt=media\&token=77a1cf92-6d08-4902-a78f-9a1632cea36e)
8. Set the **Start date** and **End date**

   ![](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2FwZDytbSf37IMfBLSmJmj%2FScreenshot%202026-09-29%20at%2014.58.40.png?alt=media\&token=07b446db-ff5e-4a32-9938-d547ef42b688)
9. Check the **Pages** and **Geo targeting** for the test

   ![](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2FFpm8pOBkKw276XCvBlY3%2FScreenshot%202026-09-29%20at%2015.02.17.png?alt=media\&token=5ffe7cb8-3472-4a9b-9b74-04b9fa40d0f6)
10. Add any **Notes**

    ![](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2FDwif8MeXqnnl7gbxQPUP%2FScreenshot%202026-09-29%20at%2014.58.59.png?alt=media\&token=5163cc00-ff90-494f-9632-04f9fca5a1de)
11. Click **Create** — or **Save as draft** to finish later

{% hint style="info" %}
If your account had a test running or scheduled when the wizard launched, it moves over to the wizard once that test ends. If you don't see **Create new** on the Popup A/B Tests page yet, contact the Recart Support team.
{% endhint %}

### Additional notes per step

#### Test participants

* Click **+ Add variant** to choose from your active, inactive and draft popups. You can filter by channel and status, or search by name
* When you add the first variant, you'll choose whether the test is for **desktop** or **mobile**. Run separate tests for each device
* The first popup you add becomes the **Control**. To change it, open the **…** menu on another variant and select **Set as control**
* A test needs at least 2 participants and can include up to 8 popups
* A popup that's already part of another in-progress, scheduled or draft test can't be added
* To test a brand-new design, create it as a draft popup in the popup editor first, then add it here

**Draft popups** can join a test and are activated automatically when the test starts. If a draft is missing something it needs — for example a welcome flow message — you'll see a warning on its row, and you'll need to fix it before you can schedule the test. Once a draft popup has been activated by a test, it can be set to inactive later but can't go back to draft.

#### Traffic split

* **Auto** splits traffic evenly across all variants and updates as you add or remove them
* **Manual** lets you set a percentage for each variant. The percentages must add up to exactly 100%

#### Primary metric

Choose the one metric that decides the winner: SMS opt-in rate, Email opt-in rate, Order conversion rate, Revenue per visitor, Average order value, Total revenue or Total orders. It's marked as the **Primary metric** on the results page. Every other metric is still tracked and shown.

{% hint style="info" %}
**Choosing a primary metric that fits your test**

* **Using a manual split?** Avoid Total revenue and Total orders — a variant with more traffic will win on totals regardless of quality. Use a rate instead, like Revenue per visitor or Order conversion rate
* **SMS-only or email-only variants?** Don't pick the opt-in rate for the channel they don't collect — it will always be 0
* **Variants collect different channels?** Pick a metric every variant can produce, or a revenue metric, which works for any channel mix
  {% endhint %}

#### Pages and Geo targeting

The test has its own Pages and Geo targeting, which applies to every variant while the test runs.

* By default, it's copied from the control popup. You can also import the targeting from any other variant
* The country list only shows countries that every variant can target
* While the test is running, each participating popup's own Pages and Geotargeting settings are locked, with a note that they're being overridden by the test
* When the test ends, each popup goes back to its own targeting automatically

Device targeting, Visitor Targeting and trigger settings (delay, exit intent, frequency limits) stay with each popup.

#### Notes and tags

* **Tags** are required to schedule a test. Choose from tags like **Copy**, **CTA**, **Discount test**, **Layout test**, **Teaser test** or **BFCM**, so you can filter your tests later — for example, to find every CTA test you ran before BFCM
* **Notes** are optional. Use them to record your hypothesis or why you ran the test

#### Save as draft

Click **Save as draft** if you're not ready to schedule — for example, if one of your variant popups isn't finished. A draft test doesn't activate any popups or change any targeting. Open it from the Popup A/B Tests list to keep editing.

### Test statuses

A test moves from **Draft** to **Scheduled** to **In Progress** to **Ended**. A scheduled test can also be **Canceled**.

| Status          | What you can still change                                         | What opens when you click it |
| --------------- | ----------------------------------------------------------------- | ---------------------------- |
| **Draft**       | Everything                                                        | The wizard                   |
| **Scheduled**   | Name, notes, tags, start date, end date                           | The wizard                   |
| **In Progress** | Name, notes, tags, end date (extend, or shorten to a future date) | The results page             |
| **Ended**       | Name, notes                                                       | The results page             |
| **Canceled**    | Name, notes                                                       | The results page             |

Variants, traffic split, primary metric and targeting are locked once a test is scheduled, so the results stay reliable. To change them, end the test and create a new one.

* **Cancel** a scheduled test to release its popups. Nothing ran, so each popup goes back to its own targeting and keeps its own active or inactive status
* **End** an in-progress test to stop it. The confirmation shows what will happen to each popup, and you can change any popup by hand afterwards

### Viewing your results

Click any in-progress, ended or canceled test on the [**Popup A/B Tests**](https://app.recart.com/optin-tools/experiments) page to open its results. You'll see each variant side by side against the control, with your primary metric marked, across:

* Impressions
* Opt-ins
* Opt-in rates
* [Bounce rate](/analytics-and-reports/popup-bounce-rate.md) — lower is better, so a variant that beats the control shows in green
* Orders
* Revenue per visitor
* Average order value
* Conversion rate

The results page also includes graphs of opt-in rates, conversions and revenue over time, and an opt-in funnel for each variant.

![](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2FZ1QKCUW24Nod20l6EhZZ%2Fpopup-a-b-testing-8.png?alt=media)

The Popup A/B Tests list shows all your tests, with filters for status and tags and a search by name. Tests are sorted in-progress first, then scheduled, draft, ended and canceled.

### Configuring 3rd-Party A/B Testing

If you're using an external A/B testing tool like Intelligems or Convert, you can connect it to a popup so your testing tool decides which popup each visitor sees. This is set up in the popup's own settings, not in the A/B test wizard, and these tests don't appear on the Popup A/B Tests page.

#### Where to find the settings

1. Go to **Opt-in tools** and open the popup you want to use in your test
2. Click **Edit**, then click **Settings** in the left menu
3. Scroll down to **Third-party A/B testing** and switch it to **Active**
4. Fill in the fields described below
5. Click **Save** at the top right

#### What to fill in

The numbers match the screenshot below.

1. **Javascript value to access A/B testing** is the place on your website where your testing tool keeps the group each visitor is in. See [Values for common tools](#values-for-common-tools) for the value to use
2. **Variant ID** is the ID your testing tool gives to the group this popup belongs to
3. **Experiment ID (optional)** is the ID of the whole test. Some tools need it and some don't, as shown in the table below

**Default visibility** is what Recart does if it can't work out which group a visitor is in. Choose **Hide popup** or **Show popup**.

![The Third-party A/B testing panel in the popup settings, with numbers marking the Javascript value, Variant ID and Experiment ID fields](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2F8aL9M0ZYTbUOKOiMfrfQ%2Frecart-panel-intelligems.png?alt=media)

#### How Recart uses these values

* **Your testing tool stores the visitor's group:** Recart compares it with the Variant ID. If they match, the popup shows
* **Your testing tool has to be asked:** Recart asks using your Experiment ID, then compares the answer with the Variant ID. If they match, the popup shows
* **Nothing comes back:** Recart uses your **Default visibility** setting

The Variant ID has to match exactly what your testing tool returns for that group.

#### Values for common tools

| Tool        | Javascript value                                              | Experiment ID field      |
| ----------- | ------------------------------------------------------------- | ------------------------ |
| Intelligems | `window.igData.getTestGroup`                                  | **Required.** Fill it in |
| Convert     | `window.convert.currentData.experiences.<your-experiment-id>` | Leave it **empty**       |

Using a different tool, such as VWO? [Contact the Recart Support team](https://help.recart.com/contact-recart-support) and share your tool's documentation. We'll confirm the right value for you.

#### Example: Intelligems

1. In Intelligems, open your A/B test and click the **three dots** in the upper-right corner

   ![The three dots menu in the upper-right corner of an Intelligems test](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2FhsBPULfn1Xn8E7gSWpwF%2Fig-1-three-dots.png?alt=media)
2. Click **Show Info**

   ![The Show Info option in the Intelligems menu](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2FluIyRV5HVBFEGGupfMfv%2Fig-2-show-info.png?alt=media)
3. Copy the **Experiment ID**. Then copy the ID of the group this popup belongs to (for example **New Group 1 ID**). Use the long ID, not the short **For GA** one. Each popup in your test gets the ID of its own group

   ![The Intelligems information window showing the Experiment ID and the group IDs](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2FVV7qZBCk3yQKiANEfwgu%2Fig-3-info-modal.png?alt=media)
4. In Recart, open the popup's **Third-party A/B testing** settings and switch it to **Active**
5. Enter `window.igData.getTestGroup` as the **Javascript value**, the group ID as the **Variant ID**, and the Intelligems Experiment ID as the **Experiment ID**. The Experiment ID is required for Intelligems
6. Choose your **Default visibility**, then click **Save**

#### Example: Convert

1. In Convert, open your project, go to the **Experiences** tab and open your experience. The number next to **A/B - ID** is your experiment ID

   ![The experiment ID shown at the top of a Convert experience](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2F3ikNzORvNRQ03zTTV9q9%2Fconvert-1-experience-id.png?alt=media)
2. Click the **three-dot menu** next to the variation you want and select **Preview**

   ![The three-dot menu next to a variation in Convert](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2FeZV99Guo4oMmmAqMTVeo%2Fconvert-2-variation-menu.png?alt=media)
3. In the window that opens, the variant ID is the last number in the preview link, after `convert_v=`

   ![The Convert preview link with the variant ID highlighted at the end](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2FY4pzFxj7R6YANhPs9QzV%2Fconvert-3-preview-link.png?alt=media)
4. In Recart, open the popup's **Third-party A/B testing** settings and switch it to **Active**
5. Add your experiment ID to the end of the **Javascript value**, so it reads `window.convert.currentData.experiences.` followed by your experiment ID. Enter the variant ID as the **Variant ID** and leave the **Experiment ID** field empty. Convert needs the experiment ID as part of the Javascript value

   ![The Third-party A/B testing panel set up for Convert, with the Experiment ID field left empty](https://2167119381-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F2I45aTZ733MBi8Vps3Ib%2Fuploads%2FMPfYZpsaCkhdGlmy1AWN%2Frecart-panel-convert.png?alt=media)
6. Choose your **Default visibility**, then click **Save**

### Troubleshooting

**Getting an error when downloading your A/B test results?**

If you see an error page when trying to export your A/B test results as an XLSX file, check whether your test name contains any special characters (such as `+`, `&`, or `%`). These can cause the export to fail.

To fix this, rename your test to remove the special character — for example, change "Summer + Fall" to "Summer and Fall" — then try downloading the export again.

If you have any questions, do not hesitate to [contact the Recart Support team.](/contact-recart-support.md) We'd be happy to help!


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://help.recart.com/list-growth/testing-and-scheduling/popup-a-b-testing.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.