Is your exit popup earning money, or giving it away?

The monthly figures look good. The "Leaving already? Here's 10% off" popup issued 900 codes, 310 of them were used, and those orders came to a respectable sum. Someone suggests making the discount 15%.

Before anyone does, it is worth asking a less comfortable question: how many of those 310 people would have ordered anyway? A visitor with £80 in the basket who moves the mouse towards the tab bar is not always leaving. Sometimes they are checking another tab, looking for a code on a voucher site, or just moving the mouse. If the popup catches them and they order with 10% off, the report counts a coupon order. Without the popup, some of them would have paid full price.

Counting coupon orders tells you how popular the discount is. It does not tell you whether the popup made you money. This post is about measuring the second thing on a Magento 2 store, and about choosing who sees an exit offer in the first place.

Why coupon orders are the wrong number

An exit offer has two effects. It brings back some people who really were leaving, which is the point. And it gives a discount to some people who were going to buy anyway, which is the cost. A report that only counts coupon redemptions adds both together and calls the total a success.

The standard way to separate them is a control group. For a share of the visitors who qualify for the offer, you show nothing. Then you compare how often each group ordered within the same period. If 6% of the visitors who saw the popup ordered within a week and 5% of the visitors who did not see it also ordered, the popup is worth about one percentage point of conversion, and you can weigh that against the discount you gave to everyone who used the code.

How the control group works in our module

In our Exit Intent Offers module, every offer has a Control Group (%) setting, from 0 to 90 (default 0). When a visitor qualifies for an offer and its trigger fires, they are either shown the popup or held back and see nothing. Both are recorded. The group is fixed per browser and offer, so a visitor does not flip between groups on every page.

An order counts as a conversion when the same browser, customer account or basket became eligible for the offer within the attribution window before it (seven days by default, 1 to 90). Shown and held-back visitors are measured in exactly the same way. Orders that used the offer's own code are counted separately as coupon orders, without a time limit.

The conversion report then shows, per offer: eligible visitors, shown and held back, claims, coupon orders and their revenue, the conversion of each group, the uplift (conversion shown minus conversion held back, in percentage points) and the revenue per visitor in each group, with a daily chart and CSV export.

The honest caveat, which the module's guide also makes: a small control group needs a lot of visitors before the difference means anything. With 10% held back, a quiet shop may wait weeks for a reliable answer. Keep the offer unchanged while you measure it, or you are comparing two different offers. And a guest who deletes cookies gets a new draw, while visitors who block cookies are never held back, so the groups are not perfect. They are still far better than no comparison at all.

Showing the offer to the right visitors

Measuring the offer is half of it. The other half is not showing a discount to people who do not need one. Each offer has an audience, and all conditions must match: customer groups (guests included), empty or non-empty basket, basket subtotal from and to, number of items, new or returning visitors, a minimum number of visits, a minimum number of page views, the landing utm_source (only visitors who arrived from your newsletter, for example), page types and device. The checkout never shows an exit offer.

Basket, customer group and segments are checked on the server. Page, device and visit history come from the browser and only decide which offer fits. When several offers fit, the one with the highest priority wins, and it also competes with the popups of other CommerceGears promotions, so only one promotion popup opens per page view.

That lets you be specific. A coupon for baskets over £50 from first-time visitors. A newsletter sign-up instead of a discount for empty baskets. A plain message and a link to your delivery page for returning customers, with no discount at all. Each can have its own control group and its own line in the report.

What the offer can be

An offer is one of four things. A personal coupon: percent off, an amount off the basket or free shipping, with an optional minimum subtotal and a validity in days. Each visitor gets one single-use code; guests enter their e-mail address, logged-in customers just claim it. Asking again in the same browser or account shows the same code, and someone who only types an e-mail address that already has a code is told the offer was claimed and does not see the code. An e-mail sign-up, stored as a lead, with newsletter subscription when the visitor ticks the consent box. A message with a link. Or, when those modules are installed, a Spin to Win wheel or Scratch & Win card, with the game's own prizes and limits.

Triggers are the mouse leaving the top of the window on desktop, a fast scroll up on touch screens, inactivity, coming back to the tab, and optionally the back button on phones. There is a minimum time on the page before anything fires, and a frequency: once per visit, once every N days, or once per visit until the visitor claims the offer or orders.

Limits worth knowing

Exit intent on phones is a guess. There is no mouse, so a fast scroll up or the back button stands in for it. The back-button trick adds one entry to the browser history, some browsers ignore it, and some visitors find it annoying, which is why it is off by default. Orders from another device by a guest are not matched to the popup. And game offers count their prizes in their own module; the exit report counts visitors and orders.

If you run one exit coupon for everyone and you are content to judge it by redemptions, you do not need any of this. The point of a control group is to find out whether the discount is worth what it costs, and to have the answer before the next person suggests making it bigger.

The Exit Intent Offers module page has the full audience options, the report fields and screenshots of each popup layout.