I think any team that runs only one or two experiments a month should reserve one test in four for what happens after the sale, and write that quota into the calendar. Figures published this month show how rarely retention gets a slot.

What the numbers say

Wingify’s State of Experimentation 2026 report analysed feature experiments run by brands on its platform and found that 45% focus on conversion rate optimization. Retention and engagement get 6%. Tests of whether users adopt new features get 5%. The report also says 22% of all tests in its dataset target average order value.

The same report surveyed 108 organizations that use the platform. Of those, 56% run only one or two experiments a month, and 55% said limited bandwidth and human resources hold them back. Money is less of a limit. In the survey, 62% said their experimentation budget stayed the same year over year and 26% said it increased.

Advertisement

MarTech Your brand belongs here. Reach the decision-makers who read MarTech every day. Premium placements across the site and newsletter. Advertise with us

Put those figures together and a typical team has about 24 test slots a year. Roughly 11 go to the path to purchase, and one or two go to keeping the customer.

My position

I would set a quota. One test in four goes to a retention, engagement or feature-adoption question, and the quota is fixed before the quarter starts. For a team at two tests a month, that is six such tests a year where the report’s average would give two or three.

The first reason is cost. Every customer who does not return has to be replaced through paid acquisition, and the report itself notes that customer acquisition costs “are continuously climbing”. A checkout test that lifts conversion pays once per order. A change that brings a buyer back for a second order pays every time that buyer returns.

The second reason is measurement. Jin Ma, Director of Experience at Dentsu, is quoted in the report: “Setting a consolidated KPI, whether it is revenue per visitor, cost per acquisition, or customer lifetime value, creates a unified definition of success.” A programme whose tests almost all end at the order confirmation page has already chosen its KPI in practice, and it is conversion rate.

The strongest counter-argument

The best case against a quota comes from the report. It argues that brands should “first focus on their core transaction funnel”, because otherwise acquisition and retention investments both return less. Conversion tests also read out quickly. A checkout change shows its effect within the session, while a retention test has to wait for the customer to come back, and a team with two slots a month cannot leave one occupied for a quarter.

There is a second objection. Wingify sells experimentation software, and a finding that brands under-test retention and feature adoption suits its product. The survey respondents are its own customers rating themselves.

Newsletter

Get the week's best tech coverage.

Free. Read by thousands of HR, tech, and business leaders.

I accept the first point for any site whose checkout is visibly losing orders. Fix that first. For everyone else, the speed objection has a practical answer, which is to pick retention tests with an early signal. Second-visit rate within 14 days, completion of onboarding and first use of a reorder button all move before lifetime value does. As for the vendor’s interest, the direction of the finding can be checked against your own test log in an afternoon.

What to do this quarter

Count the last 12 months of tests by goal. If fewer than one in four addressed a returning customer, book the next open slot for one. Use the report’s own advice to choose it: score candidate tests on impact, confidence and ease. Formal prioritization was the lowest-scoring of the 17 capabilities in the survey, so most teams have room here.

Then give the retention tests somewhere to run. Loyalty budgets are expected to grow, and a loyalty programme is a logged-in audience that can be split and measured. Our earlier piece on asking a loyalty vendor for offer relevance next to scan rate describes one metric worth testing against.

Report the result in the same place as the checkout tests. A retention test that sits in a separate deck will lose its slot the first time the calendar gets tight, and the quota only works if the head of marketing sees both kinds of result side by side each month.

Agree the success metric before launch, as argued in Keep One Written Copy of Your Conversion Definition, so that a retention win is not argued over after the fact.

Source: Wingify, The State of Experimentation 2026