Different Open Rates on an AB Test with Same Subject Line and Same Email Design

Hi!
We are observing a high difference of Open Rates (a difference of up to 10 points of percentage) between two variants of an email, which have the same Subject Line, same Preheader and same Email Layout.

The only thing that differs between the two variants are the dynamic offers in the emails, but the design and layout is exactly the same.

We have used the variant split of Braze canvases to randomly distirbute the variants.

After 2 weeks of test, the Open Rate seems to start stabilizing between the two variants.

Has anyone already experienced such a use case with Braze?

Regards,
Liva

Find more posts tagged with

Reporting and Analytics

Comments

TedScott

What is the quantity for each split? If it's not a large quantity, a variation is not unexpected.

In part you have likely answered your own question, they are not the same (you are inserting dynamic offers).

Manoj__

Hello @Liva - My guess is that one variant was sent to a highly engaged audience that checks their emails frequently, while the other was sent to a less active audience.

Over two weeks, the open rates evened out as recipients from both variants had sufficient time to check their emails.