Google Tag Manager

Forum Discussion

Liva's avatar
Liva
Collaborator
2 months ago

Different Open Rates on an AB Test with Same Subject Line and Same Email Design

Hi!
We are observing a high difference of Open Rates (a difference of up to 10 points of percentage) between two variants of an email, which have the same Subject Line, same Preheader and same Email Layout.

The only thing that differs between the two variants are the dynamic offers in the emails, but the design and layout is exactly the same.

We have used the variant split of Braze canvases to randomly distirbute the variants.

After 2 weeks of test, the Open Rate seems to start stabilizing between the two variants.

Has anyone already experienced such a use case with Braze?

Regards,
Liva

Reporting and Analytics
  • TedScott's avatar
    TedScott
    Collaborator
    2 months ago

    What is the quantity for each split? If it's not a large quantity, a variation is not unexpected.

    In part you have likely answered your own question, they are not the same (you are inserting dynamic offers).

  • Manoj__'s avatar
    Manoj__
    Expert
    2 months ago

    Hello Liva - My guess is that one variant was sent to a highly engaged audience that checks their emails frequently, while the other was sent to a less active audience.

    Over two weeks, the open rates evened out as recipients from both variants had sufficient time to check their emails.