Forum Discussion
How long do you run A/B or similar tests and why? Use cases?
Hi all,
Curious about how long you run A/B tests, or any other tests for before making a call on the data. I imagine it varies with the use case and would be keen to hear other's thoughts and scenarios.
Thanks 😊
Until I get the result I want 😂
Okay so the real answer would be until I get a statistically significant result. Sometimes that's a single send analysed the following week in 15 min and sometimes it's a longitudinal test that runs for 6 months and takes weeks to unpack. As you said it really does vary based on what you're testing, the size of your (engaged) base and what you're wanting to learn.
Until I get the result I want 😂
Okay so the real answer would be until I get a statistically significant result. Sometimes that's a single send analysed the following week in 15 min and sometimes it's a longitudinal test that runs for 6 months and takes weeks to unpack. As you said it really does vary based on what you're testing, the size of your (engaged) base and what you're wanting to learn.
Statistically significant result. That actually makes a lot of sense to me. Great approach.
As far as getting the results you want, just play with the X and Y axis until the graph goes up. 😆
