Do you use control groups in your Canvas set up?

I tend to run controls on most canvases except those where it is critical for a user to get that information such as a transactional email.

On topic of a true control group, I tend to use 10% as they can be a great measure of how effective the campaign is compared to not sending it all, taking into account all the great points mentioned above. We also run a global control group to see the impact of Braze vs no-Braze at all.

Not to throw a spanner in the works, but if you have been running a campaign for a while then you create a new variant, in theory, the old variant now becomes your control group. We often say this is an A/B test but you are really using the old version as a control vs the new version. So, instead of testing if it is better than not sending the message at all, you are now testing if it is improved over the previous version which is often a better control for campaigns that are ongoing. 

On measurement, I come from a science background and I do believe statistical significance is really important, but I have learnt that even indicators or outliers that are not significant should not be ignored. Not that marketing is like a medication trial, but imagine you run a medication trial that is statistically significant in helping most people vs the control, but you also see 10 people got hospitalised as a result of the new medication but that is not statistically significant, you can't just ignore that figure. Sure, no one is probably going to hospital over your 20% off voucher but I still believe we need to think beyond statistical significancy sometimes. 😊