Hi there,
I was just wondering if anyone could clarify whether or not user selection for canvases / campaigns is randomised, or will users always be entered into a canvas in the same order after running through segments and filters?
I thought I'd read somewhere that user entry into a canvas is random, but now I can't find reference to that anywhere. Basically, if we had a canvas that entered a given number of users every day based on some criteria, would it always be the same set of users that are entered, or would this be random?
Thanks in advance.
Exciting update Malcolm_Murray 🙌🏻
5th Canvas trigger: user 5
I'm more inclined to believe that user entry is random each time!
Braze support have just got back to me and confirm that user entry is indeed randomised, so this backs up your findings AllanHeoFNM. The response from support on this was:
"Regarding the entry of users into your canvas, it's indeed will be randomized based on the specific filters you have set for your canvas. However, in your particular case, it's also worth noting that some users who have previously entered your canvas might still qualify to re-enter it after 2 days, depending on the re-eligibility setting you have configured."
So, this is good news as it appears that canvas entry does indeed work in the way that I'd hoped. Was a bit worried otherwise as it changes some major assumptions, that may have involved re-working some of our strategies.
Thanks all for this, particularly AllanHeoFNM for your tests to confirm this behaviour. Really useful.
Here the answer from the Braze documentation:
With regards to multivariate testing, Braze determines variant re-eligibility for all campaigns, triggered in-app messages, and Canvases using the following rules:
So no, users do not have the option to enter a new variant to re-engage in the canvas or campaign. Antoher article from Braze support:
Thanks for the question. Malcolm_Murray, because I thought Braze randomize the variant for every entry. Now I have to rethink a few of my strategies :D.
Maybe as feedback here MaggieBrennan: It would be great if Braze would have the opportunity to randomize every new entry.
Hey Max ! So I'd love to confirm with Malcolm_Murray , but my understanding was that he was asking for a slightly different question. It sounded like he has a Canvas that lets x users enter each time the Canvas fires, and he's wondering if it'll be the same x people that gets entered each time. Malcolm_Murray can you confirm? I'm also about to test this right now.
However, Max your discovery is also a very helpful/interesting one! I believe you have the option of turning on the randomization at the dashboard level by reaching out to Braze. Meaning once you enable the random variant entry for one Canvas, that will apply for every Canvas that you build in Braze. Hope this helps!
Malcolm_Murray I just created 5 test users and launched a Canvas that enters 1 random user each day. I'll let you know about my results in a few days! 🙂
Braze support have just got back to me and confirm that user entry is indeed randomised, so this backs up your findings AllanHeoFNM. The response from support on this was:
"Regarding the entry of users into your canvas, it's indeed will be randomized based on the specific filters you have set for your canvas. However, in your particular case, it's also worth noting that some users who have previously entered your canvas might still qualify to re-enter it after 2 days, depending on the re-eligibility setting you have configured."
So, this is good news as it appears that canvas entry does indeed work in the way that I'd hoped. Was a bit worried otherwise as it changes some major assumptions, that may have involved re-working some of our strategies.
Thanks all for this, particularly AllanHeoFNM for your tests to confirm this behaviour. Really useful.
Glad to hear it all! Happy to help!
haha I'm like you Malcolm_Murray I'm pretty sure I read random as well and I can't remember where.
You could always stop users from re-entering, which would stop users who have already been through re-entering even if it is random, assuming that's the goal. Off the top of my head, you could also potentially set it up to let certain random bucket numbers in each day, and once all buckets had entered it would go back to the start (untested).
Hey Malcolm_Murray , Braze support can definitely confirm this for you! Please keep us posted on the answer. I think this can also be tested. I'll let you know if I'm able to find out.
Hi AllanHeoFNM - yes that's right, I'm wanting to confirm if users are entered into a canvas randomly each time a canvas is run. The reason being that I've got some daily API driven canvases that target the same set of users. I use connected content to generate message content, and in some cases this can abort the run for a given user if they don't meet some other criteria on our API. If a user has been sent a message, then they won't be sent the same one again, and we limit the daily sends. Users then become re-eligible to receive after a certain amount of time has passed (time since message sent).
What I'm worried about is that if user entry is not random, then over time we'll basically end up only trying to send to people who cannot receive the message. If it's random then this is less of an issue. I had assumed user entry was random (I was sure I read that somewhere) but now I'm not so sure.
I'm in contact with Braze support about this, but haven't got anywhere yet. I'll update here when I do.
I'd be really interested to hear how your experiment goes.
Hey Malcolm_Murray ! Thanks for the clarification and extra context. Knowing that you're working with an API-triggered Canvas, that makes testing a bit easier and faster as I won't have to wait for daily tests 🙂
Here's what I did for my test. I created an API-triggered Canvas with 5 eligible users, and I made the Canvas "Maximum users to potentially enter this Canvas = 1" in the Audience page. This means that each time my Canvas is triggered, it should get sent to 1 out of the 5 eligible users. What we're testing for is whether that 1 user will be random each time, or if it'll be the same user each time.
1st Canvas trigger: user 1
2nd Canvas trigger: user 2
3rd Canvas trigger: user 1
4th Canvas trigger: user 1
I'm limited to one trigger per hour (Braze limitation), so I haven't been able to collect as much data as I would've liked so far... I want to say it's random just because of the single occurrence of user 2, but user 1 is getting entered suspiciously often. I'll do some more tests and let you know, and also, please let me know if you hear from Braze!
Exciting update Malcolm_Murray 🙌🏻
5th Canvas trigger: user 5
I'm more inclined to believe that user entry is random each time!
