nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Randomisation of users filtered by a segment for a canvas or campaign

Malcolm_Murray

Hi there,

I was just wondering if anyone could clarify whether or not user selection for canvases / campaigns is randomised, or will users always be entered into a canvas in the same order after running through segments and filters?

I thought I'd read somewhere that user entry into a canvas is random, but now I can't find reference to that anywhere. Basically, if we had a canvas that entered a given number of users every day based on some criteria, would it always be the same set of users that are entered, or would this be random?

Thanks in advance.

Find more posts tagged with

Segmentation

Accepted answers

AllanHeoFNM

Exciting update @Malcolm_Murray 🙌🏻

5th Canvas trigger: user 5

I'm more inclined to believe that user entry is random each time!

Malcolm_Murray

Braze support have just got back to me and confirm that user entry is indeed randomised, so this backs up your findings @AllanHeoFNM. The response from support on this was:

"Regarding the entry of users into your canvas, it's indeed will be randomized based on the specific filters you have set for your canvas. However, in your particular case, it's also worth noting that some users who have previously entered your canvas might still qualify to re-enter it after 2 days, depending on the re-eligibility setting you have configured."

So, this is good news as it appears that canvas entry does indeed work in the way that I'd hoped. Was a bit worried otherwise as it changes some major assumptions, that may have involved re-working some of our strategies.

Thanks all for this, particularly @AllanHeoFNM for your tests to confirm this behaviour. Really useful.

All comments

DavidO

haha I'm like you @Malcolm_Murray I'm pretty sure I read random as well and I can't remember where.

You could always stop users from re-entering, which would stop users who have already been through re-entering even if it is random, assuming that's the goal. Off the top of my head, you could also potentially set it up to let certain random bucket numbers in each day, and once all buckets had entered it would go back to the start (untested).

AllanHeoFNM

Hey @Malcolm_Murray , Braze support can definitely confirm this for you! Please keep us posted on the answer. I think this can also be tested. I'll let you know if I'm able to find out.

Max

Here the answer from the Braze documentation:

With regards to multivariate testing, Braze determines variant re-eligibility for all campaigns, triggered in-app messages, and Canvases using the following rules:

When variant percentages are not changed, each user will always enter the same variant of a campaign, triggered in-app message, or Canvas entry every time they are re-eligible.
If the variant percentages change, users may be redistributed to other variants.
Control groups will remain consistent if the variant percentage is unchanged, and no users who previously received messages will ever enter the control group on a later send, nor will any user in the control group ever receive a message.

So no, users do not have the option to enter a new variant to re-engage in the canvas or campaign. Antoher article from Braze support:

Screenshot 2023-10-10 at 18.27.02.png

Thanks for the question. @Malcolm_Murray, because I thought Braze randomize the variant for every entry. Now I have to rethink a few of my strategies .

Maybe as feedback here @MaggieBrennan: It would be great if Braze would have the opportunity to randomize every new entry.

AllanHeoFNM

Hey @Max ! So I'd love to confirm with @Malcolm_Murray , but my understanding was that he was asking for a slightly different question. It sounded like he has a Canvas that lets x users enter each time the Canvas fires, and he's wondering if it'll be the same x people that gets entered each time. @Malcolm_Murray can you confirm? I'm also about to test this right now.

However, Max your discovery is also a very helpful/interesting one! I believe you have the option of turning on the randomization at the dashboard level by reaching out to Braze. Meaning once you enable the random variant entry for one Canvas, that will apply for every Canvas that you build in Braze. Hope this helps!

AllanHeoFNM

@Malcolm_Murray I just created 5 test users and launched a Canvas that enters 1 random user each day. I'll let you know about my results in a few days! 🙂

Malcolm_Murray

Hi @AllanHeoFNM - yes that's right, I'm wanting to confirm if users are entered into a canvas randomly each time a canvas is run. The reason being that I've got some daily API driven canvases that target the same set of users. I use connected content to generate message content, and in some cases this can abort the run for a given user if they don't meet some other criteria on our API. If a user has been sent a message, then they won't be sent the same one again, and we limit the daily sends. Users then become re-eligible to receive after a certain amount of time has passed (time since message sent).

What I'm worried about is that if user entry is not random, then over time we'll basically end up only trying to send to people who cannot receive the message. If it's random then this is less of an issue. I had assumed user entry was random (I was sure I read that somewhere) but now I'm not so sure.

I'm in contact with Braze support about this, but haven't got anywhere yet. I'll update here when I do.

I'd be really interested to hear how your experiment goes.

AllanHeoFNM

Hey @Malcolm_Murray ! Thanks for the clarification and extra context. Knowing that you're working with an API-triggered Canvas, that makes testing a bit easier and faster as I won't have to wait for daily tests 🙂

Here's what I did for my test. I created an API-triggered Canvas with 5 eligible users, and I made the Canvas "Maximum users to potentially enter this Canvas = 1" in the Audience page. This means that each time my Canvas is triggered, it should get sent to 1 out of the 5 eligible users. What we're testing for is whether that 1 user will be random each time, or if it'll be the same user each time.

1st Canvas trigger: user 1
2nd Canvas trigger: user 2
3rd Canvas trigger: user 1
4th Canvas trigger: user 1

I'm limited to one trigger per hour (Braze limitation), so I haven't been able to collect as much data as I would've liked so far... I want to say it's random just because of the single occurrence of user 2, but user 1 is getting entered suspiciously often. I'll do some more tests and let you know, and also, please let me know if you hear from Braze!

AllanHeoFNM

Exciting update @Malcolm_Murray 🙌🏻

5th Canvas trigger: user 5

I'm more inclined to believe that user entry is random each time!

Malcolm_Murray

Braze support have just got back to me and confirm that user entry is indeed randomised, so this backs up your findings @AllanHeoFNM. The response from support on this was:

Thanks all for this, particularly @AllanHeoFNM for your tests to confirm this behaviour. Really useful.

AllanHeoFNM

Glad to hear it all! Happy to help!