Google Tag Manager

Forum Discussion

Siva's avatar
Siva
Supporter
3 months ago

Guidance on Limiting Audience per Experiment Path in Canvas

Hi everyone,

I'm looking for some guidance on a use case I'm currently working on. I want to implement a spinning wheel IAM using a single Canvas that includes 5 experiment paths.

The goal is to limit the number of users entering each path — for example:

  • Offer A (Path A): 150 users
  • Offer B (Path B): 100 users
  • And so on...

While the experiment path node in Canvas allows user distribution using percentage splits, I’ve found that it doesn't always behave as expected — especially with small total audience sizes (e.g. 10–20 users). During a live test with internal team members, the users were not evenly distributed across paths, despite using balanced percentage settings.

My questions:

  1. Is there a minimum user count required for the experiment path % splits to work accurately?
  2. Is there any official documentation explaining how this percentage-based distribution works under the hood?

As an alternative, I’m considering building 5 separate Canvases or campaigns, each with entrance caps to control how many users receive each offer. However, it would be ideal if Canvas supported audience caps per path directly.

Any tips, documentation, or suggestions would be greatly appreciated! 

Thanks in advance!

Siva

Administrative
Canvas
Inspiration
Segmentation

4 Replies

  • DenisK's avatar
    DenisK
    Supporter
    2 months ago

    Hi Siva,

    this is not confirmed by Braze or any documentation, but I'm pretty confident in why this is happening

    When you have an A/B test or an experiment path, Braze uses probability to sort users into variants. It essentially flips a coin for every user 

    The less coin flips (users) you have, the higher the margin for error is. 

    For example, if you flip a coin 10 times, you are very likely to get outcomes like 7 heads / 3 tails, 6/4... which is far from the expected 50/50 split.

    But if you flip a coin 10,000 times, you will get an outcome like 5,032/4,968 which is much closer to the 50/50 split.

    This is a calculation for different audience sizes:

    Users - Margin of error
    50 - 13.9%
    100 - 9.8%
    500 - 4.4%
    1000 - 3.1%
    10,000 - 1.0%

    • This is expected and happens in all tools that do any kind of A/B Testing
    • The more users you have in your canvas, the more even the split will be
    • The less variants you have, you'll get a more even split faster
    • Using multiple canvases will not help, you'll get the same problem

     

    In simple terms, a solution is to wait until you have at at least around 500 users per variant in your case.

    Best,
    Denis

  • HariSirigiri1's avatar
    HariSirigiri1
    Influencer
    2 months ago

    Hi Siva​ 

    To build on DenishK's point about Braze experiment paths:

    Braze uses random bucket numbers in the backend to distribute users across different experiment paths. Each user is assigned a random bucket number and the split percentages determine which range of bucket numbers corresponds to each path.

    For example:

    • Path A (20% split): Users with random bucket numbers 0-1999
    • Path B (30% split): Users with random bucket numbers 2000-4999
    • Path C (50% split): Users with random bucket numbers 5000-9999

    Even with 1000 users entering the canvas, you may not get exactly 200 users in Path A and 300 in Path B due to the randomness of bucket number assignment. The actual distribution will approximate the target percentages but may vary, especially with smaller audience sizes.

    The random bucket system ensures fair distribution over time and larger audiences, but expect some variance in exact split numbers, particularly in smaller test groups or individual canvas runs.