A/B Testing Splitting Mechanics

Introduction to A/B Testing Splitting

A/B testing platforms need to reliably split users between different experiment variants to ensure valid statistical results.

The splitting mechanism must be:

Deterministic - Same user always gets same variant
Uniform - Even distribution across variants
Stable - Doesn't change when adding new variants
Fast - Low computational overhead

User splitting visualization

Hashing for User Assignment

Hashing converts a user identifier (like user_id) into a numerical value that can be used for variant assignment.

user_id: "user123" → hash("user123") → 0.742

if (hashValue < 0.5) → Variant A else → Variant B

Common hash functions:

MurmurHash
xxHash
FNV-1a

Hash Properties

Deterministic

Same input always produces same output

Uniform Distribution

Outputs are evenly spread across range

Fast Computation

Low CPU overhead for high traffic

Salting for Experiment Isolation

Salting ensures that different experiments don't assign the same users to variants in correlated ways.

hash("user123" + "experiment1") → 0.15

hash("user123" + "experiment2") → 0.82

Without salting, users might consistently land in the same variant across experiments, creating bias.

Select experiment:

Why Salting Matters

Experiment Independence

Ensures one experiment doesn't affect another's user distribution

Avoid Correlation

Prevents users from always being in the same variant across tests

Statistical Validity

Maintains proper randomization for accurate results

Bucket Allocation Strategies

After hashing, we need to map the hash value to specific variants with desired weights.

Variant A %

0% 50% 100%

Variant B %

0% 50% 100%

Variant A

0

Control

0

Variant B

0

Implementation Code

function assignVariant(userId, experimentId, weights) {
    // Combine user ID with experiment salt
    const salt = userId + experimentId;
    
    // Generate hash value between 0 and 1
    const hashValue = hashFunction(salt) / MAX_HASH_VALUE;
    
    // Determine variant based on weights
    let cumulativeWeight = 0;
    for (const [variant, weight] of Object.entries(weights)) {
        cumulativeWeight += weight;
        if (hashValue < cumulativeWeight) {
            return variant;
        }
    }
    
    // Fallback to first variant
    return Object.keys(weights)[0];
}

Advanced Splitting Considerations

Layered Experiments

When running multiple experiments that might interact, you need to ensure proper isolation between layers.

hash("user123" + "layer1" + "experimentA")

hash("user123" + "layer2" + "experimentB")

Sticky Bucketing

For some experiments, you might want to keep users in the same variant even if weights change.

Normal Assignment

Users may switch when weights change

Sticky Assignment

Users stay in original variant

Traffic Allocation

You may want to limit what percentage of users enter an experiment at all.

Experiment Traffic %

0% 100% 100%

Adjust slider to see traffic allocation

Summary of A/B Testing Splitting

Key Takeaways

Use consistent hashing for deterministic assignments
Always salt with experiment ID for proper isolation
Implement proper bucket allocation for weighted distributions
Consider traffic allocation and layered experiments for complex setups

Final Interactive Demo

See all the concepts working together in this complete simulation:

Experiment ID:

Control %

33%

Variant A %

33%

Variant B %

34%