The Load Balancer's Gambit: A Probabilistic Illusion

The Puzzle (The Classic)

Originally based on the American television game show Let’s Make a Deal, this probability puzzle—known as the Monty Hall Problem—famously baffled thousands of readers and even PhD mathematicians when it was popularized in 1990.

You are a contestant facing three closed doors. Behind one door is a brand-new car; behind the other two are goats. You randomly choose Door 1.

The host, Monty Hall, who knows exactly what is behind every door, does not open Door 1. Instead, he opens Door 3, revealing a goat. He then turns to you and asks: “Do you want to keep Door 1, or do you want to switch your choice to Door 2?”

Is it to your mathematical advantage to switch your choice?

The Redefinition (The Hot Cache Scenario)

Let’s transpose this into a practical routing dilemma for an edge computing cluster.

You are designing an algorithm for a client load balancer. It needs to query a specific piece of critical data from a cluster of three edge servers ($S_1, S_2, S_3$). Due to a recent network partition, the “hot cache” containing this data only exists on one server. The other two servers have “cold” (empty) caches.

Your load balancer randomly routes the initial request to $S_1$.

Before the request resolves, the network’s master controller—which possesses global state awareness and knows exactly where the hot cache is—intervenes to perform emergency maintenance. It forcefully takes $S_3$ offline, noting that $S_3$ had a cold cache anyway.

The master controller then sends a prompt to your load balancer: “S_3 is offline. You can maintain your connection to S_1, or you can switch your request to S_2.”

Should your load balancer’s algorithm be hardcoded to stay, switch, or does it not matter?

Formalization

Let the location of the hot cache be a random variable $C \in \{1, 2, 3\}$ where $P(C=i) = 1/3$. Let the load balancer’s initial choice be $X = 1$. Let the controller’s action of taking a cold server offline be $M \in \{2, 3\}$. The controller’s logic dictates:

• If $C=1$, the controller picks $M=2$ or $M=3$ with equal probability ($1/2$).
• If $C=2$, the controller MUST pick $M=3$ ($P(M=3 \mid C=2) = 1$).
• If $C=3$, the controller MUST pick $M=2$ ($P(M=2 \mid C=3) = 1$).

Assume the controller takes $S_3$ offline ($M=3$). You must calculate the conditional probabilities $P(C=1 \mid M=3)$ and $P(C=2 \mid M=3)$.

import random

def simulate_load_balancer(iterations=10000):
    stay_wins = 0
    switch_wins = 0

    for _ in range(iterations):
        servers = [1, 2, 3]
        hot_cache = random.choice(servers)
        initial_choice = random.choice(servers)
        
        # The controller filters out a cold server that is NOT the initial choice
        available_for_offline = [s for s in servers if s != hot_cache and s != initial_choice]
        taken_offline = random.choice(available_for_offline)
        
        # The remaining server to switch to
        switch_choice = [s for s in servers if s != initial_choice and s != taken_offline][0]
        
        if initial_choice == hot_cache:
            stay_wins += 1
        if switch_choice == hot_cache:
            switch_wins += 1
            
    print(f"Win rate if algorithm STAYED: {stay_wins / iterations:.1%}")
    print(f"Win rate if algorithm SWITCHED: {switch_wins / iterations:.1%}")

simulate_load_balancer()
# Output converges precisely to:
# Win rate if algorithm STAYED: 33.3%
# Win rate if algorithm SWITCHED: 66.7%