The Fallacies of Distributed Computing & Time
Shattering the illusions of reliable networks, zero latency, and the physical impossibility of absolute time.
The Deployment Problem
When code runs on a single machine, the physical realities of the system are highly predictable. Memory access is effectively instantaneous, failures are usually binary (the process is either running or dead), and events are strictly ordered by a single CPU clock.
However, hardware has physical limits. To scale beyond a single machine’s CPU and RAM, or to survive a total datacenter outage, engineers must deploy their systems across multiple machines. This introduces the network as the connective tissue between state. The moment you introduce a network, the predictable physics of a single machine collapse.
The Eight Fallacies
In 1994, L. Peter Deutsch and other engineers at Sun Microsystems formulated the Eight Fallacies of Distributed Computing. These are false assumptions that programmers new to distributed applications invariably make, leading to catastrophic system designs:
- The network is reliable.
- Latency is zero.
- Bandwidth is infinite.
- The network is secure.
- Topology doesn’t change.
- There is one administrator.
- Transport cost is zero.
- The network is homogeneous.
If you architect a microservice to make a synchronous HTTP call to another service and simply wait for the response without configuring strict timeouts and retry budgets, you are falling victim to Fallacies #1 and #2.
The Physics of Time and Causality
The most difficult paradigm shift in distributed engineering is abandoning the concept of absolute physical time.
In a distributed system, there is no global clock. Every server has its own physical quartz crystal oscillator. Due to temperature variations and hardware imperfections, these clocks drift apart. While the Network Time Protocol (NTP) attempts to synchronize them via the internet, NTP can be delayed, drift by milliseconds (or seconds), or even artificially jump time backward to account for leap seconds.
created_at = GETUTCDATE()) to determine the exact order of events across different servers. If Node A processes an event and sends it to Node B, NTP drift could cause Node B’s physical clock to record the received event before Node A’s clock says it was sent. This violates causality.Logical Clocks and “Happens-Before”
In 1978, Leslie Lamport realized that distributed systems do not actually need to know when exactly something happened (physical time); they only need to know the strict sequence of events (causality).
He introduced the concept of the “happens-before” relationship (denoted by $\rightarrow$). If event $a$ causes event $b$, the system only needs a mathematical guarantee that the logical timestamp of $a$ is strictly less than $b$:
$$a \rightarrow b \implies L(a) < L(b)$$Instead of relying on quartz crystals, systems use Logical Clocks (like Lamport Clocks or Vector Clocks). These are simple integer counters that increment with every local event. When Node A sends a message to Node B, it attaches its current counter. Node B looks at the attached counter, compares it to its own, updates its counter to the highest value, and increments by one. This guarantees a mathematically correct sequence of events, completely immune to NTP drift.
Updates to MAX(5,2) + 1 = 6. NodeB->>NodeA: Reply (Attached: 7) Note over NodeA: Compares 2 vs 7.
Updates to MAX(2,7) + 1 = 8.
Test Your Understanding
Q:You are investigating a production incident. You pull the central logs from Service X (located in AWS US-East) and Service Y (located in AWS EU-West). You sort the combined logs by the exact millisecond timestamp. The logs show that Service Y finished processing a payment 40 milliseconds before Service X even received the 'Start Payment' HTTP request. How is this possible, and how should you fix your logging architecture? Reveal â–¾
This is a classic manifestation of NTP Clock Drift. The physical clock on the EU-West server was slightly ahead of the US-East server. Sorting distributed events by physical wall-clock time will consistently result in causal impossibilities.
To fix this, you must abandon physical time for ordering. Implement Correlation IDs (Logical Clocks). When the request enters the API Gateway, assign it a unique ID and a sequence counter. Pass this ID and counter in the HTTP headers to every downstream service, allowing you to reconstruct the exact causal graph of the request regardless of physical server time.