One faulty relay, line of code or stuck ship has caused huge disruptions, from blackouts and lost spacecraft to frozen airlines and a blocked canal. Modern life depends on vast, interconnected systems: power grids, telephone networks, computer software, shipping routes and pipelines. Most of the time they work so smoothly that people barely notice them. But when a system has a single point of failure, one small fault can bring the whole thing down.
Engineers use the phrase single point of failure for any part of a system whose failure stops everything else. It might be a single component, a single line of code, a single password or a single narrow channel. Good design tries to eliminate such weak points through backups and redundancy, but they are often hidden until something goes wrong.
The ten events below are the Northeast blackout of 1965, the AT&T network collapse of 1990, the Ariane 5 explosion, the loss of the Mars Climate Orbiter, the Northeast blackout of 2003, the Knight Capital trading disaster, the blocking of the Suez Canal, the Colonial Pipeline shutdown, the Facebook outage of 2021 and the CrowdStrike outage of 2024.
This list focuses on failures of systems, software and logistics rather than collapses of buildings or bridges. In each case, official investigations or company reports identified the key weak point, and the descriptions here follow those findings.
The lessons from these events have shaped engineering and business practice. Many led to new rules, standards or design changes intended to prevent a repeat, though as the more recent examples show, new kinds of single points of failure keep appearing as technology changes.
Key Facts
| # | Event | The single weak point | Year |
|---|---|---|---|
| 1 | Northeast blackout of 1965 | A single protective relay at a power station in Ontario | 1965 |
| 2 | AT&T network collapse | A software flaw in long-distance switches | 1990 |
| 3 | Ariane 5 Flight 501 | Reused software that could not handle a large number | 1996 |
| 4 | Mars Climate Orbiter | A mismatch between imperial and metric units | 1999 |
| 5 | Northeast blackout of 2003 | An alarm system that failed without warning | 2003 |
| 6 | Knight Capital trading loss | A deployment error that revived obsolete code | 2012 |
| 7 | Suez Canal obstruction | One vessel blocking a narrow waterway | 2021 |
| 8 | Colonial Pipeline shutdown | A single account without extra security | 2021 |
| 9 | Facebook outage | A configuration command that cut off the network | 2021 |
| 10 | CrowdStrike outage | A single faulty update to security software | 2024 |
1. A Relay Set Too Low
On the evening of November 9, 1965, a protective relay on a transmission line at a hydroelectric station near Niagara Falls in Ontario tripped. It had been set to disconnect the line at a power level lower than the line was actually carrying, a setting that had not been updated as demand grew [1].
When the line disconnected, the power it was carrying surged onto other lines, which also tripped. Within minutes, the failure cascaded across the interconnected grid. About thirty million people in Ontario and the northeastern United States lost power, some for up to thirteen hours, including the whole of New York City [1].
The blackout led to major changes in how grids were managed. It contributed to the creation of the North American Electric Reliability Council in 1968, an organization set up to coordinate reliability standards across the continent.
2. A Bug That Spread Between Switches
On January 15, 1990, much of AT&T's long-distance telephone network in the United States stopped working for about nine hours. Tens of millions of calls failed to connect, affecting businesses, airlines and ordinary customers [2].
The cause was a flaw in a recent software update installed in the network's switching computers. When one switch reset itself after a minor problem, it sent messages to neighboring switches. The bug caused those switches to fail and reset as well, sending more messages and spreading the problem across the network in a chain reaction [2].
Related: Top 10 Engineering Disasters That Changed the Rules
Engineers eventually solved the problem by reducing the network's message load and reverting the software. The incident became a classic case study in how a small coding error can be amplified by the connections in a large network.
3. An Overflowing Number
On June 4, 1996, the first Ariane 5 rocket of the European Space Agency launched from French Guiana. About forty seconds after liftoff, it veered off course, broke apart and was destroyed, along with the scientific satellites it carried [3].
Investigators found that the rocket's guidance software had been reused from the smaller Ariane 4. The new rocket flew faster, and one value became too large to fit into the space allotted for it when the software converted it into a smaller number format. The resulting error shut down the guidance system. A backup system running the same software failed in the same way [3].
The failure cost hundreds of millions of dollars and became one of the most cited examples of software error. It showed that reusing proven code in a new context can be risky if its assumptions are not checked.
4. Mixed-Up Units
NASA's Mars Climate Orbiter was designed to study the Martian atmosphere and serve as a relay for other missions. On September 23, 1999, as it approached Mars to enter orbit, it passed far too close to the planet and was lost [4].
An investigation found that one piece of ground software, produced by a contractor, calculated thruster impulses in pound-force seconds, an imperial unit, while NASA's navigation software expected the values in newton-seconds, a metric unit. The mismatch meant small errors built up over the months of the flight [4].
The mission cost more than 300 million dollars. The investigation emphasized that the error itself was simple, but that the failure to catch it reflected broader problems in communication and testing. The lesson is still taught in engineering courses.
5. A Silent Alarm
On August 14, 2003, power lines in Ohio sagged into overgrown trees and shut down. Normally, control room operators would have been warned and taken action. But a software bug had caused the alarm system at the utility's control room to stop working, and operators did not know it [5].
Related: Top 10 Most Destructive Typhoons in Recorded History
Without alarms, the operators did not realize the scale of the problem until it was too late. Failures spread through the interconnected grid, and within hours about fifty-five million people in the northeastern and midwestern United States and Ontario had lost power. Some areas were without electricity for days [5].
A joint American and Canadian task force investigated. Its recommendations led to mandatory reliability standards for grid operators in North America, enforced by the North American Electric Reliability Corporation.
6. Old Code Brought Back to Life
On August 1, 2012, Knight Capital Group, a major American trading firm, deployed new software to its trading servers. One of the eight servers did not receive the new code. On that server, a reused setting activated old, dormant software that had not been used for years [6].
When the market opened, the faulty server began sending millions of unintended orders. In about forty-five minutes, the company lost around 440 million dollars, an amount that threatened its survival [6].
Knight Capital was rescued by investors and later merged with another firm. The Securities and Exchange Commission fined the company and cited weaknesses in its controls. The case became a warning about the dangers of untested deployments in automated trading.
7. A Ship Stuck in a Canal
On March 23, 2021, the huge container ship Ever Given ran aground in the Suez Canal in Egypt, during high winds and a sandstorm. The ship, about 400 meters long, became wedged diagonally across the canal, blocking it completely [7].
The Suez Canal is one of the world's most important shipping routes, connecting Europe and Asia. For six days, hundreds of ships waited at both ends, and others were rerouted around Africa. Estimates suggested billions of dollars of trade were held up each day [7].
Related: Top 10 Facts About the Power Grid Few People Know
Dredgers and tugboats freed the ship on March 29, helped by high tides. The incident drew global attention to how a single narrow chokepoint can disrupt global supply chains already strained by the pandemic.
8. One Compromised Password
In May 2021, a ransomware attack hit Colonial Pipeline, which operates the largest pipeline system for refined fuel in the United States, carrying fuel from Texas to the East Coast. The company shut down the pipeline as a precaution while it dealt with the attack [8].
The company's chief executive later told Congress that the attackers gained access through an old virtual private network account protected only by a password, without multifactor authentication. The password had apparently been exposed in an earlier leak [8].
The pipeline was shut for several days, causing fuel shortages and panic buying in parts of the southeastern United States. The company paid a ransom, part of which was later recovered by the United States Department of Justice. The attack led to new cybersecurity rules for pipelines.
9. A Network That Disconnected Itself
On October 4, 2021, Facebook, Instagram and WhatsApp disappeared from the internet for about six hours. Billions of users could not reach the services, and businesses that depended on them lost sales [9].
According to the company's own explanation, a command issued during routine maintenance unintentionally disconnected its global backbone network. Its systems then withdrew the routing information that tells the rest of the internet how to reach Facebook's servers [9].
Because the company's internal tools also depended on the same network, engineers had difficulty diagnosing and fixing the problem remotely and had to work on site at data centers. The outage showed how centralizing many services on one network can create a single point of failure.
Related: Top 10 Most Bizarre Phobias: Strange Fears That Exist
10. A Faulty Update
On July 19, 2024, the cybersecurity company CrowdStrike released a routine update to its Falcon security software for Windows computers. The update contained a defect that caused computers to crash and fail to restart properly [10].
Microsoft estimated that about 8.5 million Windows devices were affected. Airlines grounded flights, hospitals postponed procedures, banks and broadcasters experienced disruptions, and many businesses had to fix computers one by one [10].
The incident was described as one of the largest information technology outages in history. It raised questions about how much of the world's infrastructure depends on a small number of software providers, and about testing and rollout practices for updates.
Conclusion
These events show how small faults can have enormous effects when systems are tightly connected. A single relay, a line of code or a password can be the weak point that brings down systems serving millions of people.
They also show how hard it is to find these weak points in advance. Many were hidden until the moment of failure, and investigations often found that the technical fault was made worse by gaps in communication, testing or oversight.
Engineers continue to design systems with backups and safeguards. Yet as technology becomes more interconnected, new single points of failure are always emerging, and learning from past disasters remains one of the best defenses.
- ^ [1] Accounts of the 1965 blackout. Wikipedia: Northeast blackout of 1965 Wikipedia: NERC
- ^ [2] Accounts of the AT&T network collapse. Wikipedia: 4ESS switch Wikipedia: AT&T
- ^ [3] Accounts of the Ariane 5 failure. Wikipedia: Ariane flight V88 Wikipedia: integer overflow
- ^ [4] Accounts of the Mars Climate Orbiter. Wikipedia: Mars Climate Orbiter Wikipedia: newton-second
- ^ [5] Accounts of the 2003 blackout. Wikipedia: Northeast blackout of 2003 Wikipedia: race condition
- ^ [6] Accounts of Knight Capital. Wikipedia: Knight Capital Group Wikipedia: algorithmic trading
- ^ [7] Accounts of the Suez Canal obstruction. Wikipedia: 2021 Suez Canal obstruction Wikipedia: Suez Canal
- ^ [8] Accounts of the Colonial Pipeline attack. Wikipedia: Colonial Pipeline ransomware attack Wikipedia: ransomware
- ^ [9] Accounts of the 2021 Facebook outage. Wikipedia: 2021 Facebook outage Wikipedia: Border Gateway Protocol
- ^ [10] Accounts of the CrowdStrike outage and single points of failure. Wikipedia: 2024 CrowdStrike outages Wikipedia: single point of failure
Discussion
Log in with