The Spire Problem: Why Big Projects Fail and What It Means for Governance
Salisbury Cathedral’s spire rises 404 feet above the Wiltshire plain. It is the tallest in Britain, built in the fourteenth century without structural calculations or engineering blueprints. What medieval builders called divine vision, we can now recognize as optimism bias.
The spire weighs 6,500 tons. The cathedral beneath it was never designed to bear such weight. The four pillars at the crossing visibly bend inward under the strain. Iron bracing was added in the fourteenth century, and the structure has been monitored and reinforced continuously ever since. Seven centuries later, it still stands — not through the soundness of its foundations, but through perpetual intervention and maintenance.
This is what I will call the ‘spire problem’: visionary ambition that exceeds structural capacity, producing systems that survive only through continuous rescue. It is one of the defining patterns of megaprojects across countries, centuries, and industries, and, when viewed critically, it offers governance professionals a more honest frame for thinking about the systems they build, inherit, and maintain.
I. The Clinical Diagnosis
Bent Flyvbjerg has spent three decades building the world’s largest database of project performance. His findings are unambiguous: megaprojects fail at rates that would be unacceptable in any other domain of human activity.
The numbers are stark. Of the 16,000+ projects in Flyvbjerg’s database, spanning 136 countries and 20 fields, 91.5 percent go over budget, over schedule, or both. Only 8.5 percent hit cost and time targets, and only 0.5 percent — 1 in 200 — achieve their intended cost, schedule, and benefits. This is what Flyvbjerg calls the ‘Iron Law of Megaprojects’: “Over budget, over time, under benefits, over and over again.”
These risks are not normally distributed and do not have predictable ranges. Project outcomes follow fat-tailed distribution, meaning extreme overruns are far more common than standard risk models suggest. A 20 percent contingency buffer — standard practice in many organizations — is almost always inadequate when the distribution has no meaningful upper bound.
Flyvbjerg, 16,000+ projects across 136 countries. Sorted by severity.
| Project Type | Average Cost Overrun | Overrun % |
|---|---|---|
| Nuclear Storage | 238% | |
| Olympic Games | 157% | |
| Dams | 75% | |
| IT Projects | 73% | |
| Rail | 40% | |
| Tunnels & Bridges | 33% |
IT projects carry additional fat-tail risk: 18 percent of IT projects exceed 50 percent overrun, and those that do average 447 percent — an order of magnitude beyond the mean.
What causes this systematic failure? Flyvbjerg identifies two primary drivers at the root of all large-scale human endeavours: psychology and power.
Psychology operates through well-documented cognitive biases. Optimism bias leads planners to underestimate risks and overestimate their ability to manage them. The planning fallacy produces forecasts indistinguishable from best-case scenarios. WYSIATI — “What You See Is All There Is” —causes planners to build models based only on readily available information, ignoring what they cannot easily see. uniqueness bias convinces every project team that their situation is special, exempt from the base rates that govern similar endeavors.
Power mostly operates through what Flyvbjerg calls strategic misrepresentation: the deliberate distortion of forecasts to win approval or contracts. Former San Francisco mayor Willie Brown was candid about the mechanism: “In the world of civic projects, the first budget is really just a down payment. If people knew the real cost from the start, nothing would ever be approved.” This logic, accepted at face value for everyone with some experience in project management, is self-reinforcing. Underestimate costs to get approved, and once construction begins, the sunk cost fallacy and political pressure ensure continued funding. “Start digging a hole,” Brown advised, “and make it so big, there’s no alternative to coming up with the money to fill it in.”
The result is what Flyvbjerg terms the “survival of the unfittest”: because cost underestimation and benefit overestimation increase the likelihood of approval, the projects most likely to proceed are precisely those with the most deceptive forecasts. The system selects for delusion. This leads to Flyvbjerg’s central insight: projects don’t go wrong; they start wrong. The commitment fallacy — premature lock-in based on superficial planning — is the original sin. By the time construction reveals the true costs, the political and psychological forces of escalation make reversal nearly impossible. The hole is already too big, and there’s no way to fill it and start afresh.
The pattern repeats with numbing regularity:
Green shows the original estimate as a proportion of final cost. Red shows the overrun beyond estimate.
Berlin Brandenburg Airport, Germany’s showcase infrastructure project, opened nine years late — proof that even the nation famous for engineering precision cannot escape the Iron Law. The Montreal Olympics mayor declared they “can no more have a deficit than a man can have a baby.” Each project was presented as unique; each fell into the same traps that have plagued ambitious construction since the pharaohs.
II. The Anatomy of Ambition
William Golding’s novel The Spire anatomizes the psychology of overreach and boundless ambition with surgical precision. Dean Jocelin has received what he believes is a divine vision commanding him to build a 400-foot spire atop his cathedral. The master builder, Roger Mason, discovers the truth almost immediately: the cathedral has no real foundations. It rests on a raft of brushwood over mud. The existing structure was never designed to bear additional weight.
What follows is a study in visionary pathology.
Jocelin dismisses every warning. When Roger Mason reports that the earth is “creeping” and the stones are “singing” under strain, Jocelin responds: “They’ve sung and stopped before.” When Mason pleads that he has “seen a building fall,” Jocelin accuses him of cowardice: “You’re a man for a very little dare.” When Mason reports that the four pillars are visibly bending inward, Jocelin acknowledges the observation but refuses its implication: “Bending.”
The dean’s theology makes structural reality irrelevant. “I told you the building was a miracle and you would not believe me,” he says. “God will provide.” Jocelin’s will is the central force of the narrative — almost a character in itself: “My will has other business than to help… I have so much will, it puts all other business by.”
Mason knows the project cannot safely succeed, yet finds himself unable to resist, walk away, or be heard. When he tries to seek employment elsewhere, Jocelin blocks his exit: “I wrote to Malmesbury, Roger. To the abbot. I knew what was in his mind. I let it be known how long we shall need you here. He will look elsewhere.” The expert is captured, his professional judgment weaponized against him.
The pivotal confrontation comes when Mason, desperate, begs Jocelin for release: “Father, Father — for the love of God, let me go!” Jocelin is silent. Mason tries again: “Make me go!” And Jocelin hears, internally, only one sound: “Click.” The trap has closed. The master builder who understands that the structure cannot stand is now compelled to build it anyway. His expertise becomes not a shield against folly but an instrument of it.
The human cost mounts. A workman falls to his death; Jocelin does not wonder “that no miracle interposed between the body and the logical slab of stone.” The sacristan Pangall is murdered by the workers, his body hidden in the foundations. Goody Pangall, whom Jocelin used as an incentive to keep Mason working, dies in childbirth. Mason himself is ultimately destroyed by the project — “blind and dumb,” his wife reports, “like a baby.”
Near the end, also dying, Jocelin receives the cruelest revelation. His aunt, the king’s former mistress, explains how he became dean of this great cathedral. It was not divine selection but court patronage — a “plum” dropped in his mouth on a whim while she and the king lay together on a daybed. His entire ecclesiastical career, his sense of being chosen for a sacred mission, rests on pillow talk and the crude exercise of power. The “Holy Nail” that he drove into the spire’s apex? His aunt laughs at the idea that it might be genuine. Even the theological counselor who hears his dying confession observes that his mystical vision was merely the lowest level of spiritual experience — “where we are given an encouragement, a feeling, an emotion. Just so, you might give a child a spoonful of honey.”
Jocelin finally grasps what he has done: “I thought I was doing a great work; and all I was doing was bringing ruin and breeding hate.” And then, the devastating accounting: “I traded a stone hammer for four people.”
The spire, against all structural logic, does not fall. It stands, but damaged — “impacted into the parapet at the top of the tower,” leaning, threatening the cloisters. It becomes exactly what Jocelin feared: “Jocelin’s Folly.” A monument to visionary ambition that will require perpetual rescue.
Golding refuses an easy resolution. The spire exists. It may even be beautiful. But it was built on lies, coercion, and sacrifice. It may fall at any time. The foundation, literally and metaphorically, is rotten.
This is the pattern Flyvbjerg documents across centuries. The Sydney Opera House, that “magnificent doodle,” was begun without engineering testing, driven by a premier who understood Willie Brown’s logic avant la lettre: start digging, and the money will follow. It came in 1,400 percent over budget. California’s high-speed rail went from $33 billion to over $100 billion—a “bullet train to nowhere” that may never reach its intended destinations. The Channel Tunnel ran 80 percent over construction costs and still carries less than half the projected traffic after decades of operation.
Each of these projects had its Jocelin: a visionary leader whose conviction overrode evidence, whose will substituted for foundations. And each had its Roger Mason: experts who knew better but found themselves captured, complicit, unable to stop what they could clearly see.
III. Completeness Without Hierarchy
Even though psychology and power surely contribute, as they do in all things, there might be a deeper problem than either of them. Even with perfect forecasts and honest brokers, we face a more fundamental cognitive obstacle: the systems we build increasingly exceed our capacity to comprehend them.
Jorge Luis Borges understood this better than any management theorist. In “The Library of Babel,” he imagines the universe as an infinite library of hexagonal galleries, containing every possible book composed of 25 orthographic symbols. The Library is total — it contains every truth, every lie, every possible history, every refutation of every history. “When it was announced that the Library contained all books,” Borges writes, “the first reaction was unbounded joy. All men felt themselves the possessors of an intact and secret treasure. There was no personal problem, no world problem, whose eloquent solution did not exist — somewhere in some hexagon.”
The tragedy follows immediately: “The certainty that some bookshelf in some hexagon contained precious books, yet that those precious books were forever out of reach, was almost unbearable.”
This is completeness without hierarchy — the condition where everything is documented, but nothing can be found. The Library contains every answer, but provides no way to distinguish answers from noise, truth from fabrication. “For every rational line or forthright statement,” Borges notes, “there are leagues of senseless cacophony, verbal nonsense, and incoherency.”
The parallel to modern governance is striking. Enterprise architectures that document everything but help no one navigate anything. Compliance frameworks so comprehensive that they become un-navigable. Requirements specifications that capture all cases, but in their completeness, obscure the essential ones. The 1:1 map that coincides point-for-point with the territory — and is therefore useless (see: map vs. territory).
In another parable, Borges describes exactly this fate: an empire whose cartographers created maps of increasing detail until they produced “a Map of the Empire whose size was that of the Empire, and which coincided point for point with it.” Subsequent generations, “who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast Map was Useless, and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and Winters.” In the deserts, Borges concludes, “there are Tattered Ruins of that Map, inhabited by Animals and Beggars.”
Every governance framework that tends toward overcompleteness faces this fate. The aspiration to completeness — to capture every requirement, every exception, every edge case —produces documentation that is simultaneously total and useless. The map becomes as complex as the territory. It provides no abstraction, no guidance, no value.
Borges’s story “Funes, His Memory” reveals why this is inevitable. After an accident, Ireneo Funes gains perfect recall. He remembers every leaf on every tree, every moment of every day, every sensation in perfect detail. The gift destroys him. “To think is to ignore differences, to generalize, to abstract,” Borges writes. “In the teeming world of Ireneo Funes there was nothing but particulars.”
This is the paradox that professionals working in complex environments must confront: comprehensive documentation prevents thought. Abstraction — the willingness to forget, to simplify, to leave things out — is not a failure of rigor but a prerequisite for understanding. The model’s value lies precisely in what it omits.
There is another Borges story that illuminates why project planning so consistently fails. “The Garden of Forking Paths” describes a novel that is also a labyrinth—one in which, at every decision point, the character chooses all possibilities simultaneously. “In the work of Ts’ui Pên,” Borges writes, “the character chooses—simultaneously—all of them. He creates, thereby, several futures, several times, which themselves proliferate and fork.”
This is exactly the reality that Gantt charts deny. A megaproject is not a single path through time but an infinite garden of forking paths. Every procurement decision, staffing choice, technical architecture, and regulatory interpretation creates branches. Traditional project management assumes a single, linear future that cuts through the noise, while reality is, in effect, a “growing, dizzying web of divergent, convergent, and parallel times.” The project plan captures one path; the project itself walks many, if not all, of them.
IV. Orientation, Not Mastery
If the Library of Babel represents the failure mode — completeness that produces incomprehension — Susanna Clarke’s Piranesi offers an alternative.
The novel’s protagonist, named Piranesi, inhabits an infinite House: endless marble halls filled with statues, tides rising through lower vestibules, clouds drifting through upper galleries. The House extends beyond any horizon, and Piranesi has traveled hundreds of halls in every direction. During his journeys, he has found only “the regular progression of Halls and Passageways into the Far Distance.” He cannot comprehend the totality of the House. He does not try. Instead, he catalogues.
“I have begun a Catalogue,” Piranesi writes, “in which I intend to record the Position, Size and Subject of each Statue, and any other points of interest. The enormity of this task sometimes makes me feel a little dizzy, but as a scientist and an explorer I have a duty to bear witness to the Splendours of the World.”
He maintains a Table of Tides, meticulously documenting their patterns. He keeps journals with alphabetical indices so he can retrieve what he has learned. He names the years by significant events. He does not claim to understand the House; he orients himself within it.
Another character, Ketterley, takes the opposite approach. He believes the House contains “a Great and Secret Knowledge” that will grant enormous powers—immortality, telepathy, dominion over lesser intellects. He sees the House as “endless dreary rooms all the same, full of decaying figures covered with bird shit.” He seeks to extract the House’s value, to master it, to make it serve his purposes.
Ketterley dies in the House. Piranesi thrives. The difference is not competence but posture. Ketterley treats the House as a puzzle to be solved, a resource to be exploited. Piranesi treats it as something valuable in itself: “The House is valuable because it is the House. It is enough in and of Itself. It is not the means to an end.”
Clarke makes clear what the extractive mindset misses. Ketterley cannot perceive the House’s life: “How can a man as intelligent as him say there is nothing alive in the House? The Lower Halls are full of sea creatures and vegetation… The Tides themselves are full of movement and power.” The seeker of secret knowledge is blind to the obvious abundance before him. He searches for hidden power while ignoring the sustenance the system openly provides.
This reframe has practical consequences. When Piranesi confronts the limits of his understanding, he does not despair or grasp for illusory control. He trusts: “Do you trust the House? I ask Myself. Yes, I answer Myself. And if the House has made you forget, then it has done so for good reason… You are the Beloved Child of the House. Be comforted.”
At the novel’s end, returned to our world, Piranesi walks through a snowy city and sees statues he recognizes in the faces of strangers. The traffic noise sounds like tides. “The Beauty of the House is immeasurable,” he thinks. “Its Kindness infinite.” He carries the House with him—not as mastered knowledge but as orientation, as a way of seeing.
V. What This Means for Practitioners
The spire problem will not be solved. Visionary ambition will continue to exceed structural capacity. Systems will continue to grow beyond comprehension. The Iron Law will hold. But practitioners can work differently within these constraints.
First, take the outside view. Flyvbjerg’s most actionable prescription is reference class forecasting: instead of treating your project as unique, treat it as “one of those”— a member of a class whose outcomes are already known. The Sydney Opera House was not a singular act of genius; it was a performing arts venue with exotic design, and such projects are overrun by an average of 157 percent. Your project is special, but unless you are building a time machine, it is not unique.
Second, think slow, act fast, especially at the start. Planning is cheap; delivery is expensive and dangerous. The Empire State Building was “finished entirely on paper” before construction began — and came in under budget, weeks early. Thorough planning, especially during the intake and project engagement phases, enables rapid execution; the reverse produces the break-fix cycle that traps projects in perpetual overrun.
Third, design for the reality that ambition distorts risk assessment. Build in structural redundancy and plan for perpetual maintenance. Question whether your spire needs to be the tallest — or just tall enough to serve its purpose.
Fourth, resist the completeness trap. Your project or validation documentation does not need to capture everything. It needs to provide useful abstraction — a map smaller than the territory, a model that enables thought by deliberately leaving things out. The 1:1 map is neither rigorous nor useful. GAMP 5 and similar frameworks have long recognized this: validation documentation should be risk-based, scaled to the system’s impact, not exhaustive for its own sake.
Fifth, reframe your relationship to incomprehensible systems. You will inherit cathedrals built on inadequate foundations. You will maintain architectures whose full extent you cannot survey. The question is not whether you can master these systems — you cannot — but whether you can orient yourself within them.
This is Piranesi’s wisdom: documentation that orients rather than comprehends, that maps a navigable corner rather than aspiring to totality. He catalogues statues not to achieve completeness but to find his bearings — to know where he is within something immeasurable. He stewards rather than masters, inhabits rather than replicates. The practitioners who thrive in complex systems are not those who pretend to a comprehensive understanding, nor those who build 1:1 maps of infinite territories. They are the ones who, like Piranesi, document enough to navigate without claiming to know more than they do.
The spire stands. It leans, it threatens, it requires continuous rescue — but it stands. Seven centuries of interventions have kept it upright. Seven centuries of people who did not build it, who inherited its structural problems, who could not undo the original commitment, have nonetheless tended their corner of it with sufficient care to prevent collapse. This is perhaps the more remarkable achievement — not the building of the spire, but the custodianship that followed.
In the end, that may be the only relationship we can have with the systems we build and inherit: not comprehension, but a modest measure of orientation. Not the fantasy of control, but the discipline of careful attention to the small corner we’ve been given to tend. We are all, in the end, living in houses we did not build, maintaining spires we did not plan, navigating libraries whose catalogs we cannot trust.
The Beauty of the House is immeasurable. Its Kindness infinite.