How much of a role does luck play in our success or failure?

Theresa May has had a run of bad luck.

At a moment of acute political vulnerability, she was hoping to rally her supporters behind a rousing speech at last week’s Conservative Party conference in Manchester. Instead, she was besieged by a humiliating prankster and a nagging cough that reduced her voice to a faltering whisper. Adding insult to injury, towards the end of her speech, the letter ‘F’ mysteriously tumbled to the floor from the party’s slogan written in bold letters on the sky-blue panel behind her, leaving viewers to read ‘BUILDING A COUNTRY THAT WORKS OR EVERYONE’.

After what was widely seen as her extremely lucky ascent to power last year - following the unexpected referendum result, and her leadership opponents variously stabbing each other in the back and shooting themselves in the foot - it is perhaps ironic that Wednesday's series of unlucky events may seal her fate.

To be sure, many of Mrs May’s political troubles stem not simply from bad luck but from questionable decisions on her part, not least her poor campaign during the snap election she called, which cost the Conservatives their majority in June. But her unfortunate experience on Wednesday illustrates an interesting human tendency to underestimate the role that pure chance plays in success.

As the American essayist E.B. White wrote in the 1930s, “Luck is not something you can mention in the presence of self-made men.” Some people are of course suitably humble about their achievements. But White was surely correct, that most of us are far more likely to overestimate our responsibility for whatever successes we achieve. That’s especially true now, in our even more individualistic age, obsessed with the power of personal agency.

I am a lucky man. Ten years ago, I suffered an episode of sudden cardiac death on the tennis court in Ithaca, New York, where I teach economics at Cornell University. By happenstance, just before I collapsed, ambulances had been dispatched to two separate auto accidents nearby. Since one of them involved no serious injuries, an ambulance was able to peel off and travel just a few hundred yards to rush me to our local hospital. More than 90 percent of people who experience such episodes don’t survive. If that ambulance hadn’t happened to have been nearby, I would have remained dead.

Having cheated death does not make me an authority on luck. But it has motivated me to learn much more about the subject than I otherwise would have. In the process, I have discovered that chance plays a far larger role in life outcomes than most people realise, or would like to think. The luckiest among us appear especially unlikely to appreciate our good fortune. According to the Pew Research Center, people in higher income brackets are much more likely than those with lower incomes to say that individuals get rich primarily because they work hard.

Other surveys bear this out: wealthy people overwhelmingly view their success as having been all but inevitable. In their attempts to construct narratives to explain it, they search their memory banks for details that are consistent with successful outcomes. And because the overwhelming majority of successful people are in fact extremely talented and hardworking, they’ll find many ready examples of the long hours they logged, the many difficult problems they solved, and the many formidable opponents they vanquished.

But as the psychologist Tom Gilovich has shown, they’re much less likely to remember external events that may have helped them along the way — the teacher who once steered them out of trouble, perhaps, or the early promotion received only because a slightly more qualified colleague had to care for an ailing parent.

This asymmetry, Gilovich points out, resembles the one with which people react to headwinds and tailwinds. When you’re running or bicycling into a strong headwind, for example, you’re keenly aware of the handicap you face. And when your course shifts, putting the wind at your back, you feel a momentary sense of relief. But that feeling fades almost immediately, leaving you completely unmindful of the tailwind’s assistance. Gilovich’s collaborations with the psychologist Shai Davidai demonstrate the pervasiveness of analogous asymmetries in memory. People are far more cognizant of the forces that impede their progress than of those that boost them along.

That we tend to overestimate our own responsibility for our successes is not to say that we shouldn’t take pride in them. Pride is a powerful motivator; moreover, a tendency to overlook luck’s importance may be perversely adaptive, as it may encourage us to persevere in the face of obstacles.

Being born in a favourable environment is one of the few dimensions of luck we can control — that is, at least we can decide how lucky our children will be. But that requires extensive investment in the future. So even as a shrinking group among us has been growing steadily luckier, a growing number of the unluckiest have been falling still further behind. The good news is that we can easily do better. For it turns out that when successful people reflect on how chance events affected their paths to the top, they become much more inclined to pay forward for the next generation.

But simply telling successful people that they’ve been lucky doesn’t always help. On the contrary, it often has precisely the opposite effect, making them angry and defensive. It’s as if you’ve told them that they don’t really deserve to be on top, that they aren’t who they think they are.

Mysteriously, however, a seemingly equivalent rhetorical move often has precisely the desired effect: if you ask your successful friends whether they can think of any lucky breaks they might have enjoyed, you’ll almost invariably discover that they seem to enjoy trying to recall examples. You’ll see, too, that their eyes light up as they describe each one they remember. These memories cause them to experience the emotion of gratitude. And as countless studies have demonstrated, people who feel grateful are much more willing to incur costs to promote the common good.

In an unexpected twist, we may even find that recognising our luck actually makes us more likely to be lucky in the future. Social scientists have been studying gratitude intensively for almost two decades, and have found that it can produce a remarkable array of physical, psychological, and social changes: less frequent and less severe aches and pains, improved sleep quality, greater happiness and alertness. Those in whom gratitude is regularly stoked describe themselves as more outgoing and compassionate, and less likely to feel lonely and isolated.

Many of us seem uncomfortable with the possibility that personal success might depend to any significant extent on chance. But as the whirligig of politics accelerates over the next few days, Mrs May should take solace in the knowledge that she has not been the sole architect of her own fate.

Success and Luck: Good Fortune and the Myth of Meritocracy by Robert H. Frank is published by Princeton (£21.95). To order your copy for £16.99 plus p&p call 0844 871 1514 or visit books.telegraph.co.uk