Prince Charles' Failed Marriage to Princess Diana 'Will Always Haunt Him,' Says Andrew Morton

Tim Graham Photo Library via Getty Prince Charles and Princess Diana

Twenty-five years after Princess Diana's tragic death, Prince Charles is still remembered for their failed marriage.

Queen Elizabeth's eldest son and heir was considered the world's most eligible bachelor when he wed 20-year-old Lady Diana Spencer in July 1981. However, there were soon struggles in the relationship, and they separated in 1992. Many blamed Prince Charles' affair with Camilla Parker Bowles — especially after the car accident that killed Diana and two others.

Prince Charles married Camilla, Duchess of Cornwall in 2005, and the Queen recently expressed her wish for Camilla to be known as Queen Consort when Charles takes the throne. But still, despite Prince Charles' many achievements in his royal life, Princess Diana's legacy lingers over the heir.

RELATED: Prince Charles and Princess Diana's Relationship Timeline

Anwar Hussein/Getty Princess Diana and Prince Charles

"His tragedy is whatever he does, whatever he says, however he behaves, he will be remembered for one thing: the fact that his fairy-tale marriage ended," Andrew Morton, Princess Diana's biographer, tells PEOPLE in this week's issue. "Just as Henry VIII is remembered for his six wives, Prince Charles is remembered for his first wife. It will always haunt him. His life has been defined by his marriage."

Adds royal biographer Ingrid Seward, "The ghost of Diana really stalked him at every turn."

Georges De Keerle/Getty Princess Diana and Prince Charles

Although Prince Charles, 73, does not publicly speak about Princess Diana, he did cooperate in an early-2000s investigation into her death. He was questioned by the U.K. police in 2005 as part of Operation Paget, the investigation into the various conspiracy theories surrounding Diana's death.

Former head of Scotland Yard John Stevens told the Daily Mail that he spoke to Prince Charles about a note that Princess Diana wrote in 1995 which said, "My husband is planning 'an accident' in my car, brake failure and serious head injury," so that he could marry Tiggy Legge-Bourke, a nanny for Prince William and Prince Harry.

"Yes, allegations had been made about the Prince of Wales and other royals but we had to find or examine the [existing] evidence before we approached him with formal questions," Stevens told the outlet. "We found no other evidence to support the scenario suggested in Diana's note."

PEOPLE presents The Story of Diana: Archival footage and conversations with those who knew the Princess of Wales best reveal how her story remains relevant today. Now on PeopleTV.