Meghan Markle Says Oprah Was Surprised by the Size of Nottingham Cottage: 'No One Would Ever Believe It!'

Alberto E. Rodriguez/Getty Images; Chris Jackson/Getty Images Oprah Winfrey; Meghan Markle and Prince Harry

Meghan Markle and Prince Harry's first home of Nottingham Cottage took Oprah Winfrey by surprise when she popped by for a visit.

"Oprah came over for tea, didn't she?" the Duke of Sussex, 38, asked his wife in episode 4 of their new Netflix docuseries Harry & Meghan, the second volume of which premiered on Thursday.

"She did," said Meghan, 41, to which Harry responded, "And when she came in, she sat down and she goes, 'No one would ever believe it!' "

" 'No one would ever believe it!' " the Duchess of Sussex echoed, laughing.

The couple had been discussing the size of Nottingham Cottage, the residence on the grounds of Kensington Palace in London where they first lived together following their 2017 engagement.

Netflix Prince Harry outside Nottingham Cottage

Can't get enough of PEOPLE's Royals coverage? Sign up for our free Royals newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!

"As far as people were concerned, we were living in a palace — and we were, in a cottage on palace grounds," Harry said.

Andrew Parsons/PA Images via Getty Nottingham Cottage

RELATED: Inside Prince Harry and Meghan Markle's Cozy 1,300-Square-Foot Cottage at Kensington Palace

"We were living on palace grounds, yeah," Meghan added. "Kensington Palace sounds very regal — of course, it does, it says 'palace' in the name, but Nottingham Cottage was a small [house]."

For more on the Royals, listen below to our daily podcast PEOPLE Every Day.

"The whole thing's on a slight lean. Really low ceilings — I don't know who it was ever for. They must've been very short," the prince joked.

Netflix Prince Harry in Nottingham Cottage

"He would just hit his head constantly in that place, 'cause he's so tall," Meghan said with a laugh. "Me with a hoe, and H varnishing. It was just a chapter in our lives where I don't think anyone could believe what it was actually like behind the scenes."

Meghan and Harry lived at Nottingham Cottage until the spring of 2019, when they relocated to Windsor Castle's Frogmore Cottage ahead of the birth of their first child: son Archie Harrison, now 3.

Additionally, the Duke and Duchess of Sussex — who are also parents to daughter Lilibet Diana, 18 months — reflected on their March 2021 sit-down with Oprah, 68.

Netflix Meghan Markle in Nottingham Cottage

Meghan and Harry revealed that they watched the interview together for the first time as it aired last year, and went on to address the reaction of the public afterward.

"I thought that me being very open about the depression that I experienced and just how extreme that became, I thought that would be the biggest takeaway," said Meghan.

RELATED VIDEO: Oprah Winfrey Just Revealed How Meghan Markle and Prince Harry's Royal Wedding Left Her Feeling

But to her surprise, "It was entirely eclipsed by the conversation surrounding race," she added — likely referring to the reaction to her comments to Oprah about how, within the royal family, there were "concerns and conversations about how dark [their child's] skin might be when he's born."

Prince William previously refuted the racism claims during an outing after the interview aired, saying, "We are very much not a racist family."

All episodes of Harry & Meghan are now streaming on Netflix.