Are Meghan Markle and Prince Harry Moving to Africa?

Meghan Markle and Prince Harry have just settled into their new country home of Frogmore Cottage in Windsor, but are they already planning on another big move?

The royal couple could relocate to a Commonwealth country in Africa when their baby is a little older to capitalize on their appeal to young people, according to reports.

David Manning — the former British ambassador to the U.S. and adviser to Harry and his brother Prince William — has proposed a plan for the couple to take on an international position that could build on their work for the Commonwealth,” reports the Sunday Times.

In a statement, Buckingham Palace did not deny the reports, saying, “Any future plans for The Duke and Duchess are speculative at this stage. No decisions have been taken about future roles.”

The couple both hold roles within the Queen’s Commonwealth Trust — Harry is President and Meghan is Vice-President.

“This is a 1000-year-old monarchy overseeing a Commonwealth of 54 territories. Lots of people depend on its existence and continued success,” a source tells PEOPLE. “Eventually Harry and Meghan will be dispatched somewhere.”

While the Sunday Times says their move to Africa would be for two to three years, ITV claims that it would be more like a few months. The plan is in very early stages and a decision is not expected until 2020, which gives Meghan and Harry time to settle into life as a family of three at Frogmore Cottage.

Claims that the Africa plan came about in order to put distance between the two couples has also been refuted. While there is tension between the brothers (they didn’t speak during Easter celebrations and arrived separately), the plan is likely more of a response to their different roles within the royal family.

“William and Kate are completely in transition as he prepares to take over as the Prince of Wales,” the source adds. “Their life together is all about preparing to be in charge, which is a significant reason for the wall between them and Harry and Meghan.”

A move to Africa would also be meaningful for the couple, who have a shared history there. They traveled to Botswana together early on in their relationship in the summer of 2016 and again to celebrate Meghan’s 36th birthday in 2017.

“I managed to persuade her to come and join me in Botswana. We camped out with each other under the stars. She came and joined me for five days out there, which was absolutely fantastic,” Harry said after their engagement. “So then we were really by ourselves, which was crucial to me to make sure that we had a chance to know each other.”

RELATED: Never-Before-Seen Footage of Meghan Markle’s Powerful Humanitarian Trip to India Released

They shared a never-before-seen photo of their 2017 trip on their new Instagram account, which showed them assisting with conversation efforts with elephants.

Africa holds a special place in Harry’s heart, and he has focused much of his conservation work there. He is president of African Parks and patron of the Rhino Conservation Botswana. It has long been a go-to place for Harry, who has visited the continent since his teens and refers to it as “his second home.”

• Can’t get enough of PEOPLE’s Royals coverage? Sign up for our newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!

Harry also started his charity, Sentebale, in Lesotho in 2006 to help vulnerable children and young people in the country.

Meghan has ties to Africa as well. In 2016, she became the Global Ambassador for World Vision Canada, traveling to Rwanda for the Clean Water Campaign.

“I’ve never wanted to be a lady who lunches – I’ve always wanted to be a woman who works,” she once wrote. “And this type of work is what feeds my soul, and fuels my purpose.”