Palace Spokesman: Prince William and Prince Charles Did Not Snub Donald Trump

Was President Donald Trump given the royal snub?

After Queen Elizabeth met with the American president on Friday, a source told the Sunday Times that Prince Charles and Prince William, who are first and second in line to the throne, “refused to attend.”

However, a palace spokesperson tells PEOPLE that Charles and William were never supposed to take part in the meeting.

“In line with standard practice for guest of government visits the President was received by The Queen as Head of State. The arrangements were mutually agreed between the Palace and the U.K. and U.S. governments,” the spokesperson says. “There was never a suggestion that other members of the royal family would take part.”

Adds Dickie Arbiter, former spokesman for the Queen and author of On Duty with the Queen, “It was always going to be a short visit and as soon as you have too many people it gets longer. It was a courtesy call – that’s all it was.”

After more than a year of back-and-forth about whether he would be heading to the U.K. for an official visit, President Trump — along with the first lady — met the Queen at Windsor Castle on Friday.

The Trumps were welcomed at the Quadrangle of the castle, where a Guard of Honor gave a royal salute and the U.S. national anthem was played. The Queen and President Trump inspected the Guard of Honor before watching the military march past. The president and first lady then joined the Queen inside the castle for afternoon tea.

But the meeting didn’t exactly go off without a hitch. Tens of thousands of protesters marched through the streets of London — about 20 miles from Windsor Castle — it what was dubbed “The Stop Trump March.”

Carrying signs that read “Trump Not Welcome” and “Dump Trump,” the crowd made it clear that not everyone in the U.K. supported the country’s diplomacy.

In addition, Trump was slammed online for awkwardly blocking the Queen during their walk while inspecting her Guard of Honor. In a viral clip, the monarch appears to make a hand gesture as they turn the corner. She then attempts to pass behind the president, who stops walking and completely eclipses Queen Elizabeth from view for a moment. She returns to his right side and continues their walk.

Grant Harrold, a former butler for the royal family, told ITV News that letting the Queen lead in every scenario is a top rule in royal protocol.

William, Kate Middleton and Prince Harry hosted former President Barack Obama and former First Lady Michelle Obama for a private dinner in 2016, with Harry warmly greeting the first lady with a double kiss.

Memorably, Prince George stayed up late to greet the commander-in-chief clad in a bathrobe, matching gingham blue pajamas and slippers before taking a ride a rocking horse – a gift from the Obamas when he was born.

The special evening – which followed lunch Buckingham Palace with the Queen – was arranged by William on behalf of all three royals, and “they were delighted that [the First Couple] were able to accept,” a palace source told PEOPLE.

“The three of them have built up a relationship with the President and First Lady over the years,” added the source.

In 2014, William visited the Oval Office, where he and President Obama discussed the campaign to clamp down on illegal wildlife trafficking.

• Can’t get enough of PEOPLE’s Royals coverage? Sign up for our newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!

Harry also hosted Michelle and daughters Malia and Sasha – along with grandma Marian Robinson – for tea at Kensington Palace back in June 2015. In October 2015, the Obamas played host to a visiting Prince Harry – where the chemistry between Michelle and the royal was on adorable display.

The former president and his former Vice President Joe Biden also showed support for Harry’s Invictus Games in September. President Trump did not attend the event in Toronto, but First Lady Melania Trump did. She met with Harry on Sept. 23 in Toronto, where he thanked her for attending the games.