Swifties Lose It As Taylor Swift and Joe Alwyn Get Cuddly During Ed Sheeran's Jingle Bell Ball Set

After Taylor Swift‘s boyfriend Joe Alwyn cheered her on from the audience at iHeartRadio’s Jingle Ball in New York City, the couple hopped across the pond for another concert — where they made a rare display of PDA.

The singer, who turns 28 on Wednesday, found her beau in the crowd after taking the stage at Capital FM Jingle Bell Ball in London on Sunday night. The couple didn’t let the swarms of fans armed with cameras stop them from taking advantage of a romantic moment: slow-dancing together while Ed Sheeran crooned his songs onstage.

“THEY LITERALLY SLOW DANCED TO ALL EDS SONGS WITH THEIR ARMS AROUND ONE ANOTHER AND JUST KEPT LOOKING AT EACH OTHER AND KISSING,” wrote one enthusiastic concert goer on Twitter.

“TAYLOR AND JOE ARE SWAYING WITH HER HEAD IN HIS F—ING SHOULDER KISSING TO PERFECT I GOTTA HCKING LEAVE THE ARENA,” wrote one witness.

Swift and Alwyn, who secretly dated for months before going public with their relationship in May, held hands as they entered Madison Square Garden on Friday for the concert in New York.

“Taylor and Joe are spending time together in N.Y.C. Taylor was in the best mood. She was smiling and waving to fans,” a source tells PEOPLE. “Joe stayed for the whole show and they later left together as well. They are staying at Taylor’s N.Y.C. apartment.”

Alwyn was also with Swift for her pre-Jingle Ball private party and the duo hung with the singer’s pals Blake Lively and Ryan Reynolds, Jack Antonoff and Camila Cabello backstage.

Later, during the show, Swift performed her hits “Shake It Off,” “…Ready for It?,” “I Don’t Want to Live Forever” and “Look What You Made Me Do.” She then brought out Sheeran to perform their duet “End Game.”

An insider previously told PEOPLE that Swift’s family is hoping that her latest relationship will endure.

“It’s a great relationship that everyone hopes will last. Joe could definitely be the one,” the insider said “Their relationship is very different from Taylor’s past ones. Just like Taylor, Joe is a homebody. He loves being an actor but doesn’t like the celebrity part of it. It’s pretty special to find someone you’re on the same page with.”

Along with the release of her latest album, Reputation, Swift announced a global arena tour — which will likely take her away from her new British boyfriend. However, a source previously told PEOPLE that the love birds are determined to make it work.

“When Taylor tours internationally next year, they will figure it out,” the source said. “This is not anything they are stressing about. Their relationship is amazing, because they are both committed to making it work. It’s just a very normal and respectful relationship, built on give-and-take. Joe is a gentleman and just a great guy to date.”