Justin Bieber and Selena Gomez Attend Church Together Days After Packing on the PDA in Jamaica

Take them to church!

Just two days after Justin Bieber and Selena Gomez were spotted packing on the PDA in Jamaica, the two pop stars attended a church service in Los Angeles together.

The lovebirds were both dressed casually with Gomez, 25, opting for a patterned jacket and a black t-shirt, while the 32-year-old “Sorry” singer — who’s set to become a big brother for the third time — wore a grey sweatshirt and black pants.

“Yesterday was the most amazing day ever. I married my best friend! I love you!” Rebelo wrote on Instagram. Jeremy also posted a photo of their big day, simply captioning it, “Family.”

Sources told PEOPLE that prior to the ceremony, Bieber flew to Texas on Sunday to pick up Gomez so the two could share some quality time together as they traveled to their sun-filled destination together.

And on Wednesday, Bieber’s father revealed that his new bride is pregnant, which means the 23-year-old will have another little sibling to look out for.

The proud papa-to-be made the big announcement by sharing a shot on social media of a bikini-clad Rebelo, displaying her growing baby bump amid a picturesque natural landscape.

“My beautiful wife and….” he captioned the sweet photo.

This will be the second child for Rebelo, who is mom to daughter Allie, and the fourth for Bieber, who is father to Jaxon, 8, Jazmyn, 9, and, of course, the Canadian superstar.

Previous to their Jamaican getaway, Gomez and Bieber were last seen together celebrating Valentine’s Day at church, and later, at the Montage in Beverly Hills. “They celebrated a romantic Valentine’s Day,” a source told PEOPLE at the time. “They had been apart for a couple of days and seemed happy to be reunited.”

And while Bieber and Gomez may not officially be back on, the two are still seeing each other — they’re just keeping their dates low-key.

“Their relationship is very special. They have both changed, their lives are so different and it really seems things could work out great this time,” a Bieber source previously told PEOPLE. “They have both hugely matured. Justin is very supportive of Selena feeling her best.”

“Justin is still very happy. They have had quiet date nights at home and outings to local, low-key spots. They are still catching up and just enjoy hanging out,” another source previously told PEOPLE. “They have learned from past mistakes and are not putting any pressure on each other. Things are much different and more mature.”