Nick Carter Admits Juggling Touring and Parenting Is 'Tough' After Missing Daughter's Birthday

nick carter/instagram

Nick Carter is getting candid about the challenges of being a working dad.

The Backstreet Boys singer, 42, is currently on the European leg of their DNA World Tour and celebrating the holidays a little early with the release of the band's new album, A Very Backstreet Christmas.

However, because of his busy work schedule, there was one important day he was forced to miss out on: daughter Saoirse's 3rd birthday earlier this month.

"It's definitely tough," Carter tells PEOPLE of juggling dad life with his tour schedule while currently in Europe with fellow boybanders Howie Dorough, AJ McLean, Brian Littrell and Kevin Richardson.

"I have the youngest out of all of the [Backstreet Boys] kids," he says, referring to daughter Pearl, 18 months (he is also dad to son Odin, 6, all of whom he shares with wife Lauren Kitt, 39).

"And right before we left to go on the road," he adds, "I totally missed my middle daughter's 3rd birthday."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

RELATED: Nick Carter Says He's 'Figuring Out' Being a Father of Three: 'I Love the Challenge'

Though he says it was "sad" to have not been there for Saoirse's big day, he is able to relate to his band members, all of whom are fathers.

"Some of the other guys have actually gone through it, where they haven't been able to be home because the schedules just pretty much conflict. So I experienced that for the first time," he explains. "But again, FaceTime helps, talking to them all the time, touching base with them. And, for us, it's our life. It's what we've done our whole life in regards to traveling. So they're kind of adjusting to it."

Though the five BSB members have been busy on the road, McLean tells PEOPLE that they'll have their families join them when their schedules align.

"We try to have a healthy balance as best we can. If there's a chance for our families to come join us on tour, they will," he says. "There's definitely been moments where all of our families have been out on tour at the same time, and it's just one big, giant potluck, family-picnic vibe."

McLean is dad to 9-year-old daughter Elliott (who recently changed her name from Ava) and Lyric, 5. Richardson and Dorough each have two children, while Littrell is father to 19-year-old Baylee — who is a singer and performer just like Dad.

RELATED VIDEO: AJ McLean's Wife Explains Why Daughter, 9, Changed Her Name to Elliott: 'Not a Gender Thing'

The best part, McLean says, is that "everyone's kids all get along," adding, "it's just one big happy family."

"It's a little bit easier when we're on the U.S. run because each of us has our own tour bus, so we can have our family out for as long as we want," he explains. "Kevin had his family out pretty much the entire summer tour. My family came out for two weeks. Europe's a little bit more of a challenge. It's further distance, as well as most of our kids are all in school now."

McLean credits the BSB wives for their help and support while the pop stars are out on the road performing for their fans.

"Mommys are definitely the hardest working in all of our families because they have to hold down the fort while we're out here singing and dancing," he says. "And they're out there holding down."

A Very Backstreet Christmas is currently available to purchase or stream.