Uncle Milo Ventimiglia Loves to 'Spoil' His Nieces and Nephews: 'They Are Constantly Inspiring'

Emma McIntyre/Getty Milo Ventimiglia

Milo Ventimiglia got plenty of parenting practice as patriarch Jack Pearson on This Is Us — but offscreen, the actor puts his skills to good use.

"I've got two nieces and two nephews," the 45-year-old actor tells PEOPLE in this week's issue. "It's great. I just spoil them and have fun with them and see them grow up. It's really fun to watch my sisters as parents too. They are constantly inspiring."

In his newest role, as a con man on the action series The Company You Keep, airing Sundays on ABC, Ventimiglia's days are decidedly action-packed.

"I had a couple injuries in the months we've been filming," he admits. "I ran into a door with my face and split my eye open... and I kneed a stone pillar and got an impact fracture on my femur. So yeah, different than holding babies and changing diapers!"

ABC Milo Ventimiglia on The Company You Keep

RELATED: This Is Us's Mandy Moore Celebrates 'Best TV Husband and Dad' Milo Ventimiglia on His 45th Birthday

As for moving on from his This Is Us role, for which Ventimiglia scored three Emmy nominations, the actor says it has been bittersweet.

"I'm grateful for the experience of playing Jack," he says of his character, whom he portrayed for six seasons on the NBC hit. "I think he gave an ideal to people, an attainable superhero. I definitely miss aspects of the character and of the show, like working with Mandy [Moore]."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

"But like any job in the acting profession, you have to let it go," he continues. "You perform, you put your heart into it and then you have to be okay stepping away from it."

Maarten de Boer/NBCUniversal This Is Us cast

RELATED: Where to See the Cast of This Is Us Next

That's not to say Ventimiglia would step away from Hollywood anytime soon.

"In some aspect, I think I'll be in front of the camera or adjacent to it for the rest of my life," he shares.

Ventimiglia adds: "That said, connecting on that mother Earth level is also really important to me. It reengages me. So maybe I'll be living on a beach somewhere one day. I don't know. But when [the future] presents itself, I'm sure I'll embrace it fully."

For more on Milo Ventimiglia, pick up the latest issue of PEOPLE, on newsstands Friday, or subscribe here. 

The Company You Keep airs Sundays at 10 p.m. ET on ABC.