White Lotus Star Steve Zahn Was in the Best Shape of His Life While Studying Ballet in His 20s

While Steve Zahn certainly wowed viewers with his shirtless scenes in the hit HBO series The White Lotus, the actor recalls a time he was even more fit.

"When I studied ballet at conservatory in my 20s, it was intense," Zahn, 54, told PEOPLE on the set of his Sexiest Man Alive shoot last month. "I was in really good shape."

Zahn's regimen shifted gears after he and his wife Robyn Peterman became parents (Their son Henry is now 21 and daughter Audrey is 19).

"When you have little kids, it's like, 'No more waffles? You sure? Great. All mine,'" he says. "That was bad news."

RELATED GALLERY: 25 of the Sexiest Men You Can Watch on TV Now

"With kids, your responsibilities are different, and your focus is different," Zahn adds. "I remember when I was driving my kids around after school, that's all I did. I was a driver. That's it. I would take Audrey to dance and then take Henry to lacrosse. While he was at lacrosse, I would change in the truck, and I would go run. Then I would take him home. So it was almost possible to work out."

Yu Tsai Steve Zahn

These days, Zahn, who is starring in the new HBO Max film 8-Bit Christmas, is back to working out on the Kentucky farm he calls home — and running about 25 miles every week.

"I love to run — it's a kind of therapy for me," he says. "And I think I look alright. I mean, if you had all the guys I graduated with in here, I'd be really hot."

In addition to running, Zahn also keeps fit by eating vegan food with his daughter.

"Audrey is very healthy," he says. "About three years ago, she decided she was going vegan. We always ate pretty good, but that really put it in a new spectrum."

Zahn does admit that staying in shape "gets harder when you get older."

"In your 50s, your start getting panicky. You're like, 'My hand hurts. Am I dying?'" he jokes. "They're like, 'No, that's the beginning of arthritis. You're good. Don't worry about it.'"

For more on the Sexiest Man Alive — and more of Hollywood's hottest men! — pick up the latest issue of PEOPLE, on newsstands now.

The plus side of growing older has been getting "wiser," Zahn says.

"You realize you don't know as much as you thought you did when you were 25. So, you just shut up," he shares. "You're the quiet guy, and people are like, 'The quiet guy? He knows a lot.' It's truly the opposite. I'm just not going to say anything anymore!"

While Zahn says his kids don't need his advice, there are a few things he reminds them as they pave their own way in the entertainment industry: "Overprepare, show up early and don't be a d—."

"No one cares what your last name is or what school you went to," he says. "Kill it."

RELATED VIDEO: Ryan Phillippe on Trying to Keep Up with His Son, 18, in the Gym — 'It's Really Defeating'

He's also taught them that fame and success can mean two different things.

"I've never thought of myself as famous," he says. "I just think of myself as a recognizable person. If somebody asks you, 'Are you famous?' 'Well, obviously not because you just asked me.'"

"I remember when I was young I thought, 'When I make a movie, [I'll be famous],'" he continues. "Not true. It's really your 20th movie where someone's like, 'Do I? Do we?' They do the high-pitched thing. It's really fascinating. I had a gradual kind of existence. Right now I'm on a big high, because I was in a water cooler show, but I know what that means, and I know how long that lasts. It'll give me some more opportunities to do some really good work and extend it a little longer."

RELATED: Jennifer Coolidge to Reportedly Return to HBO's The White Lotus for Season 2

Despite all of his success, Zahn says it's the "simple things" that he's most grateful for now, including his family and health.

"I cannot imagine not having kids," he says. "And I'm grateful to live where I live and wake up and go, 'Hmm. What should I do today?' Some days you get a lot of stuff done. Then some days I play Bubble Pop!"