Former NFL QB Kurt Warner Says Tom Brady, Aaron Rodgers Look 'Exhausted': 'It's OK to Walk Away'

Patrick McDermott/Getty Images; Scott Winters/Icon Sportswire via Getty Images; Grant Halverson/Getty Images Aaron Rodgers; Kurt Warner; Tom Brady

Former NFL quarterback Kurt Warner has some observational notes and words of wisdom for Tom Brady and Aaron Rodgers.

"I think both of those guys look like they're exhausted," the NFL commentator tells PEOPLE exclusively. "It's OK to go, 'You know what? It's not how it used to be. I may not be who I used to be, or, it's just too hard overall to get everybody up to the standard. Maybe it is time to walk away.' "

Brady and Rodgers have been struggling for various reasons on the field this season — in Brady's case, coming out of retirement and dealing with marriage trouble with Gisele Bündchen at the same time. But based on his own retirement experience, the 51-year-old wants his fellow quarterbacks to know that no matter what their futures hold, it's all going to be OK.

"It's not being defeated," Warner advises. "That's not saying you can't do it anymore. I just think sometimes that stuff outweighs the joy of the game, which is what it looks like is happening for those two guys and I feel for them."

Phillip Faraone/Getty Images; Kevin Mazur/Getty Images; Cooper Neill/Getty Images

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Warner's own career began as an undrafted free agent with the St. Louis Rams in 1998, where he eventually moved from a back-up to the team's leader, earning himself two MVP titles and a Super Bowl title before leaving in 2003. After a quick stint playing for the New York Giants in 2004, he signed with the Arizona Cardinals in 2005 where he led the team to its first Super Bowl before calling it on his career in 2009.

The pressures of the game and balancing his wife and seven children at home were tough, and helped Warner decide when it was time to walk away from the game.

"You try to balance that the best you can or you try to balance it in the off-season," the Hall of Famer shares. "It's a high pressure job with a lot of expectations and it's tough. Another reason why those last couple years I was doing more and more and more [was] because I felt like there was more expected of me or there was more I needed to do for us to be successful. It started to take more and more away from my family and my time, and again, me physically to be able to give them the time and effort that I wanted, because my mind was always on other things. So all of that really just kinda played into why I made that decision, but it's hard."

Jamie Squire/Getty Images

RELATED: First Look! See Zachary Levi as NFL MVP Kurt Warner in American Underdog Biopic

Warner knew going into his last season on the field that it would be his final one, but the years leading up to the decision to retire weren't easy, and he sees that in players like Brady today.

"As I was going through the last couple years, it was becoming more and more prevalent that, 'OK, this is taking too much from me and it's not just about enjoying the game anymore,' " Warner says, adding, "I mean, I kinda look at Tom Brady now, and that's how I feel.'"

Mark Cornelison/Lexington Herald-Leader/Tribune News Service via Getty Images

As he got older and his stats weren't what they once were, Warner dealt with the harsh critics towards the end of his career and did his best to tune them out.

"That's what we've done our whole lives," he shares. "No matter when you play, whether it's about retirement or it's about you're great or you're awful, those headlines are always out there. People are always gonna say what they say. At the end of the day, all I did was prepare like I always prepare. That to me was what you do. I didn't dwell on the headlines any longer. I didn't think about retirement more often."

Today, Warner gets to sit back and relax while still being a part of the game by coaching high school and providing commentary, but he also gets to focus on other passion projects — his charities and family.

RELATED VIDEO: Inside Kurt and Brenda Warner's Real-life Love Story That Inspired 'American Underdog'

Warner has not one, not two, but three charities. One he began while he was still in the NFL, called First Things First, focused on connecting to the community and giving back. "For instance, we take Make-A-Wish families with us to Disney every year and spend a week with them down at Disney," he says. Another, Homes for the Holidays, works with groups like Habitat for Humanity to help single parents obtain homeownership.

 RELATED: Inside Kurt and Brenda Warner's Real-life Love Story That Inspired American Underdog

"Because we remember those days when my wife was a single mom and and we were dating and we were trying to figure out those pieces of life," he says.

"And then our newest foundation is called Treasure House, and it is a community living facility for young adults with intellectual and developmental disabilities."