Tom Brady's Return to NFL 'Put a Strain' on Marriage with Gisele Bündchen: She 'Believes Enough Is Enough'

Gisele Bündchen is "frustrated" with husband Tom Brady's decision to unretire, multiple sources tell PEOPLE.

The 42-year-old supermodel "has been spending time away from" the Tampa Bay Buccaneers quarterback, who initially retired from the NFL in February before announcing his return for the 2022-23 season less than two months later.

"Going back to Tampa after the retirement put a strain on the personal side of his life," the source says. "Gisele is frustrated and sick of his career coming before their family, who has always supported him."

Dimitrios Kambouris/Getty

The couple spending time apart from one another "has happened before," the source explains, adding that "they are trying" to make the relationship work.

For more on Gisele Bundchen and Tom Brady, listen below to our daily podcast PEOPLE Every Day.

This is the 45-year-old Brady's 23rd season playing in the league, and according to a second source, "There is a point Gisele believes that enough is enough."

RELATED: Gisele Bündchen Tweets Support for Husband Tom Brady, but Skips His First Game amid 'Rough Patch'

The supermodel has "always been an amazing wife, totally supporting Tom and his career and bringing the kids aboard," the source says of the couple's two children, Benjamin Rein, 12, and Vivian Lake, 9, along with Brady's son John "Jack" Edward, 14, whom he shares with ex Bridget Moynahan.

Mike Ehrmann/Getty Images

RELATED: Gisele Bündchen Currently Not Set to Attend Tom Brady's First Game: 'Nothing Is in Place'

Bündchen is "not a stay-at-home mom," a third source close to the family says, and only retired from doing runway modeling. "She has been working all along. She's in N.Y.C. right now working, and the kids are enrolled in school in Florida."

The two had a "happy summer together," prior to the NFL season, the insider adds, with Bündchen throwing Brady "a big birthday party in Italy." But their relationship grew rocky as the season approached.

"He already announced his return to football this spring, so friends don't understand why she's upset now."

"They've been very private about what's really going on," the source added.

Kevin C. Cox/Getty

Bündchen's absence during Brady's first game of the regular season on Sunday was noted by fans and members of the Buccaneers organization. "It feels very different this year than last year," an additional source told PEOPLE prior to Sunday's kick-off. "She was excited to be here, we saw her around."

Despite her absence from Tampa Bay's first game, Bündchen tweeted, "Let's go @TomBrady ! Let's go Bucs ! ✨✨✨," on Sunday.

The insider went on to say they're staying away from bringing up Bündchen around the 7-time Super Bowl champion. "Everyone here knows that there's tension, but we don't know how serious it is. So we're just not talking to him about Gisele. It's not a topic anyone wants to bring up to him."

RELATED VIDEO: Tom Brady and Wife Gisele Bündchen 'Hitting a Rough Patch' in Marriage: 'There's a Lot of Tension'

A separate source previously told PEOPLE that Bündchen "wasn't thrilled" about Brady coming out of retirement. "There's a lot of tension. She was so happy when he announced his retirement, and she wasn't thrilled at all when he went back on that."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Rumors of a rift between the couple increased after Brady was excused from practicing with the Buccaneers last month "to deal with some personal things," head coach Todd Bowles said during a press conference at the time. According to Bowles, Brady and the team discussed the absence before the start of training camp in July.

Brady spoke about his absence from the team after he eventually returned 11 days later. "Everyone has different situations they're dealing with, and we all have unique challenges to our lives," Brady told reporters. "I'm 45 years old, man."