'It's Hard to Live Until You're 95,' Jimmy Carter Says: How Wife Rosalynn and His Faith Keep Him Going

For the record, Jimmy Carter never expected to live this long either.

That’s what he told PEOPLE with a smile last week as he took a break from helping build Habitat for Humanity houses in Nashville, Tennessee. For 36 years, he and wife Rosalynn Carter have led an annual build for Habitat, erecting and fixing up more than 4,000 homes.

The partnership has inextricably bound them together — Habitat and the organization’s most famous supporters, there almost since the beginning.

Recalling the start of what would become a decades- and globe-spanning volunteer project, President Carter says he and Mrs. Carter weren’t exactly thinking about longevity when they and a few dozen others bussed up to New York City to work on a six-story apartment building in 1984. The couple had already volunteered with Habitat back in Georgia when, in passing while he was in N.Y.C. to speak at a church, President Carter swung by a Habitat build site there and said, “We need to bring some volunteers in to help.”

The Carter Work Project began.

“We knew that we had undertaken a major and very enjoyable hobby on the side,” he tells PEOPLE now. “We stayed busy doing other things” — including Nobel Peace Prize-winning diplomatic work — “but we devoted 36 years to Habitat.”

This year alone the Carters helped build 21 homes in the Nashville area, expanding a neighborhood Habitat first started about 10 years ago.

“One of the things Jesus taught was: If you have any talents, try to utilize them for the benefit of others,” says President Carter, now 95. “That’s what Rosa and I have both tried to do.”

They arrived in Tennessee last week only hours after he fell at his home in Plains, Georgia, requiring 14 stitches on his head and leaving him with a nasty black eye. But if cancer hadn’t kept him from Habitat four years ago, why should this?

“I had a No. 1 priority and that was to come to Nashville to build houses,” he told a crowd the night after his fall, rallying them to begin building for Habitat that week. By his side, Mrs. Carter, 92, said: “I look forward to this week all year long.”

RELATED: Jimmy Carter Recalls First Date with His Wife — and Telling His Mom He Knew He’d Marry Rosalynn

The oldest of four children born to Lillian Gordy Carter and James Earl Carter Sr., the 39th president — the oldest living one in American history — has survived both of his parents and each of his siblings by decades. In 2015, when he was diagnosed with cancer, which had killed the rest of his family, he prepared for an end that still has not come.

“I just thought I had a few weeks left, but I was surprisingly at ease,” Carter said at the time. “I’ve had a full life, I have thousands of friends … so I was surprisingly at ease, much more so than my wife was.”

Within months, the cancer was gone following successful surgery and innovative immunotherapeutic treatments. That November, the Carters were back at their annual Habitat build.

“It’s hard to live until you’re 95 years old,” he tells PEOPLE. “I think the best explanation for that is to marry the best spouse: someone who will take care of you and engage and do things to challenge you and keep you alive and interested in life.”

RELATED: ‘He Never Stops’ — Inside Jimmy Carter’s Surprisingly Active Life

He and Mrs. Carter, married since 1946, “have had a good life together,” he says. Decades of memories bond them as do some of their shared hobbies, including bird-watching (“Rosa and I have seen about 1,300 different species of bird”), tennis (they have a court behind their house) and, yes, downhill skiing — which they took up when he was 62.

Older now and physically frailer (President Carter actually uses the word “decrepit”), the couple plan only one year in advance. They savor periodic pauses in the public life from which neither will yet retire.

“Now when we have a quiet moment, like a birthday or something, we like to stay at home, just by ourselves, and enjoy a quiet day in our own house without any visitors and with minimum phone calls and emails coming in,” he says.

But there is always more work to do. That’s something to savor, in its way, as well.

“We can take a lot of pride out of the folks with whom we meet,” President Carter says.

“Sometimes, when we go into a community where we built houses maybe 35 years ago, or 25, 20 years ago, we try to visit those Habitat sites just to look at them and meet some of the longtime home owners,” he says. “They’re very proud of their house. We never find any houses that we have built with graffiti on the outside walls or with broken windows or un-mowed lawns. … They set an example for everybody that lives around [them].”

In 2020, the Carters plan to travel with Habitat to the Dominican Republic.

“I think both mine and Rosa’s minds are almost as good as they used to be, we just have limited capability on stamina and strength,” he says. “But we still try to stay busy and do a good job at what we do.”