The Challenge' s Natalie Anderson Reveals Why She Unexpectedly Left Double Agents

MTV

Natalie Anderson's sudden departure from The Challenge: Double Agents shocked viewers and her castmates alike. The Survivor winner and Challenge rookie, 34, already won her gold skull, meaning she secured a spot in the final. So when host T.J. Lavin announced that she'd been compromised and would be leaving the game, Double Agents lost one of its strongest competitors.

"Wes [Bergmann] reached out because he heard I left unexpectedly and he was like, 'I'm just letting you know, if you stayed, you would have won,' " Anderson tells PEOPLE of her first Double Agents partner. "I was like, 'Dude, I know.'"

RELATED: The Challenge's Big T Opens Up About Losing Her Parents and Brother: 'You Don't Get Used to That'

On the show, Anderson simply cited "a personal matter" as her reason for exiting. Now, Anderson feels comfortable sharing that she left the show because she found out she was unexpectedly pregnant.

"It would be really unfair to say I left because it was difficult. My fans know me. I embrace difficult. They're not going to buy, 'The house was too stressful for me,'" Anderson says of The Challenge. "I want to own my story."

Anderson and her boyfriend of over one year Devin Perez hadn't been trying to conceive — production only asked her to take a pregnancy test after she told her twin sister and Amazing Race partner Nadiya over FaceTime that she had missed her period. Anderson didn't think anything of her missed period though because of the stressful circumstances.

Never miss a story — sign up for PEOPLE's free weekly newsletter to get the biggest news of the week delivered to your inbox every Friday.

"On all my reality TV shows I'm always really irregular, especially on Survivor, because I'm starving," Anderson says. "I never got my period the entire time I was out there. So I just chalked up the differences in my body to the stress of the situation."

Still, production wanted to "take the precautionary steps," Anderson says, "and so they asked me to take a pregnancy test. I literally did not think it was going to be positive."

When it turned out that way, Anderson knew she couldn't continue to compete, as much as she wanted to. "I literally said, 'I think I can do this pregnant. Let me stay and compete.' And production just looked at me like, 'This girl is crazy,'" she recalls.

"I know my body and I would've put myself in that. If I could have signed some waiver and competed pregnant, trust me, I would have done it. But there was really no way from a liability perspective they would ever allow me to compete pregnant. I knew that as soon as I found out," says Anderson.

(On last season's The Challenge: Total Madness, Melissa Reeves unknowingly competed pregnant before audiences saw her quit at the end of season 35. "I was thinking to [my]self, why am I struggling so much? I was at peak fitness," Reeves, who welcomed daughter Vienna in May, recalled during the reunion, according to MTV.)

RELATED VIDEO: Tori Deal DM'ed Nelson Thomas Saying She Made the 'Biggest Mistake' Sending in Fessy Shafaat

Tori Deal DM'ed Nelson Thomas Saying She Made the 'Biggest Mistake' Sending in Fessy Shafaat

MTV's 'The Challenge: Double Agents' airs on Wednesdays at 8 p.m. ET

Anderson's pregnancy news came as she found herself in a good spot in the game, especially for a newcomer.

"It seemed like this really unfortunate thing at a really bad time," the former Amazing Race contestant says. "I was sure that none of the girls would try and challenge me for my gold skull. I was sitting pretty for the final. The check was already written for me. I was ready to win. I couldn't feel happy about it because I was out there for one reason and that was winning."

Reluctantly, Anderson left Iceland and once she returned to the U.S., the reality star learned she was eight weeks pregnant. "Watching back the season, I look at myself and I'm so proud of what my body was able to do pregnant," Anderson says. "It's crazy. I never once got sick. I never had any morning sickness. I was just stuck in this crazy house sleeping on a bunk bed with a bunch of lunatics."

Catch People (the TV show!) Monday through Friday. Check your local listings for exact times or stream the show every day at 7 p.m. ET/ 4 p.m. PT at People.com and PeopleTV.com.

Anderson soon started looking forward to motherhood. She adjusted her exercise routine accordingly and anticipated starting a family with her boyfriend.

"I said, 'You know what, maybe I didn't get to win half a million dollars, but everything happens for a reason,'" she continues. "My boyfriend is already a father and he's somebody I could see myself having a family with. We were like, 'let's do this.' I was ready for this new chapter."

One week after coming home in September, Anderson's story changed once more when she suffered a miscarriage. "I had to digest losing the pregnancy and being like, 'Now I have no Challenge win under my belt and I don't have this new journey I was looking forward to,'" she says. "It was really difficult."

Anderson relied on working out for a sense of normalcy and leaned on her boyfriend for support. "I had to really go through the healing process with myself and my boyfriend," she says. "I focused on what always makes me feel good, which is the routine of training."

She also found strength in having missed out on the $2 million prize in Survivor: Winners at War.

"Losing $2 million, it seemed like the hardest thing I would ever have to do until this happened to me," Anderson says. "And this was even harder because the $2 million, I had to take responsibility for it playing out the way it did. I never thought I would lose something that would be as hard as losing $2 million. But this was even harder."

CBS via Getty Natalie Anderson on Survivor: Winners at War.

Though none of her Challenge castmates knew about her situation at the time, Anderson has since confided in Tori Deal and Aneesa Ferreira, the latter of whom she says "really helped me embrace this and use this in a way that can be positive."

Now Anderson feels "excited to move forward and figure out the next steps in my life."

RELATED: 34 Celebrities Who've Bravely Opened Up About Their Miscarriages, in the Hopes of Helping Others

"I really did do the work necessary to get to this space that I'm in now," she says. "After any kind of loss, it's up to us to create that safe environment to come back to a healthy head space. And if I can help anybody, that was my goal. I want anybody who is going through something hard to remember: the last thing you need to do is be hard on yourself. Be kind to yourself and embrace all the difficult stuff that happens in a way that you can grow and become a stronger person on the other side."

Anderson hopes to return to The Challenge and jokes that after this whirlwind experience, "I feel like the universe owes me $2 million from Survivor and half a million from The Challenge."

The Challenge: Double Agents airs Wednesdays (8 p.m. ET) on MTV.