Evan Vucci/AP/Shutterstock Dr. Anthony Fauci
In an interview with The New York Times published Sunday, Fauci — who often found himself at odds with the Trump administration, which downplayed the severity of the coronavirus pandemic — shared he began receiving death threats last March.
"It was the harassment of my wife, and particularly my children, that upset me more than anything else. They knew where my kids work, where they live. The threats would come directly to my children's phones, directly to my children's homes," said the coronavirus task force member and director of the National Institute of Allergy and Infectious Diseases, 80.
"And there was chatter on the internet, people talking to each other, threatening, saying, 'Hey, we got to get rid of this guy. What are we going to do about him? He's hurting the president's chances.' You know, that kind of right-wing craziness," he added.
Describing one particularly "disturbing" incident, Fauci shared that he once received a letter, and upon opening it a "puff of powder came all over my face and my chest."
"That was very, very disturbing to me and my wife because it was in my office. So I just looked at it all over me and said, 'What do I do?' " he said. "The security detail was there, and they're very experienced in that. They said, 'Don't move, stay in the room.' And they got the hazmat people. So they came, they sprayed me down and all that."
Although the powder turned out to be "nothing," Fauci said it was a "frightening" ordeal.
"My wife and my children were more disturbed than I was," he continued. "I looked at it somewhat fatalistically. It had to be one of three things: A hoax. Or anthrax, which meant I'd have to go on Cipro for a month. Or if it was ricin, I was dead, so bye-bye."
RELATED VIDEO: Dr. Anthony Fauci Publicly Receives the Moderna COVID-19 Vaccine
Dr. Anthony Fauci Publicly Receives the Moderna COVID-19 Vaccine
The infectious-disease expert received the vaccine on TV Tuesday morning
Despite all of the challenges he experienced, Fauci said that he "never" thought about quitting.
"When people just see you standing up there, they sometimes think you're being complicit in the distortions emanating from the stage. But I felt that if I stepped down, that would leave a void," he explained. "Someone's got to not be afraid to speak out the truth."
Even after his wife, he says, "brought up that I might want to consider" stepping away, Fauci remained committed.
"And after a conversation, she ultimately agreed with me," he said. "Even if I wasn't very effective in changing everybody's minds, the idea that they knew that nonsense could not be spouted without my pushing back on it, I felt was important. I think in the big picture, I felt it would be better for the country and better for the cause for me to stay, as opposed to walk away."
"One of the new things in this administration is, if you don't know the answer, don't guess. Just say you don't know the answer," he said, adding later that a crucial importance will be placed on making "everything we do be based on science and evidence."
As information about the coronavirus pandemic rapidly changes, PEOPLE is committed to providing the most recent data in our coverage. Some of the information in this story may have changed after publication. For the latest on COVID-19, readers are encouraged to use online resources from the CDC, WHO and local public health departments. PEOPLE has partnered with GoFundMe to raise money for the COVID-19 Relief Fund, a GoFundMe.org fundraiser to support everything from frontline responders to families in need, as well as organizations helping communities. For more information or to donate, click here.