Today Show Colleague Says Rejecting Matt Lauer’s 'Flirting' Would 'Put a Target on Your Back'

Matt Lauer had a habit of “flirting” with junior NBC staff members long before he was fired for alleged sexual misconduct or was accused of raping colleague Brooke Nevils, a source tells PEOPLE.

According to an excerpt of Ronan Farrow‘s upcoming book, Catch and Kill: Lies, Spies, and a Conspiracy to Protect Predators, published in Variety, Nevils alleges that Lauer anally raped her in his hotel room at the 2014 Sochi Olympics, when she was working for Meredith Vieira. Nevils reportedly told Farrow she was “too drunk to consent” and stated multiple times that she did not want to have anal intercourse. Nevils said in the book that she had more sexual encounters with Lauer back in New York City, according to Variety, telling Farrow: “It was completely transactional. It was not a relationship.”

Lauer, 61, penned a letter to Variety denying the allegations, calling the alleged encounter “extramarital, but consensual.” But a former Today show colleague tells PEOPLE that Lauer was known for engaging in inappropriate relationships with junior staffers — and punishing those who rejected his advances.

“I wish I could say any of it surprised me, but it doesn’t. I guess I’m a little surprised that he still doesn’t get it, that he still thinks it was okay to have ‘affairs’ with co-workers who absolutely had every reason to feel nervous if they didn’t find his attentions flattering,” the source says. “If you didn’t find his ‘flirting’ behavior charming — and just using the word ‘flirting’ feels wrong, because it was so much more predatory than that, and it was not harmless — then you put a target on your back.”

RELATED: How Meredith Vieira Helped Brooke Nevils Come Forward with Matt Lauer Rape Allegation

“If you didn’t find his comments about your a– or your weight loss, or your ‘beautiful eyes’ to be funny or flattering, if you dared to say it was inappropriate or worst of all, asked him to please not say that again, then you were either dead to him, like you really became invisible, or worse, you were someone he’d talk about in negative terms: ‘She’s so uptight, she’s not a team player, she’s really uncooperative, she’s got a terrible work ethic.’ You were either able to be ‘one of the guys’ with him, or you were in jeopardy.”

RELATED: Matt Lauer ‘Believes He Is Innocent’ and ‘Feels Angry’ About Rape Allegation, Source Says

In his lengthy letter to Variety, Lauer — who was married to now ex-wife Annette Roque at the time — repeatedly characterized his relationship with Nevils as “mutual.”

“At no time, during or after her multiple visits to my apartment, did she express in words or actions any discomfort with being there, or with our affair,” he said. “She also went out of her way to see me several times in my dressing room at work, and on one of those occasions, we had a sexual encounter. It showed terrible judgment on my part, but it was completely mutual and consensual.”

Lauer acknowledged that people were aware of the affair, but insisted he has “never assaulted anyone or forced anyone to have sex. Period.”

RELATED: Matt Lauer’s Ex-Wife Annette Roque Breaks Her Silence on Rape Allegation

PEOPLE’s source calls Lauer’s statement “aggressive and menacing.”

“That’s the Matt I saw — someone who could be vicious, especially when cornered or challenged,” the source says. “As for Brooke, I absolutely believe her. And I also believe that there will be elements of this story that will remain hard for people to understand ⁠— how you can find yourself numbly going along with something that isn’t what you want, only for it to take some time to realize how thoroughly you were manipulated and taken advantage of. How you were far less of a willing participant than you led yourself to believe at the time, because doing so at the time would’ve meant realizing how toxic and warped and unfair your situation was, which can be hard to realize while you’re in the middle of it.”

The source also mentions Addie Collins Zinone, who came forward in 2017 about her alleged month-long affair with Lauer when she was a 24-year-old production assistant at Today. (In a statement Thursday, Zinone accused Lauer of attempted to slut-shame Nevils and lambasted his “brazen and predatory abuse of power on young, vulnerable women who had no voice.”)

“In hindsight, I think what Brooke is saying, I think what Addie said, I think what any of the women who come forward will say, will not only be the truth, but there will be a ton of similarities to each of their stories,” the source says.

Nevils herself responded to Lauer’s letter on Wednesday night, calling it a “case study in victim shaming.”

“There’s the Matt Lauer that millions of Americans watched on TV every morning for two decades, and there is the Matt Lauer who this morning attempted to bully a former colleague into silence,” Nevils said on a statement that aired on NBC Nightly News.

“I am not afraid of him now,” Nevils said. “Regardless of his threats, bullying, and the shaming and predatory tactics I knew he would (and now has) tried to use against me.”

If you or someone you know has been sexually assaulted, please contact the National Sexual Assault Hotline at 1-800-656-HOPE (4673) or go to online.rainn.org.