"There were good and bad days," she says of co-hosting Live! With Regis and Kelly from 2001 to 2011. "I don't want to feel like I'm slamming anyone or that I'm being disrespectful. But I also want people to know it was not a cakewalk. It took years to earn my place there and earn things that are routinely given to the men I worked with. Including an office and a place to put my computer."
The Live with Kelly and Ryan co-host recalls her early struggles after joining the daytime talk show in 2001 following the departure of Philbin's previous co-host Kathie Lee Gifford.
"The biggest misconception is that it all came easily," Ripa says. "People think I just showed up one day and was handed a job and I lived happily ever after and now everything's perfect. But it never is that way."
After serving as a guest co-host a third time and being told repeatedly that the show was not looking for a permanent co-host, Ripa's agent called her with an offer. It also included a warning: "They want you to know who your boss is."
"It was very ominous, and it did not feel good," she says.
Her agent also said the show didn't want her "bringing an entourage." She brought two people. "I came with hair and makeup," she says. "It was not an unusual thing for people on a television show to show up with."
However, moments before she walked on stage, Philbin greeted her and the show's executive producer, Michael Gelman, with: "Uh-oh, Gelman, it's got an entourage."
"I felt horrible," she says. "He was probably trying to be funny, but at the same time it felt like a pile-on. I understand that probably he didn't want a co-host, but the network wanted me to be the co-host and I didn't think I should pass up that opportunity. I don't think it was fair to him. But it was also not fair to me."
Ripa says she still treasured spending time away from the show with Philbin, who died at 88 in 2020.
"Off camera and outside of that building, it was a different thing," she shares. "The handful of times we spent together, I so enjoyed. We went to the same resort once on vacation and he came to a dinner I hosted—one of the favorite nights of my life. I never laughed so hard."
In 2017, Philbin told Larry King that Ripa was "very offended" after his departure from the morning talk show, adding he had no communication with her and had been invited back to the program.
"The show had reached out many times," says Ripa, pointing out that he appeared on its 2015 Halloween episode. "It makes me sad but it's not something I will ever understand. I was steadfast in my attempt at communication. You can't make a person befriend you."
Never miss a story — sign up for PEOPLE's free daily newsletter to stay up to date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.
No matter their history, Ripa still considers Philbin "the world's best storyteller" and credits him for showing her the way.
"If I could become a tenth as good, I'd be happy," she says. "It's taking the audience on a ride with you and remembering that you are always the butt of the joke."
"I loved him," she adds, "and I still do."
For more about Kelly Ripa pick up the new issue of PEOPLE, on newsstands Friday.