Jeff Lewis Says Custody Deal with Ex Has Fallen Apart: 'I’m Co-Parenting My Child with a Child'

Jeff Lewis and his ex Gage Edward’s tentative custody agreement over their 2-year-old daughter Monroe has begun to fall apart after the pair cancelled a scheduled mediation, a source tells PEOPLE.

On Tuesday’s episode of his SiriusXM show Jeff Lewis Live, the former Flipping Out star detailed the decline of his and Edward’s co-parenting situation. The exes, who were together for 10 years before splitting in February, had reached a temporary shared legal custody agreement in August, but are now unable to keep the peace as they continue to negotiate with lawyers and privately.

“I have begged him to try to work this out together, but what happens is, sometimes, in all honesty, I feel like I’m co-parenting my child with a child,” Lewis, 49, said on the show of Edward, 34.

He added, “On Sunday, I literally put my face in my hands and thought, ‘This guy is going to torture me for the rest of my life. I’m a prisoner.’”

The pair were scheduled to go to private mediation before a judge on October 3 to set a permanent custody agreement, but a source close to the situation told PEOPLE exclusively they were trying to work it out instead, without mediation.

“They have tried for two months now to negotiate custody since the original temporary order was put in place,” the insider said.

Since parting ways with Edward, Lewis has consistently said his ex is “a good dad” and “good with [Monroe],” despite their often contentious personal relationship. But on Tuesday he seemed to walk some of that back.

“Whatever he did to me, I can get over,” Lewis said on his show. “Here’s the problem: I don’t believe that he always put our child first. I don’t believe he always considered our child’s well-being and what was best for her, and for that, I can never forgive him.”

Lewis had previously accused Edward of not creating a child-friendly home after their split, saying his new apartment “looks like a prison,” and criticizing him for choosing to live farther away from their daughter’s previous preschool, from which she was recently expelled over comments Lewis had made on his radio show. (She’s now attending a new school, after Lewis said he made “a donation.”)

The XM host also claimed on Tuesday’s show that Edward never has custody of Monroe on Friday or Saturday nights, allowing him more flexibility in his social life.

Taking things a step further, Lewis made the case that splitting custody “50/50 was not the best for [Monroe].”

“For her best interest, I have been fighting for the majority of the custody,” he said. “I have the house, I have the infrastructure, I have the playroom, the pool, I’ve got the backyard. I’ve got the nanny, the housekeeper, and everything. Plus, all of her classes and her school are within two miles of my house.”

He added, “I have made the choice to live in Sherman Oaks. I have made the choice to live in this house 150 percent for my child. Every day — everything I do — she’s on my mind 24/7.”

Edward did not reply to PEOPLE’s request for comment about Lewis’ claims.

In the wake of their tentative August agreement, Lewis said he and Edward had been co-parenting relatively peacefully — that was, until Edward’s allegedly flip-flopping on parenting decisions caused their most recent falling out.

“We were getting along great after the temporary schedule was in place,” he said. “We had a great week, we started really deeply talking about, ‘What does a permanent custody schedule look like?’ Holidays, the whole thing.”

Days later, “everything we discussed was changed,” Lewis said on Jeff Lewis Live. “I think what happens is he goes and he consults someone. And I don’t know if it’s his boyfriend, or a good friend or whatever. But suddenly, everything he has talked about is off the table. … I get this email from him that says, ‘None of this works. I think the attorney should handle it.’ “

All that has left Lewis at a crossroads.

“I have been very communicative and very open. That’s going to shift now,” he said. “I’m going to stonewall it. If this is the way he’s going to play it, there will be no more communication. … I want limited contact [with Gage]. I want a legal schedule. I’m done. I’m f—king done.”

Jeff Lewis Live airs weekdays (12 p.m. ET) on SiriusXM’s Radio Andy (Ch. 102).