Chip Gaines Explains Why He and Joanna Feel They Were 'Naive' About Putting Their Kids on TV

Cindy Ord/Getty

Chip and Joanna Gaines have seen shifts in reality TV in their times on the airwaves.

Reflecting on their journey in the business during an appearance on the Kennebec Cabin Company, the Magnolia mogul talked about how he and his family have always just seen themselves as "business operators" who happen to also film.

In feeling that way, Chip, 48, shares that the couple struggled with whether to have their kids appear on camera.

"Where we got a little dazed and confused through it, I think — even the kids, a little bit — was, 'I don't know. I mean, do the kids need to be a part of this, do they not?'"

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

RELATED: Joanna Gaines Shares Her Selfie Fail with Daughter Emmie on Windy Day — See the Hilarious Photo!

"And for sure, we've gone through that and that cycle where — at the very beginning, we had a production company, and again, I'm not throwing them under the bus. I think it was a great strategy and we didn't even mind it. We were kind of naive, and dumb, really, to be frank enough to be like, 'Let's have the kids be involved.'"

He explained that their feelings about it started to change when the kids started getting recognized.

"But then as the kids got into fourth and fifth seasons and they're getting a little bit older in age, and now they're getting recognized out in the wild, mama bear came out and papa bear. I'm like, 'I don't know that I want my kids to be visible in that way unless they want to be visible.'"

Todd Owyoung/NBC via Getty

"If this is the path they want to be on and they choose to go this route, well then I want to be the biggest cheerleader for them as if they picked up tennis or piano. Let's go, let's go make you the best in those spaces as you can possibly be. But none of my kids were really drawn to it that way."

"So Jo and I started realizing, 'Man do we have to have the kids involved in this particular thing and that?' And as we pushed back on that, the answer could be yes sometimes and no sometimes."

The Fixer Upper alum and husband Chip Gaines share daughters Emmie, 13, and Ella, 16, and sons Crew, 4, Duke, 14, and Drake, 18.