Hands Off: Luke Bryan Wants Fans to Know His Butt Is Officially Off-Limits (Except to Ellen)!

Luke Bryan fans, control yourselves!

Apparently, Bryan's booty is so popular that he's actually had to start a "no touching" policy at meet and greets.

"They'll sneak one in on ya," Bryan told Ellen DeGeneres on Wednesday. "It still happens – we've learned to understand it."

Bryan, who displays some pretty impressive butt-shaking dance moves on stage, explains the problem all started back when he was just a rising star.

"It's funny 'cause you see, like early on in your career you don't want to tell your fans that they can't interact with you so you let them get away with kisses on the cheek and all that," says Bryan. "Someone will go, 'Can I grab your butt?' ... and back then if it was a polite ask, we would oblige them. But then you got 90 others..."

A stunned Ellen asked if this was a problem for all country stars or just him, but Bryan needs to do a bit more research before he can answer.

"Me and other country stars don't necessarily sit around and talk about it – I mean, how many butt grabs you had last week," joked Bryan. "I'll run it by Blake, see what he says."

"It's country music," Bryan continued. "We make the fans happy."

Don't miss a beat of country music news, photos and videos! Click here to get all this and more in the PEOPLE Country Newsletter.

The "Move" singer also talked about turning the big 4-0 in July.

"Thirty was tough – 30 hit me pretty rough, and then 40, I think you just mentally kind of give up at that point," he joked.

But Bryan did make sure to celebrate the milestone even though he was on the road.

"I had a show in Bangor, Maine on my 40th and so [my wife] Caroline and a bunch of friends flew in and we rented a log cabin up on some lake that we didn't know the name of," says Bryan. "They probably filmed Friday the 13th there, so we survived that, but we had a great time – it was a great 40th. "