Walton Goggins breaks down his latest musical moment on The Righteous Gemstones

A pay day is coming. Over the first two seasons of The Righteous Gemstones, a consistent thorn in the side of the titular televangelist dynasty was their louche uncle, Baby Billy Freeman (Walton Goggins) — or, as Jesse Gemstone (Danny McBride) and siblings memorably refer to him, "Uncle Baby Billy." He's connected to the family through late matriarch Aimee-Leigh (Jennifer Nettles), whose death left Baby Billy out of sync with the children of Eli Gemstone (John Goodman).

"In season 1 we explored what Baby Billy meant to the family and contextualized what the loss of a sister and wife does to two men," Walton Goggins tells EW. "It was both John and I dealing with the loss of the most important person in our lives. And then season 2 was exploring him being a father. But for this one, Baby Billy was unencumbered by that kind of dramatic storyline. So Danny just said, 'Let's go have some fun.'"

But for all the frustration that Baby Billy caused the Gemstone family in seasons 1 and 2, he also delivered some of the show's most iconic musical moments, such as his performance of "Misbehavin'" alongside Aimee-Leigh in season 1's flashback episode. So when Baby Billy took center stage in season 3 this week, it was naturally in the form of another song: "Pay Day." Watch it above.

HBO Walton Goggins on 'The Righteous Gemstones'

One of The Righteous Gemstones' main thematic interests is the intersection of Christianity and capitalism, where Sunday services are as much about making money as praising the Lord. "Pay Day," a new version of an old song unearthed by music supervisor Devoe Yates, is a perfect encapsulation of that fusion, where a long-awaited paycheck can feel the same as the heavenly reward waiting for devout Christians. Much like "Misbehavin'," it's also catchy as hell.

"They sent me this song while I was working in Namibia," Goggins recalls. "I just started getting it stuck in my head. At first I couldn't get the rhythm down, but then it clicked. I was like, this is a great story, and it speaks to his deep need for upward mobility. The words to that song are very much who he is and what he believes: That if he keeps fighting the good fight, eventually his stars will align and he will also be worthy of celebration."

But it's not just the song that makes the scene; it's also that Goggins is singing in a bright blue costume (designed by Christina Flannery, taking over from the show's previous costume designer Sarah Trost) as part of his new gig as a performer at the Gemstones' beach resort, Zion's Landing.

"It was such a great experience," Goggins says. "As with a lot of these things that Danny and Jody [Hill] and David [Gordon Green] do, it's one thing on the page, and it's another on the day. It was scripted as a musical number, but it wasn't originally meant to take place in the mind of Baby Billy. But that was something that Danny had in his head. So we got there and I said, 'Man, I'll go wherever you want me to go. There is nothing that I won't do.' And he said, 'Okay, put on this clamshell then!'"

HBO Walton Goggins on 'The Righteous Gemstones'

Energized by his own song, Baby Billy decides that he needs to pursue his stardom again and get back on TV. To that end, he pitches the Gemstones on a new version of Family Feud based around Bible trivia.

As with so many things related to Baby Billy, the name of that proposed show is hilarious on its face, just at the level of words and sounds. Rest assured, Goggins tells EW that he had to practice saying Baby Billy's Bible Bonkers over and over a thousand times ("Not 999 times — a thousand times," he assures us) before he could roll it off his tongue during filming without breaking or stumbling.

"For me, playing him is exhausting," Goggins says. "But it's exhilarating too. I understand his struggle, that need to be seen for who you really are. I really think that's the motivation behind Baby Billy's kinda Machiavellian nature. He just wants to be seen for who he is."

As always, Baby Billy's partner in crime on his latest scheme is his much younger wife, Tiffany (Valyn Hall). Goggins has nothing but praise for his costar.

"She's the pepper in the sauce," Goggins says of Hall, after deciding that "secret weapon" is too trite of an expression. "I had no idea who Valyn was before we got together for season 1. But after the first scene that we did together, I just looked at her, and then I looked at Danny like, 'Who the f— is this??' We just fell in with each other. She stands toe-to-toe with everyone on that show. She has an energy about her and a willingness to play that is admirable. I really think the world of her. There would be no Baby Billy without Tiffany."

HBO Valyn Hall and Walton Goggins on 'The Righteous Gemstones'

Although his Bible Bonkers plan has stalled for now, we haven't seen the last of Baby Billy on season 3 of The Righteous Gemstones. Goggins doesn't want to spoil where future episodes are going, but he did say he had one particular inspiration in mind for whatever Baby Billy's up to.

"The only thing I will tease is that one person came into my mind for the finale and this one particular experience in that episode," Goggins says. "I said, 'If I can do a 74-year-old, limited-mobility version of Tina Turner, that's what I want to do.'"

Watch the "Pay Day" clip above. Season 3 of The Righteous Gemstones is currently airing new episodes Sundays on HBO and Max, with the finale set for Aug. 6.

Sign up for Entertainment Weekly's free daily newsletter to get breaking TV news, exclusive first looks, recaps, reviews, interviews with your favorite stars, and more.

Related content:

• Righteous Gemstones creator Danny McBride shares first look at 'nastier and crazier' season 2

• Danny McBride breaks down Righteous Gemstones season 1 finale: 'I like to see things have a completion'

• Walton Goggins on becoming Baby Billy for Danny McBride and Righteous Gemstones