Behind Sidney Poitier's 'Slap Heard Round the World' That Made History in 1967's In the Heat of the Night

United Artist/Getty

The late Sidney Poitier broke ground as a Black actor throughout his career. His portrayal of Virgil Tibbs in the 1967 film In the Heat of the Night was no different.

The movie, released amid the civil rights movement, tells the story of a Philadelphia detective that assists in a Mississippi murder case with a racist policeman and features some of the most powerful moments from Poitier's career.

Filmmaker Norman Jewison and actress Lee Grant offered a behind-the-scenes look at Poitier's time working on the critically-acclaimed film in separate interviews with PEOPLE after the actor died on Thursday at age 94.

Jewison, 95, initially planned to film in Mississippi, where the movie was set. However, Poitier requested the movie be shot above the Mason-Dixon line, so the director improvised. Poitier was concerned with the "unsettling" racism he encountered on a prior visit to the South with Harry Belafonte as part of an effort to register Black voters, per The Hollywood Reporter.

RELATED: Sidney Poitier's Family Issues Statement on His Death: 'He Is Our Guiding Light'

"I found this little town in Southern Illinois near the Mississippi River and it was called Sparta," Jewison told PEOPLE. "And I thought 'I'll just make it Sparta [Miss.] I won't have to change any of the signs on the water tower or anything.'"

Jewison eventually convinced Poitier to film below the Mason-Dixon line in a small Tennessee town for a single scene featuring a cotton plantation, but quickly came to understand Poitier's initial concerns.

Archive Photos/Getty

"I'll never forget, there was one hotel in town and it had a sign on its door saying, 'Whites only.' So we ended up at the Holiday Inn, which accepted all of us, the crew and Sidney and all the actors," the director said.

Jewison ultimately "learned that Sidney was right in a way" later in the evening when a group of pickup trucks arrived at the hotel. Some of the individuals, he said, were "shouting and saying that we had women in there and stuff."

RELATED: Inside Sidney Poitier and Joanna Shimkus' 46-Year Marriage: 'We Were Just Destined to Be'

"I called my key grip and I said, 'Get the biggest guys we got, the grips and electricians because I don't want anybody coming into Sidney's floor,'" Jewison recalled.

The director told Poitier that there was "a little demonstration" occurring outside, but assured the actor that things would "be fine." Jewison recalled Poitier replying, "I got a gun under my pillow and I'm going to blow away the first guy who comes through that door."

"And I thought, 'Oh my God, what are we going to do?'" the director said.

United Artists/Getty

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

One of the most memorable moments of the movie, and of Poitier's career, came when Poitier's character slaps Larry Gates, who played a plantation owner named Mr. Endicott.

In the scene, Tibbs and Gillespie (played by actor Rod Steiger) visit Endicott to question him about the death of Phillip Colbert, whom Tibbs was previously wrongfully accused of killing by his new partner when Endicott slaps Tibbs, who slaps him back.

Jewison told PEOPLE he wanted to make "everything believable" in the scene — meaning Poitier would have to actually slap his white costar.

RELATED: Sidney Poitier Remembered by Oprah Winfrey and More Stars After His Death: 'What a Landmark Actor'

Regarding the slap, Jewison recalled telling Poitier, "Got to do it fast because it's a total reaction. If you were back in Philadelphia, you wouldn't accept it." Poitier replied, "Don't worry, the scene will work, and it will be powerful."

He recounted the moment Poitier struck the actor as "such a shock" for both Gates and Steiger.

The scene was particularly powerful for Nelson Mandela, according to Jewison, who read about the late South African president's thoughts on the film. Jewison claimed Mandela took interest in the slap scene after it had been censored by authorities when he first watched the movie while imprisoned.

Mandela eventually found the clip and was allegedly inspired by the scene "because he felt this would never happen in a film in South Africa," as Jewison recalled.

RELATED: Sidney Poitier Honored by To Sir, with Love Costar Lulu: 'My Friend, My Teacher, My Inspiration'

"And I don't know whether Sidney knew that, but anyway, I think that was when someone said it was kind of a slap that was echoed around the world. And I think that's one moment in the film that people remember," the director said.

Grant, who is in her 90s, said Steiger, who won the Oscar for playing Gillespie, "was a great partner" for Poitier thanks to his portrayal of Southern policeman from the civil rights era.

"That was the love relationship in the movie," the actress told PEOPLE. "He started off putting him down, and ending up really respecting him and caring about him. That was the sense of the film."

Perhaps the most iconic line in the movie is "They call me Mister Tibbs!" which Poitier shouts after Steiger's character, police chief Bill Gillespie, utters a slur.

United Artists/Getty

RELATED: Watch Sidney Poitier Reflect on His Legacy at 2002 Oscars: 'No Route Had Been Established for' Me

Jewison said he will always remember Black viewers' reaction to the scene during one particular screening. "I'll never forget when the film was shown to a Black audience in New York, the only time I saw it with a Black audience and they cheered. They cheered when he said that," the director said.

"It was important," he continued. "And he [Poitier] became an important symbol, I think. And most Black actors who came after him really stood on his shoulders."

Grant, who played widow Mrs. Colbert in the film, also commended Poitier for helping uplift the Black community with his artistry.

"He had a sense of morality that went through all of his films, and if you see the fellow Black actors who honored him, it was for being the first, for being the first to break the white bar," she said.

RELATED: Halle Berry Remembers Her 'Decades of Friendship' with Sidney Poitier: 'True Measure of a Man'

"Sidney just was a hero. The first Black hero. The first," the actress added. "He was the first Black hero in the film world."

"I've lost a friend and America's lost a symbol in a way," said Jewison, "because Sidney was so highly respected."