Taylor Swift Reveals Why She Needed '10 Years of Retrospect' to Create All Too Well Short Film

VALERIE MACON/AFP via Getty

Taylor Swift is opening up about her creative process.

At the Toronto International Film Festival on Friday, Swift, 32, joined TIFF's CEO Cameron Bailey for a Q&A and a 35 mm showing of All Too Well: The Short Film.

In their talk, the Lover singer discussed why "All Too Well" was the song she chose to make into a short film. Recalling how the track was never picked by her team to be a single when it was first released, despite her love for it, Swift said, "Nobody saw the potential in it, except for the fans who loved it so much."

Looking back, Swift said that may have been a blessing in disguise. "The song was so tough because it was about something that, at that point, was very current for me," she said. "I would have had a really hard time performing it at the time."

"There would be no world in which I could've made a visual element to that song at that point in time," the singer added. "I needed 10 years of sort of retrospect in order to know what I would even make to tell a version of that story visually. And I'm so grateful that I was able to do that with some crazy stroke of all these different twists of fate. I can't believe it."

All Too Well: The Short Film was released in November 2021 and stars Teen Wolf's Dylan O'Brien and Stranger Things' Sadie Sink. (Sink, 20, was in attendance at the event.)

RELATED: Taylor Swift Releases All Too Well The Short Film: 'Feel Your Feelings'

Amy Sussman/Getty

For Swift, seeing her songs come to life visually is something she's always done. "It was always part of the process because when I would write a song, I would immediately start thinking of how do I want to present this on stage?" she said. "If I made a music video, what would it look like?"

Swift recalled a specific example of when she was 16 and pitched the idea of the "You Belong with Me" music video to its director. "I [would] reach out to a video director and I'd say, 'Hey, I think it would be cool if the concept was that we were like, I'm in love with my best friend in high school. And he lives across the street and there's this cheerleader who he's dating and then there's a football game. And then there's the dance and there's these signs.' "

"You come up with concepts and over time I started to take on more and more responsibility and the more responsibility I took on creatively, the happier I was," she said. "So here we are."

RELATED: Taylor Swift's 10-Minute Version of All Too Well Was Worth the Wait: Breaking Down the Lyrics

"It was a baby steps process," Swift added of directing the project. "It wasn't like I woke up one day and I was like, 'You know what I want to do? Direct.' That was never something that I was programmed to say to myself because I didn't go to film school. I've been on the set of 60-plus music videos. And I've learned a lot from that process."

Swift also noted that after directing a few music videos, including "The Man," "Cardigan" and "Willow," she felt confident in her skills to direct a short film. "That was when I thought, I feel like I could do more," she said. "And so that was when I decided to make the short film for 'All Too Well.' "

RELATED: Taylor Swift's All Too Well: The Short Film Is Eligible to Be Nominated at 2023 Oscars: Report

"One of the reasons I wanted to make a short film and not a music video for this song is because I've been fascinated with the dynamic of the age of the character that Sadie's playing and what a precarious age that is," she explained. "When you could fit back at your family home sort of, but you sort of don't. But you could fit in an adult cultivated apartment where they have a French press and they have all the things that adults have, but you kind of don't."

Swift also talked about the most powerful moment in the film: the fight. "I think you can tell a lot about people based on how they fight or argue," she said. "When it came time to shoot, we filmed them breaking up. We filmed them falling in love, some of the dialogue with that. But as we got closer to it, I was like, 'It's going to be the fight.' "

To make sure they aced the scene, Swift talked at length with O'Brien, 31, and Sink about their characters' intentions. "It's not about him just dropping her hand," she noted. "It's about this entire, just molecular structure where she feels out of place, and he feels unequipped to handle that."

Want to get the biggest stories from PEOPLE every weekday? Subscribe to our new podcast, PEOPLE Every Day, to get the essential celebrity, entertainment and human interest news stories Monday through Friday.

"We talked so much about this, that when we actually shot, it ended up being, I think probably 92 percent of that scene is a oner," Swift revealed.

She added that she didn't cut "until the very end" as she and her producer were just so impressed with the acting in the scene. "There was a death clutch as we're watching this because we're like, 'Oh my God, oh my God,'" she said. "We're injuring each other's arms at this point. But also trying not to make a sound. We just we're just so in awe of how much we could trust these actors with every single bit of seeing the story, the weight."

Answering questions tied to the short film, Swift also gave some hints to a favorite fan theory: the scarf. "Basically, the scarf is a metaphor, and we turned it red because red is a very important color on this album," Swift said, before cutting herself short and saying, "I'm just going to stop."

RELATED: Jake Gyllenhaal Says Taylor Swift's All Too Well 'Has Nothing to Do with Me': It's 'Her Expression'

Also in the conversation, Swift talked about her creative goals moving forward. "I think I will always want to tell human stories about human emotion," she shared after the moderator asked what other genres she'd want to dive into. "I never say never, but I can't imagine myself filming an action sequence."

"If it happens one day, honestly, that'll be funny character growth, but at this point I could see it going in a more comedic irreverent place," she admitted. "I don't always see myself telling stories about extreme guttural heartbreak at your most formative age, that really sort of debilitates you emotionally for years."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up to date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

As for directing feature films, Swift is keeping her options open. "If it were the right thing, it would be such a privilege and an honor," she said.