Somewhere You Feel Free: behind the making of Tom Petty’s Wildflowers

The album Tom Petty considered to be the best work of his career chronicled the most tumultuous period of his life. Between the summer of 1992 and the spring of 94, the stretch in which he recorded his classic album Wildflowers, Petty’s 22-year marriage to the mother of his two children fell apart, he fired the drummer with whom he had worked since his Heartbreakers band began nearly three decades before, and he left both the record company for which he recorded all of his hits and the producer who shaped some of his biggest ones. “He was blowing up every aspect of his life,” said Mary Wharton, who directed a new documentary set in that dense era, titled Tom Petty, Somewhere You Feel Free: The Making of Wildflowers. “From his personal life to his business life to his creative life, Tom was trying to figure out how to put things back together in a way that made sense to him in that moment.”

Related: Mike Campbell on life after Tom Petty: 'I think I found my own voice'

Despite the frustration, controversy and agony of the time, Wildflowers also represented a fount of opportunity, a blank check for the future. For the most part, the film focuses on that aspect, leaving some of the more troubling issues either implicit, glossed-over or denied. Small wonder the film is as interesting for what it doesn’t include as what it does. Some of that has to do with what Petty revealed, and didn’t reveal, about his life at the time; some probably came as a result of a life that ended too soon and without warning. On 2 October 2017, Petty died at the age of 66 from what was ruled an accidental overdose of opioids, sedatives and an anti-depressant. An official statement by the family at the time stated that the musician had been taking that mix of medications to numb the escalating pain of emphysema, issues with his knees and a badly fractured hip.

In the time since, his estate has been combing through the music and film that remains in the vault. Lately, they have concentrated on Wildflowers, drawn by its revered status with fans, critics and Petty himself. Last year, they sanctioned a four-CD box set titled Wildflowers and the Rest, which restored 10 songs excised from the original release, along with demos, alternative takes and live recordings of material from the original sessions performed over the next two decades. Wharton’s film draws on two primary sources – footage shot in the 90s during the Wildflowers recording sessions and fresh interviews she conducted with the Heartbreakers’ Mike Campbell and Benmont Tench, as well as with the album’s producer, Rick Rubin, and Petty’s daughter, Adria. The vintage footage, shot by the longtime Petty chronicler Martyn Atkins, ran four hours. Only 11 minutes of it has been seen before, as part of a press kit created at the time. The new documentary project began, said Wharton, after Petty’s daughter “found these cans of film in Warner Bros’ vault that she didn’t know existed. She felt something should be done with them,” she said.

Both the vintage footage and the new interviews focus on the excitement Petty felt in this new chapter in his life. In the songs he was writing, he found a fresh way to express himself by mining his own life for material. “The songs reflected what was going on in his personal life more closely than any other project he had done,” Wharton said.

In that sense, Wildflowers represents Petty’s first true solo album though his literal first, Full Moon Fever, appeared five years earlier. “The songs on Full Moon Fever were classic Tom Petty songs in that they present characters that aren’t necessarily him,” the director said.

Part of the impetus to go deeper came from Petty’s entry into middle age, having just turned 44. “He was trying to figure out how to be an adult rock’n’roll star,” Wharton said. “How do you have some dignity in this very childish career of rock star?”

Petty’s quest for creative growth also led him to vary the musicians he worked with. Originally, he intended to only hire musicians outside the Heartbreakers. While he did bring in other players for some parts – including Ringo Starr, the Beach Boys’ Carl Wilson and Michael Kamen for the tasteful orchestrations – he wound up using three-fifths of the Heartbreakers on every track, including guitarist Campbell, keyboardist Tench and bassist Howie Epstein. “He couldn’t quit them,” Wharton said. “If you’re putting together a dream team rock’‘n’roll band, those are the guys you want. Also, Tom had such a long and deep collaborative relationship with Mike Campbell. They worked together their entire lives.”

In developing the music, and looking towards his creative future, Petty came to believe that the Heartbreakers’ drummer, Stan Lynch, didn’t have the right feel. So he made the difficult decision to fire him and bring in Steve Ferrone, who remained the band’s drummer until Petty’s death. In interviews for the film, various speakers make clear how crucial Ferrone’s sharp attack was to songs on Wildflowers like You Don’t Know How It Feels. Wharton didn’t contact Lynch for his view of things. But, she said, “we did make sure Stan was presented in a respectful way. I think that Mike Campbell had a clear-eyed view of the situation so we went with his explanation.”

The lyrics to one song on the album directly address issues with another Heartbreaker – bassist Epstein – though that wasn’t obvious at the time. In Don’t Fade on Me, Petty reaches out to someone who has lost their way. It was later revealed that Epstein was addicted to heroin at the time. (He died of an overdose of that drug in 2003.) “Tom clearly loves the person he’s writing about,” Wharton said. “The song is a beautiful way to throw a life raft to someone who’s in trouble.”

At the time, Petty’s daughter interpreted the song as a pained address to her mother. In the film, she says that as soon as she heard the album as a kid she felt that the whole thing reflected her parents’ impending break-up. While Petty acknowledged as much in interviews when he was promoting his next album, Echo, which openly covered the divorce, none of that was known by outsiders at the time. As a result, he doesn’t address it in the vintage footage seen in the film. Instead, he describes, with wonder, how some songs for the album came to him unconsciously, including the title track. A shimmering ballad, the lyrics to the song Wildflowers encourage a loved one to move on and find joy on their own. In that sense, it recalls Neil Young’s classic ballads Birds and I Believe in You, two of the most sensitive rejection songs ever written. It’s both fascinating and strange to watch Petty deflect his true emotions when discussing the song in the vintage footage. “Maybe he was afraid to express those feelings even to himself at the time,” Wharton said. “It’s him not wanting to fully accept the ramifications of ending a long-term relationship that involves kids and financial implications.”

In both the old and new footage, the people interviewed are far more direct in discussing the dramatic changes the production style brought to the project. On Petty’s two previous releases he worked with Jeff Lynne, who has an elaborate and specific production sound. Hiring Rick Rubin for Wildflowers allowed him to work more spontaneously and organically. At one point in the new footage, Rubin says Wildflowers scared Petty because he felt he might never be able to create something as free-flowing again. The speakers in the film are equally frank about Petty’s contempt for his record company, MCA. While he was still under contract to them, he signed a new deal with Warner Brothers in secret.

“Tom publicly battled with them for years over everything from the price they sold his record for to the contract he signed when he was young and stupid,” Wharton said. “He never felt like they appreciated his artistry. When he turned in the Full Moon Fever album to MCA they first rejected it. When Tom played it for Mo Ostin [head of Warner Bros] he said ‘these people are crazy. This is a hit record.’ That was the beginning of his relationship with Mo.”

Yet, when Petty first submitted the sprawl of songs cut during Wildflowers sessions to Ostin, under the assumption it would result in a double album, the Warners exec shot that idea down as uncommercial. Regardless, the slimmed-down result clocked in at 63 minutes – the length of a double set in the LP era. Released in that form, Wildflowers became a huge success, selling over 3m copies in the US alone. Its songs form a perfect arc, telling a coherent story of possibility and loss. It culminates in the song Wake Up Time, which finds the artist telling himself he has to face the hard facts of his life and make changes despite the consequences.

As it turned out, more hard times lay ahead for Petty. Shortly after Wildflowers appeared, he started to use heroin. Wharton said she found footage from that time in which “Tom looks different,” she said. “In the Wildflowers footage, he’s so clear-eyed. In the other footage, he’s not. I interpreted that as ‘oh, this is when he started using heroin.’”

The director said she chose not to use that footage so her film could focus wholly on the Wildflowers period. In the years since, the album has only grown in stature. Before his death, Petty toyed with the idea of centering a whole tour around its songs. To Wharton, the legacy of Wildflowers boils down to “the message Tom was trying to convey to himself at the time – that we all belong somewhere we feel free.”

• Tom Petty, Somewhere You Feel Free will be released worldwide on 11 November on Tom Petty’s official YouTube channel