Jesse James: ‘I Think I’ve Paid the Price and Then Some’ [Exclusive]

Jesse James knows you hate him and he knows why. He's just not sure what more he can do to change your mind.

"I think I've paid the price and then some, you know? I don't know anybody anywhere that's had to endure what I had to endure for basically infidelity," the 42-year-old exclusively tells omg! from Yahoo!. "It's just the nature of my life. People always will want me sainted one minute, and strung up the next."

There was a time when things were different, when James was mostly known for his signature motorcycles. The Southern California native spent years learning his craft, and became well-known in the niche custom bike world thanks to his successful shop, West Coast Choppers, which churned out gleaming, one-of-a-kind two-wheelers. He even lucked out and landed his first cable reality show a decade ago, which followed James and his team as they scrambled to customize motorcycles against the clock. He was that tattooed guy on TV building bikes, and frankly, most of the world barely knew who he was (despite the fact he wed and divorced a porn star along the way). That all changed, of course, the day one of the most famous women in the world walked into his shop, the first chapter in what would become an almost unbelievable story.

Indeed, the rough-around-the-edges motorcycle man from Long Beach would go on to marry gorgeous mega-movie star Sandra Bullock, find himself rubbing elbows with showbiz's elite, and become half of a famous Hollywood odd couple. In 2010, when Bullock stepped on stage to accept her first-ever Oscar, she teared up as she paid tribute to her unlikely husband for his support. All eyes were suddenly on James. And hey, if America's sweetheart loved him, then America could, too. Until just two weeks later, when the world learned he had been cheating on beloved Bullock … and his tattoo model mistress starting talking to every media outlet she could. Despite being in the process of finalizing the adoption of a son (Louis, who Bullock went on to adopt alone), the couple soon divorced and James quickly became one of the country's biggest villains.

[Related Content: Celeb Tattoos Gone Wrong (Photos)]

"I stood up like a man and admitted my wrongs and what I did and all that did is make every s*** tabloid everywhere across the world come after me even harder," he recalls now. "I'm strong. I can take it. Everybody can take cheap shots and say I'm a horrible dad, and that I'm a Nazi, and all this other bulls***. Give me all you got. Because you know what? I know inside what kind of a person I am. I know I'm an awesome dad. I'm devoted to my work. And I work hard to take care of my family."

Even before the Bullock fallout, James says he was used to others' pre-conceived (and usually wrong) notions about him. He's been accused of being lots of things he's not, he insists, like being homophobic, an allegation that especially stings. "My older brother was gay, and looked just like me, and happened to die of HIV five years ago. Nobody ever cared to ask about that," explains James. "I think people just judge me by the way I look. That predates fame or any of that stuff. People have always been scared of me and always think I'm a bigot or a racist or a homophobe just by the way I look. I'm 6'3", and 215 pounds, [and a ] tattooed-up white boy. It fits a criminal or whatever. They don't realize."

While the public may still be stuck on his actions from years past, James has moved on with his life in his own way, ending an era by shutting down West Coast Choppers, and moving his three kids (two with first wife Karla James and one with his second wife, adult film star Janine Lindemulder) to Austin, Texas, to focus on his new venture called the Austin Speed Shop. He may be leaving Hollywood behind, but he's certainly not stepping out of the spotlight. On Monday he'll star in a reality special on the Discovery Channel called "Jesse James: Outlaw Garage," focused on both his life at the shop, and his life outside of work as a single dad.

Though all three children live with him full time and his youngest daughter, 8-year-old Sunny, doesn't have any contact with Lindemulder, who's had countless run-ins with the law in recent years, James would ideally want his ex back in Sunny's life, eventually. "Of course," he says. "I think kids should have both parents."

[Related Content: Louis Bullock Stays Sweet]

Until then, James seems content with his new life in Texas. He's 12 years sober and insists that everything that's happened has made him a stronger person and a better dad. But that doesn't mean he doesn't still have regrets.

"I think people will probably think since I'm doing well now and since I'm happy that I don't have any sorrow or regret for what I did, which couldn't be further from the truth. There'll never be a day that goes by for the rest of my life that I don't regret it," James admits. "If I could go back and change the way I did things, and not hurt people, and not do things the way I did, of course I would do it that way. I would never consciously bring pain to anyone, especially the people I love. It is something that I have to live with."

Despite the joy fatherhood brings him, another marriage and more kids just isn't in the cards. (Post-Bullock, he was engaged to reality star and tattoo artist Kat Von D, but the two broke it off last year.) "I just don't have a place for it anymore. Everything is so good with the kids, I have so much creative drive with my work and I'm connected with all my tools again on a level that I've never been," he says. "I don't want to screw up the balance and the goodness of all that. And I think the thing that doesn't fit into that equation is having a relationship."

For a guy who might be best known for infidelity, I realize at the end of our interview that James was refreshingly honest, answering every single question thoughtfully. No publicist listening in, no filter, no time limit. He gushed over his kids and his work, and seemed, well, dare I say like a nice guy. Maybe he just needs a fresh start, needs to stay off TV for a while, make low-key friends at the PTA meeting … you know, go back to being a regular guy.

"Do you think I could go back to being a regular guy?" James asks with laugh.

It's the only question of the day that doesn't get answered.

"Jesse James: Outlaw Garage" airs Monday at 10 p.m. ET/PT on the Discovery Channel.

More Features on Yahoo!:

Follow omg! on Twitter