A Shazam for People: Clearview’s AI App Was a Hit Among the Rich and Powerful
- Oops!Something went wrong.Please try again later.
When I first broke the story of Clearview AI in The New York Times in January 2020, people were shocked. Some were horrified. The tiny start-up had scraped 3 billion faces from Facebook, YouTube, Venmo and millions of other websites without anyone’s consent to build a groundbreaking facial recognition app. Upload a stranger’s photo to Clearview’s app and it reveals online photos of them, with links to where they can be found. Thousands of police departments around the world were secretly using the app but no one seemed to know about it.
As I dug further into Clearview AI for my new book, Your Face Belongs to Us, I discovered that in fact, some very wealthy and powerful people were among the first to know it existed. Billionaires, Silicon Valley investors, and a few high-wattage celebrities. They all had this superpower on their phone, allowing them to put a name to a face and dig up online photos of someone that the person might not even realize were online. Clearview’s database now has 30 billion faces in it. If we stay on our current path — and American lawmakers don’t pass stronger privacy laws — this may be a kind of app that we all have on our phones: a Shazam for people.
***
More from Rolling Stone
Trans Boxers Are Stepping Into the Ring. Will the Sport Let Them Stay?
George R.R. Martin, John Grisham Lead Authors Guild Class Action Lawsuit Against ChatGPT
Jimmy Jam, Terry Lewis, Desmond Child To Meet With Congressmen to Push for Legislation on AI
VENTURE CAPITALIST DAVID SCALZO came across Clearview AI in the summer of 2018. In his midforties, Scalzo had a tech background but had made a career in finance, working at a series of big-name investment banks on Wall Street, including Bear Stearns before it imploded during the 2008 recession. He now had his own small investment firm, Kirenaga Partners. Unlike the big Silicon Valley players, it didn’t have billions of dollars to throw around. It had a smaller stack of chips and had to place them wisely. Scalzo’s approach was to find a fresh tech startup with a working product and to write a check for $1 million or less, enough to give the founders a little runway to take off from but not enough to coast.
Scalzo had been chatting with an insurance broker about startups, and she mentioned one doing facial recognition. He asked for an introduction and soon had a coffee date scheduled with Richard Schwartz and Hoan Ton-That at the Algonquin Hotel near Times Square.
They gave Scalzo the pitch: Clearview AI could do instant facial identification of hundreds of millions of faces. The three chatted for an hour and a half, and in the following weeks, about what the company wanted to do and why Scalzo should invest in them. Peter Thiel had given them $200,000 the year before, and now they were trying to cobble together a million dollars more to keep going.
They’d managed to find a few customers, all private businesses. They were advertising three product lines: Clearview AI Search, a background-checking tool; Clearview AI Camera, which could sound an alert when a criminal entered a hotel, bank, or store; and Clearview AI Check-In, a building-screening system that could verify people’s identities “while simultaneously assessing them according to risk factors such as criminality, violence and terrorism.”
One bank had used their tool to screen attendees for a shareholder meeting. Another wanted to use it to identify high-net-worth customers. A real estate firm wanted text alerts whenever someone on a building’s “no-fly list” walked into the lobby. A hotel was interested in developing a seamless check-in experience. Nothing was firm yet, but Clearview was small with low overhead.
Though a tool that could identify almost anyone had obvious security applications, they didn’t bring up law enforcement as a target market. They were interested only in private companies; some of their promotional material quoted an unnamed bank security executive as saying, “I have plenty of metal detectors. Clearview is like a mental detector.”
They wanted their service to be cheap. A few thousand dollars a month? Sixty thousand per year? They wanted to undercut the competition, but they also wanted people to know that they had done something that their competitors hadn’t: They had collected headshots of people from across the web, and their algorithm was a thing of beauty with world-class accuracy. Their facial recognition worked under poorly lit conditions. It could identify a person even if he donned a hat, put on glasses, and sprouted facial hair. It could identify a person in a crowd or in the background of someone else’s photo. The person could be looking away from the camera. It could tell sisters apart.
Scalzo didn’t have to take Schwartz and Ton-That’s word for it. Sitting in the lobby of the hotel, Ton-That got out his iPhone, took Scalzo’s photo, and then ran it through the app. It found two photos of Scalzo — his headshots on LinkedIn and on the Kirenaga Partners website. Scalzo fell in love with the technology on the spot.
Schwartz and Ton-That didn’t just demo the future for Scalzo; they gave it to him free of charge. It was part of the investment pitch: “Here’s the app. Go use it.”
SCALZO WAS ONE of many backers, potential investors, and friends of the company to get free access to the private superpower, with no conditions set for how it could be wielded. Though its existence remained a secret to the public at large, it became a party trick for some of the wealthiest and most powerful people in the world. They used it as a conversation starter on dates, an identification tool for business gatherings, and an advantage at poker tables with strangers. As the science fiction writer William Gibson once observed, “The future is already here—it’s just not very evenly distributed.”
In January 2019, Schwartz and Ton-That flew across the country to California to make their pitch to the deep-pocketed investors of Silicon Valley. They met with Sequoia Capital, the firm Ton-That considered the Harvard of the venture capital world. Desperate for the validation that would come with a Sequoia investment, he gave the app to firm employees, who would go on to run hundreds of searches.
The firm’s managing partner, a billionaire named Doug Leone, was an especially avid fan, but when he failed to invest, Ton-That cut him off. In March, Leone texted Ton-That asking how to reactivate his account. “Term sheet,” Ton-That replied. Leone ignored the request for an investment but said he’d pay for the app, explaining that he used it for work, to remember people’s names when he traveled to other offices.
A month later, Leone texted again, saying he needed the app for an upcoming conference in Sun Valley, dubbed “Billionaire Summer Camp” by the press. “There are a lot of ceos and only first name tags,” Leone wrote. Ton-That gave Leone access again, still hopeful to get into his version of Harvard, but all he got was an email from a friend of Leone’s who also wanted to use Clearview. His name was Joe, and his email, saying that he might invest, was full of typos. Ton-That forwarded the message to Schwartz, expressing annoyance that Leone not only wasn’t investing himself but was telling other people about the app. Schwartz immediately called Ton-That.
“That’s Joe Montana,” Schwartz told him. “Who’s that?” Ton-That asked.
When Ton-That eventually met the legendary quarterback, he got a photo with him: Ton-That dressed in white and Montana in black, his iPhone tiny in his immense hand. Montana got access to the app but did not become an investor.
John Catsimatidis, a New York billionaire in his sixties with real estate, energy, and media holdings, but most famous for owning the grocery chain Gristedes, tested out Clearview in one of his East Side stores. His security team wanted to keep out repeat shoplifters. “People were stealing our Häagen-Dazs. It was a big problem,” he later said.
Gristedes didn’t adopt Clearview in his stores, but Catsimatidis used the app one Tuesday evening while he was having dinner at Cipriani, an upscale Italian restaurant in SoHo. His daughter, Andrea, a striking blonde who favors tight dresses and plunging necklines, walked into the restaurant on the arm of a man her father didn’t recognize. Catsimatidis asked a waiter to take a photo of the couple and then uploaded it to the app. Within seconds, it identified the mystery suitor as a venture capitalist from San Francisco. “I wanted to make sure he wasn’t a charlatan,” Catsimatidis said.
He then texted the man’s bio to his daughter. Andrea and her date were surprised but thought it was “hilarious.”
Catsimatidis had twenty-eight thousand contacts in his iPhone. He joked with the Clearview team that they needed to make a version of the software that came in the form of glasses, so that he could identify people he was supposed to know and not embarrass himself. He didn’t invest in the company, but he told them he would buy those glasses if they ever made them.
Ton-That and Schwartz asked people using the app to keep it quiet because the company was in stealth mode, but not everyone complied. Ashton Kutcher spilled the beans when he appeared on a widely watched YouTube series called Hot Ones, in which famous guests are interviewed while eating spicy chicken wings. The host asked Kutcher whether in the future “privacy will be the new celebrity.” In other words, would it eventually be harder to be an unknown than a known-by-all? Yes, Kutcher said.
“I have an app in my phone in my pocket right now. It’s like a beta app,” he said midway through the show. “It’s a facial recognition app. I can hold it up to anybody’s face here and, like, find exactly who you are, what internet accounts you’re on, what they look like. It’s terrifying.”
Ton-That was annoyed. He had originally pitched the app to Kutcher to aid the work his nonprofit, Thorn, did to prevent the sexual exploitation of children. “You almost blew our cover,” Ton-That told him.
“I didn’t say the company’s name,” Kutcher sheepishly replied, according to Ton-That.
Kutcher said he used the app only a handful of times, and that he found it “scary.” That wasn’t a bad thing; in fact it was “usually a good signal for an early-stage company,” he said. After the Hot Ones episode, Kutcher set up a due diligence call with Ton-That, but he came away from it unwilling to work with or invest in the company. He was put off both by the company’s existing investors and something in Ton-That’s past that “didn’t sit right” with him, though he couldn’t later recall what it had been.
Clearview’s biggest problem was its misfit founders. Ton-That had a couple of liabilities: the Gawker articles about his being a hacker who had created a “worm” and online photos of him in a MAGA hat hanging out with the Trump crowd — just the sort of photos a tool like his was designed to unearth. Left-leaning investors were put off. And Schwartz, nearing sixty with the fashion sense of an ex-bureaucrat, wasn’t the kind of slick startup entrepreneur that investors felt they could trust. Investors who did due diligence found out that the controversial Charles Johnson had ties to the company and that some of the people that Ton-That had hired to work on the technology and scrape faces were from China, setting off national security concerns. Plus, the app was plagued with “regulatory uncertainty,” meaning it would almost certainly face lawsuits in its future.
Investors kept turning them down. “In my mind, this thing is technologically derisked. That is just rare. It’s built,” Ton-That said later, describing his exasperation. “This thing is freaking awesome. It’s so cool. It works. It’s magical.”
Courting investors was a lot like romantic dating. Ton-That and Schwartz met person after person after person, talking up their talents, handing out their magical tool, each time hoping it would work out. But only rarely was there a spark.
With Scalzo, however, there was serious chemistry. Scalzo experimented with the app, trying it on his kids, his friends, whomever he wanted, to find out how well it worked. It would turn up their photos, or photos of people who looked a lot like them, along with links to where the photos had come from. He was impressed not just by the novelty of a global search engine for faces but by how easy it was to use. He knew it would only get better as the company indexed more of the internet and gathered more photos.
He knew it was going to scare people, but he was convinced that they would come around. “Information is power. If we give information to individuals, then they can use their God-given talents in whatever way they choose,” he would later say.
They might use them in nefarious ways, but Scalzo was an optimist. He wanted to get the app into the hands of the “moms of America,” and everyone else, so that it would become as common to “Clearview a face” as it was to google someone’s name. He believed that facial recognition would help people connect in the real world as easily as they were able to connect online. The possible use cases seemed endless to Scalzo, and the duo was trying to raise only a million dollars for the seed round, at a very reasonable valuation of $10 million — meaning it was a deal that could pay off for Scalzo in a big way if things went well. He decided to invest $200,000, a small enough amount that he didn’t think it required much due diligence. The product worked; that was all he really cared about.
Scalzo knew the company was navigating uncharted waters and that there would be legal challenges. Clearview had created biometric information for millions of people without their consent and scraped photos from many, many websites, including LinkedIn and Facebook, two powerful companies that had sued over similar behavior in the past.
But Scalzo thought that the company’s odds were good and that Ton-That and Schwartz were ready for the inevitable backlash when the wider public discovered what they’d done. He saw their push into a legal gray zone as a business advantage. If Schwartz and Ton-That succeeded in commercializing their universal facial recognition app, they’d have the jump on more risk-averse competitors.
CLEARVIEW COBBLED TOGETHER more than a million dollars from a random collective that included Scalzo, some New York–based lawyers and executives in retail and real estate, Ton-That’s old mentor Naval Ravikant, a libertarian investor in California named Joseph Malchow, and Hal Lambert, a Texan famed for starting an exchange-traded fund called MAGA ETF that invested only in companies aligned with Republican beliefs.
Lambert liked to demonstrate Clearview to his friends, who would inevitably ask how they could get it. “You can’t,” he would tell them. At a dinner one night, he showed the app to the Kentucky Republican senator Rand Paul, a staunch privacy defender who had pushed back against government surveillance bills in Congress. Paul was impressed, and relieved to see an American company building technology to rival that being created in China and Russia, but raised questions about the privacy implications.
A month or so after Scalzo invested, he met up again with Ton-That and Schwartz for a celebratory lunch at Benjamin Steakhouse, a pricey restaurant in Midtown Manhattan, just a few blocks away from the Algonquin. They brought along Charles Johnson. It was around the same time that Johnson had found out that his partners had tried to cut him out of Clearview, but if there were any tension among the three men, Scalzo did not pick up on it. Ton-That introduced the red-headed Johnson as another investor in Clearview, though it was Scalzo who picked up the tab for lunch.
Scalzo found Johnson interesting and “wicked smart”; he had insightful, well-researched theories about what society would look like as technology and science accelerated, as computers got more powerful, and as more data could be mined from people’s DNA, brains, and digital trails. Johnson talked about technology as a global battle for dominance and said that Clearview was going to help the United States win the face race so that it could identify spies and those who bore the country and its citizens ill will. He claimed to have government and “agency” connections, hinting at ties to the CIA, which he said had the power to choose the people and companies “they want to win.”
Johnson was sharp but also a little scary to Scalzo, who would come to think of him as the character Syndrome from the Pixar movie The Incredibles. “A little bit of an evil genius,” Scalzo said of Johnson. “And you’re never quite sure what is true.” He couldn’t know how apt the comparison would turn out to be. When Johnson’s relationship with Ton-That and Schwartz soured, he, like Syndrome, would wreak havoc by sharing the company’s secrets with a journalist — me. But that was years away. At that point, the company was still a well-kept secret. In the fall of 2019, Scalzo was invited to a fundraiser for a Republican running for a Senate seat in Kansas. Peter Thiel was co-hosting it with the conservative media pundit Ann Coulter at his Park Avenue apartment in Manhattan. Scalzo sidled up to Thiel and started talking about the facial recognition app that they had both invested in. Scalzo took a photo of Thiel and other people at the party and then showed him the results: a collection of their photos from elsewhere on the web.
“I’m invested in that?” Thiel asked, according to Scalzo. “That’s cool.”
“He played it a little coy,” Scalzo said. “He’s made a thousand angel investments.”
Peter Thiel had provided the seed money that allowed the company to sprout and attract other benefactors, such as Scalzo. Without that initial $200,000 from Thiel, the tool might not exist. Yet, if Scalzo’s memory is correct, Thiel was barely aware it existed. It was one plant struggling to thrive in a vast field of startups he’d cultivated.
Scalzo kept a somewhat closer watch on his own crops. He checked in occasionally to see how business was going. And soon it started going quite well. Clearview had finally discovered the right customer: the NYPD.
From the book YOUR FACE BELONGS TO US: A Secretive Startup’s Quest to End Privacy as We Know It by Kashmir Hill. Copyright © 2023 by Kashmir Hill. Published by Random House, an imprint and division of Penguin Random House LLC. Book available for sale here.
Best of Rolling Stone
