Gensler’s warning: Unchecked AI could spark future financial meltdown

A growing dependence on artificial intelligence could pose a danger to the U.S. financial system, and regulators need to rethink their siloed approach to rulemaking to minimize the risk, Securities and Exchange Commission Chair Gary Gensler said.

The SEC and other Washington regulators are grappling with the rise of powerful AI systems that can augment or replace human decision-making. But in the financial industry — where banks, brokers and investment firms oversee trillions of dollars — the potential for economic disaster is especially acute, Gensler said.

In aninterview on the POLITICO Tech podcast, Gensler described a doomsday scenario in which many of the country’s big financial institutions rely on a small number of AI algorithms to make investment decisions — creating a vulnerability that regulators could miss by focusing on only a sliver of the sector.

“We have set up a lot of our systems of oversight and rules around regulating individual entities or activities, whether it's bank regulators, insurance regulators, securities regulators, commodities regulators,” Gensler said. His approach, he said, involves “thinking about [AI] across all the entities — are they potentially all using the same base model or base data?”

Listen to the full interview on POLITICO Tech, available on Apple, Spotify and Simplecast. 

That kind of concentration already exists in other areas like cloud computing, Gensler said, with many financial institutions getting services from a handful of tech giants — Microsoft, Google and Amazon, which are emerging as dominant players in the AI market.

But AI systems could be far more consequential, said Gensler, who chaired the U.S. Commodity Futures Trading Commission during the Obama administration. AI systems can make judgment errors or generate inaccurate information known as “hallucinations.” If that happens on a large scale, it could wreak havoc on the market.

“I would be quite surprised if in the next 10 or 20 years a financial crisis happens and there wasn't somewhere in the mix some overreliance on one single data set or single base model somewhere,” Gensler told POLITICO Tech.

Wall Street firms have been dabbling in AI for years, but the industry’s adoption of the technology has taken on new speed lately. Financial regulators, in turn, have been racing to understand AI and the technology’s potential to upend the markets and economy. 

Gensler has been examining the intersection between finance and AI since before President Joe Biden tapped him to head the SEC in early 2021. During his 40 years in finance, including stints at Goldman Sachs and the Treasury Department, Gensler said he has witnessed at least five crises. The challenges posed by AI are “familiar,” he said, but have a new twist: “It's about how to promote a diverse use of models and data, and that's not an easy challenge to solve.”

At the SEC, Gensler has cracked down on companies that the agency says make misleading claims about their use of AI. On Monday, the SEC reached a $400,000 settlement with two investment advisory firms that had claimed to be using the technology in ways they allegedly were not.

The commission is also crafting regulations for how brokers and investment advisers use AI and other so-called predictive data analytics when interacting with customers. Wall Street has criticized that rulemaking effort as overly broad, but Gensler said AI must put the interests of clients first — a principle that has long applied to human advisers.

“There could be inherent conflicts if … the investment robo-adviser was skewing the advice they give you because they're optimizing for the profits or revenues or other interests of the firm,” Gensler said. “That's all we're trying to address with that.”