Bank stress tests: Where they add value, and where they add risk
I remember a conversation I had during my first month at the Office of the Comptroller of the Currency (OCC) – the primary regulator of national banks – in 2010. I had just joined the OCC as a data analyst. At the time I wasn’t a kid fresh out of school, like many other analysts there, but instead I had spent the prior six years on trading desks making markets in bonds and rate derivatives, most recently at Wachovia Bank, which had failed spectacularly.
As a new analyst but one with some pretty meaningful personal experience, including trading during the financial crisis and watching a mega bank implode from the inside, I had access to lifetime examiners who often wanted to swap stories. One day I asked a seasoned large bank regulator this simple question: How, in his opinion, did everyone fail to see the coming crisis?
My colleague answered candidly, “Regulators can stack up a bunch of banks next to each other and see which one is the riskiest. What regulators can’t do is understand if everyone is making the same mistake.”
His point was that regulators have a well-worn process for rank-ordering riskiness among the firms in their orbit. They can tell you which banks appear to be taking risk more aggressively than others, and which banks appear to be taking on less. But what they don’t do well is see the system as a whole, even though people often assume that they can.
Many regulatory and legislative changes things have taken place since 2010, including a renewed discussion about macroprudential oversight. But given the glacial pace of cultural change at regulatory agencies, it does not appear that this component of bank oversight has materially changed. Regulators still use a “checklist” to look at banks vis-à-vis each other. And in today’s post-crisis environment, the regulators have plenty of political wind at their banks when they use whatever risk metric they choose to tell the “riskier” banks to get back in line.
Which brings us to the stress tests—the highly complex means of attempting to ensure capital adequacy at banks under theoretical, forward-looking adverse economic scenarios. These test results generate so many headlines you’d think they were handed directly to Moses on Mount Sinai. This has been particularly true of late, where every hint that the Fed is making changes to the tests causes stir and attention. Now some of the changes proposed recently may make sense. But let’s be honest – these tests are so complicated that they have created an entire cottage industry not just of reporters and stress tests watchers, but of consultants to run the underlying models themselves.
So let’s take a look at the tests. Set aside for the moment that financial models have an uncanny knack of producing results in-line with the management or regulators’ expectations. Unfortunately these stress tests – initially intended to help give an accurate birds’ eye view of the entire system and hence calm the markets – may be evolving into simply another means of rank-ordering bank risk while losing the forest for the trees.
Just look at how we analyze the results. The largest banks are lined-up next to each other and anyone who fails is told to shed risk and get back into lock-step with the others. This is called horizontal analysis, and it is nothing new. It gives the appearance of safety. But it doesn’t work very well in a crisis.
Regulators, the Financial Stability Oversight Council (FSOC), Congress, and investors all benefit from seeing how banks would perform under a set of economic scenarios given two simplifying assumptions: the same caliber of management at every firm, and that historical market correlations will hold in the future. And if stress tests are simply a tool to require more capital across the industry, then that’s one thing.
But what stress tests won’t tell you is how banks will actually perform during the next economic downturn. And the belief that these tests can predict future outcomes is dangerous. To be effective, we need to appreciate that the structural weakness in horizontal analysis is that if all of the banks are making the same faulty assumptions—say, for example, that housing prices always go up or that markets will respond to the next economic hiccup the way they did last time—the real life results will be very different than those predicted by the tests.
Worse, these stress tests—given the public attention they generate—could actually push banks to take on similar risks to each other. This, in turn, can dampen economic growth, turn banks into public utilities, and, ultimately, push risk taking out elsewhere into the system. All of these trends can be seen today. Witness the massive growth in the non-bank (and non-stress-tested) financial sphere.
If we really want to analyze systemic risk in a more meaningful way, we should take three steps.
First, regulatory heads need to address the entrenched operating structure of their agencies, specifically their reliance on horizontal analysis of the firms they oversee. Just because a bank looks better than its peer doesn’t mean you’ve eliminated risk.
Second, regulators need to actually follow financial system risk as it moves from regulated to unregulated space. We haven’t made the system safer if we have just shifted risk elsewhere.
Lastly, policy-makers must avoid the trap of thinking a stress test gives an accurate assessment of how future crises will play out. This sense of false confidence could be dangerous.
If we rely on these stress tests as the final word on bank safety, we risk repeating the structural mistake that has plagued regulators in the past—regulating banks against each other instead of thinking holistically about systemic risk—just in a much fancier way.
Michael Bright is a director at the Milken Institute’s Center for Financial Markets (CFM). Prior to joining the Institute, Bright worked at BlackRock, the United States Senate Office of Senator Bob Corker (R-TN), and the Office of the Comptroller of the Currency (OCC). Bright was an interest rate derivatives trader for 6 years before coming to Washington.
