Washington is aflame in the aftermath of the Oct. 5 decision by the OPEC+ oil cartel to cut oil production for the foreseeable future. OPEC+ includes the 13 members of OPEC and 11 other nations, most notably Russia. Saudi Arabia is the biggest energy producer among the group, and its de facto leader. Russia is the second-biggest energy producer in the group.

Saudi Arabia seems to be doing Russia’s bidding by keeping oil markets tight, and pushing prices up. Oil exports are the Russian government’s biggest source of revenue, and a crucial source of financing for its diabolical war in Ukraine. Most of the Western world supports Ukraine, including tough sanctions meant to strangle Russia’s economy. President Biden has asked Saudi Arabia to pump more oil, to stabilize energy markets amid wartime disruptions. The Saudi rebuke is a win for Russian President Vladimir Putin and a political embarrassment for Biden just ahead of midterm elections. It also means consumers in all the countries supporting Ukraine will pay more for gasoline and other oil products during the coming months.

Biden has said there will be “consequences for what they’ve done with Russia,” without spelling out what those might be. But Saudi Arabia, like it or not, retains tons of leverage over the US and world energy markets, even as many nations move to curtail the use of fossil fuels and boost renewables.

“We are still going to be having to make asks of these countries when we need more oil,” Helima Croft, head of global commodity strategy for RBC Capital Markets, said at an Oct. 12 Columbia University energy conference. “Who’s going to be sitting on spare capacity? It’s going to be a small number of Gulf producers and a handful of national oil companies that continue to invest on a really big scale. That means we’re going to have to continue to have dialogue with these countries.”

Saudi Arabia has been a US ally for decades, but outraged Washington heavyweights now feel spurned—and vindictive. Democratic Sen. Joe Manchin of West Virginia, who chairs the Senate Committee on Energy and Natural Resources, wrote Biden an open letter calling the OPEC move “reckless” and demanding that the US ramp up its own energy production to counter OPEC. Senator Bob Menendez, a Democrat who chairs the Senate Foreign Relations Committee, wants to “immediately freeze all aspects” of US-Saudi cooperation, including US arms sales to the kingdom. There’s growing support in Congress, among both parties, for “NOPEC” legislation that would give the US Department of Justice more tools for addressing OPEC price hikes.

Saudi Arabia, for its part, issued an unusual rebuttal on Oct. 13, saying the OPEC+ production cut was based on economics, not political support for Russia. Saudi Arabia also said the US-Saudi relationship “is a strategic one that serves the common good interest of both countries.” But the production cut stands.

The slow shift away from fossil fuels

Many Americans hear that the United States is the world’s largest oil and gas producer—which is true—and think there must be something wrong with government policy if we can’t keep domestic energy prices low. For the most part, however, the problem is not government policy. It’s the rapidly changing nature of world energy markets and the risks investors face if they make the wrong bet.

[Follow Rick Newman on Twitter, sign up for his newsletter or sound off.]

The world is shifting away from fossil fuels, toward renewables, and government policy may only affect how quickly that happens. Consumers are demanding this shift, as the ravages of climate change become more apparent. Innovative companies such as Tesla are giving consumers what they want and earning billions. Many businesses, sensing the next big thing, aim to follow. Government incentives, such as those in the recently passed Inflation Reduction Act, will be a powerful force drawing private capital into renewables and speeding the pace of innovation.

Fossil fuels, meanwhile, are a still-profitable business, but one that seems destined to decline over the next 20 or 30 years. We will need oil and natural gas for a long time to come. But less and less of it. That makes a lousy case for big investments that could expand supply, such as new wells or refineries.

Americans tend to forget that the US energy industry is largely a private-sector business driven by the profit motive and capitalist dynamics. Energy firms have shareholders and investors and contractual obligations, which means they have to deploy capital where it gets the highest returns. They don't produce extra energy just because consumers or politicians demand it. One of the biggest concerns at fossil-fuel companies now is the risk of “stranded assets”—big projects that could rapidly lose value as the market for fossil fuels dries up. In that type of environment, nobody wants to invest in assets that could become obsolete before they’ve generated a return.

The advantage of state-run companies

This is where the OPEC+ nations have an advantage. Those nations typically have nationalized oil companies that are basically run by the government. The biggest is Saudi Aramco in Saudi Arabia, followed by Rosneft in Russia. Other big producers include the national oil companies of Kuwait, Iraq, Iran, Qatar, the UAE, Brazil and Mexico. China doesn’t export much oil, but it has two giant state-run companies in the oil and natural gas business.

State-run energy companies don’t have to worry about shareholder returns, and sometimes they don't even need to worry about profits. They operate, instead, as levers of government policy, and can make investments to expand capacity if that aligns with government goals. In many cases, it does. Saudi Arabia is diversifying its economy beyond energy, yet Saudi Aramco said earlier this year it will boost capital expenditures by up to $50 billion this year, and by similar amounts until 2025 or 2026. That could make Saudi Arabia an even more important “swing producer” able to dial output up or down as the government wishes.

In the United States, high prices are swelling profits at energy firms, just as at Aramco. But oil companies are cautious about making investments, and US production is creeping up only gradually. Nobody in the oil and gas industry wants another boom-bust cycle driven by excess supply that ultimately tanks prices. There’s more excitement about cashing in on booming demand for renewables.

There are ways Biden could impose some pain on Saudi Arabia. Democratic Sen. Chris Murphy of Connecticut wants Biden to move US Patriot air-defense missiles that are currently in Saudi Arabia to Ukraine or to NATO allies in Europe, and to redirect a forthcoming sale of air-to-air missiles from Saudi Arabia to Ukraine, as well. The Saudis face a militant foe across the Persian Gulf—Iran—and could feel vulnerable without US weaponry. Of course, the Saudis could also retaliate by cutting oil supplies even more.

Manchin, in his letter to Biden, listed a series of things Biden could do, some involving Congressional legislation, to encourage more US oil and natural gas production: fast-track permitting for pipelines and other types of infrastructure, speed oil and gas leasing, fully staff all the federal agencies with energy oversight. Even if he did all that however, it wouldn’t change much about the financial risk of investing in a declining industry.

What might really ease US reliance on non-democratic nations such as Saudi Arabia is a sharp cutback in fossil fuel use. That’s coming, as more people drive electric cars and install solar roofs, and efficiency measures get better. But the transition to renewables will take a long time and face many barriers, such as the difficulty building high-voltage transmission lines able to quickly move power around the country. For years to come, we will still need energy from producers we don’t like who face fewer constraints that we do here at home.

Rick Newman is a senior columnist for Yahoo Finance. Follow him on Twitter at @rickjnewman

Click here for politics news related to business and money

Read the latest financial and business news from Yahoo Finance

Download the Yahoo Finance app for Apple or Android

Follow Yahoo Finance on Twitter, Facebook, Instagram, Flipboard, LinkedIn, and YouTube