The Intermediate-Range Nuclear Forces, or INF, Treaty has been central to limiting the kinds of nuclear weapons both countries can deploy, and without it, experts fear there will be a new nuclear arms race.
"When an agreement is so brazenly disregarded, and our security so openly threatened, we must respond," Pompeo said at the State Department.
(MORE: Trump says he's terminating a major nuclear treaty with Russia. Here's what you need to know.)
Pompeo first announced the U.S. intention to withdraw in December, giving Russia a 60-day window to come back into compliance, and that window runs out on Saturday. Withdrawal now requires an additional six-month window, according to the treaty's terms. The agreement, signed by President Ronald Reagan and Soviet leader Mikhail Gorbachev, banned ground-launched cruise missiles with a range between 310 and 3,100 miles.
While Pompeo spoke only of Russia, U.S. officials have been concerned about China's growing military prowess as China is not a party to the Cold War pact. Officials worry that puts the U.S. at a military disadvantage, although a senior administration official denied Friday that the decision was made because of the actions of any other country besides Russia.
President Donald Trump issued a statement saying Russia had violated the INF Treaty with "impunity, covertly developing and fielding a prohibited missile system that poses a direct threat to our allies and troops abroad."
The U.S. first accused Russia of violating the pact in 2014, but Russia first denied it possessed the weapon in violation and then later said that weapon didn't violate the treaty's terms.
Kremlin spokesperson Dmitry Peskov said Friday Russia regrets the U.S. decision and accused the U.S. of being "unwilling to hold any substantial talks."
But the top U.S. diplomat for arms control had met with Russia repeatedly in the last two months, saying after each meeting that Russia continued to deny its deployment and the U.S. had no other choice.
With the six-month wind-down period beginning Saturday, Russia could still return to compliance and save the treaty: "This is their final chance," a senior administration official said.
While critics have expressed alarm about a new arms race, Pompeo argued that that arms race is already underway because of Russian proliferation. Instead of being constrained by a treaty that didn't have both parties abiding by it, Pompeo said, the U.S. had no choice but to end it.
"It does no good to sign an agreement if the parties aren't going to comply with it. The piece of paper, if it's not being complied with, doesn't reduce the risk. It doesn't take down that threat to the people around the world," he told reporters.
A senior administration official later told reporters that any talk of a new arms race was "Russian propaganda" and that the U.S. was only looking at expanding its "conventional [weapons] options at this point."
Pompeo and Trump also said that U.S. allies in Europe were with them, with Trump's statement saying, "Our NATO Allies fully support us, because they understand the threat posed by Russia's violation and the risks to arms control posed by ignoring treaty violations."
NATO Secretary-General Jens Stoltenberg said the alliance "fully supports the U.S. suspension & notification of withdrawal from the Treaty."
But there are concerns about what comes next. Stoltenberg urged Russia to "use next 6 months to return to full & verifiable compliance or bear sole responsibility for its demise," but that is not likely.
Without the INF Treaty, the question becomes how will the U.S. contain Russia's missile proliferation.
Critics like Sen. Bob Menendez, the top Democrat on the Senate Foreign Relations Committee, say the Trump administration "lacks a coherent strategy to address the threat new Russian cruise missiles pose to the interests of the United States and those of our allies."
"The administration has advanced no plan to make Europe safer now, no strategy to address the fielded missile, and no agenda for mitigating a new costly arms competition with Russia," said Pranay Vaddi, a fellow at the Carnegie Endowment for International Peace.