‘Extraordinarily reasonable,” President Obama called it in an interview aired on Sunday, referring to the multiparty deal being negotiated on Iran’s nuclear-weapons program. When the talks began, Mr. Obama said it was “unacceptable” for Iran to acquire nuclear weapons. Now it isn’t clear whether that is actually his view.

On Capitol Hill, distrust of the president is intense, fueled by resentment that he doesn’t intend to submit the nuclear deal for congressional approval. Forty-seven Republican senators on Monday sent a letter to Tehran explaining that a future U.S. president or Congress could easily revoke an agreement not validated by this Congress. Mr. Obama responded testily, accusing the signers of making “common cause with the hard-liners in Iran.”

Under the pending deal, Iran can maintain large nuclear facilities and continue enriching uranium. Mr. Obama says this is fine because Iran would have to allow inspections of specified nuclear sites. In response, also on TV Sunday, Israeli Prime Minister Benjamin Netanyahu said, “I do not trust inspections with totalitarian regimes. What I’m suggesting is that you contract Iran’s nuclear program, so there’s less to inspect.”

The key, in Mr. Netanyahu’s view, is to pressure Iran to dismantle major facilities. That way, if Tehran eventually decides to abandon the deal, the regime would have to work much more than a year to produce a nuclear weapon. In addition, Mr. Netanyahu says, sanctions should remain in place until Iran stops threatening its neighbors and supporting terrorism. Mr. Obama says such demands are unrealistic. They would kill the talks, he asserts, and leave only a military option.

But at the heart of the Obama-Netanyahu dispute—and of the president’s clash with Congress—is not diplomacy versus war. It’s the difference between cooperative diplomacy and coercive diplomacy.

By taking a cooperative approach, Mr. Obama insists, the U.S. and others can persuade Iran’s ruling ayatollahs to play by rules that all parties voluntarily accept. In contrast, the coercive option, which Mr. Netanyahu favors, assumes that Iran will remain hostile, dishonest and dangerous. The coercive approach sees Iran’s nuclear program as a symptom of the hostility between Iran and the West, but not as the source of the hostility. Coercion means America and its friends would use trade and financial restrictions, diplomatic isolation and other methods (short of military strikes) to pressure a resistant Iran into changing its behavior.

When Mr. Obama says the Israeli leader has offered “no viable alternative” to the deal being negotiated, he is denying that a coercive option exists. But Mr. Netanyahu’s point is that we can have one if we try. U.S. officials would need to exert leadership by highlighting Iranian threats, prescribing ways to limit them and soliciting other countries’ support.

There are two major problems with Mr. Obama’s cooperative approach. The first is the nature of Iran’s regime. The second is the history of attempts to constrain bad actors through cooperative approaches such as arms control and peace accords.

The Iranian regime is theocratic and revolutionary. It came to power in 1979 on a wave of extremist religious ideology and remains committed to exporting its revolution. Its leaders despise liberalism and democracy. They particularly hate Western respect for the rights of women and homosexuals. The regime remains in power through torture and murder of its domestic critics. It makes frequent use of public executions—the numbers have increased lately even though President Hasan Rouhani is commonly called a reformer.

Abroad, the Iranian regime acts as a rogue. Its agents and terrorist proxies have committed bombings and other murders in countries including France, Argentina, Saudi Arabia, Lebanon and Iraq. A U.S. court convicted Iranian agents of plotting in 2011 to assassinate the Saudi ambassador to the U.S. by bombing a Washington, D.C., restaurant. Iranian officials foment hatred of the U.S. and Israel and call for the annihilation of both.

Iranian leaders have a long record of shameless dishonesty. Their aid to the tyrannical Assad regime has been massive since the Syrian civil war began, but they routinely deny it. And they make a practice of lying to United Nations weapons inspectors. Commenting on how the inspectors have repeatedly been surprised by what Iran hides, Olli Heinonen, former deputy director-general of the U.N.’s International Atomic Energy Agency, told this newspaper in 2013, “If there is no undeclared installation today . . . it will be the first time in 20 years that Iran doesn’t have one.”

Iran is a bad actor, and history teaches that constraining bad actors through arms control and peace accords is a losing bet. The arms-control approach is to invite bad actors to sign legal agreements. This produces signing ceremonies, where political leaders can act as if there’s nobody here but us peaceable, law-abiding global citizens. The deal makers get to celebrate their accords at least until the bad actors inevitably violate them.

Nazi Germany violated the Versailles Treaty. The Soviet Union violated the Biological Weapons Convention, the Anti-Ballistic Missile Treaty, various nuclear-arms treaties and other international agreements. The Palestine Liberation Organization violated the Oslo Accords. North Korea violated the Agreed Framework.

Patterns emerge from this history. When leaders of democratic countries extract promises of good behavior from bad-actor regimes, those democratic leaders reap political rewards. They are hailed as peacemakers. British Prime Minister Neville Chamberlain was cheered when he returned from Munich in 1938 with “peace in our time.” These leaders have a stake in their deals looking good. When those deals are violated, the “peacemakers” often challenge the evidence. If the evidence is clear, they dismiss the violations as unimportant. When the importance is undeniable, they argue that there aren’t any good options for confronting the violators.

In the end, the bad actors often pay little or nothing for their transgressions. And even if the costs are substantial, they are bearable. Just ask Russia’s Vladimir Putin , or Syria’s Bashar Assad or North Korea’s Kim Jong Un.

The Obama administration has wedded itself to a cooperative policy toward Iran. The White House rejects the coercive approach as not viable. But if Iran violates its deal with us, won’t our response have to be coercive? President Obama insists that his policy is the only realistic one. In doing so, he is showing either that he is naïve and uninformed about the relevant history or that he no longer considers an Iranian nuclear weapon “unacceptable.”