Download data and study materials from OSF
Principal investigator:
Michael Tomz
Stanford University
Email: tomz@stanford.edu
Homepage: http://www.stanford.edu/~tomz
Sample size: 4413
Field period: 07/02/2004-04/10/2005
Recent models of international relations assume that leaders would suffer “domestic audience costs” if they issued threats or promises and failed to follow through. Citizens, it is claimed, would think less of leaders who backed down than of leaders who never committed in the first place. This project uses experiments embedded in public opinion surveys to study the existence and nature of audience costs. In each experiment, the interviewer describes a military crisis. Some participants are randomly assigned to a control group and told that the president does not get involved. Others are placed in a treatment condition in which the president escalates the crisis but ultimately backs down. All participants are then asked whether they approve of the way the president handled the situation. By comparing approval ratings in the “stay out” and “back down” conditions, one can measure audience costs directly without strategic selection bias that has hampered previous research.
Define absolute audience costs as disapproval when the president escalates but does not follow through, minus disapproval when the president avoids getting involved in the first place. The experiments were designed to test several hypotheses: (1) Audience costs are positive, rather than zero or negative; (2) Audience costs are positive across a wide range of international contexts, but vary with the motive, political regime, and military power of the adversary, and with the material interests that are at stake; (3) Audience costs increase with the level of escalation, e.g. they are larger when the president backs down after using force than when the president backs down after merely threatening to use force; (4) Audience costs vary with the voter’s attitudes toward the use of force—hawk versus dove—and internationalism versus isolationism.
*This Data Contains Two Separate Experiments on This Topic*
Experiment 1: This experiment involved 2 x 2 x 2 x 2 x 5 = 80 conditions. All participants in the Internet-based survey received an introductory script: “You will read about a situation our country has faced many times in the past and will probably face again. Different leaders have handled the situation in different ways. We will describe one approach U.S. leaders have taken, and ask whether you approve or disapprove.” Participants then read about a foreign crisis in which “a country sent its military to take over a neighboring country.”
The experiment involved variation in four contextual variables—regime, motive, power, and interests—that have been shown to be consequential in the international relations literature. Each contextual variable had two possible values, and the four were independently randomized, thereby creating a total of 2 x 2 x 2 x 2 = 16 configurations of background conditions.
Specifically, the invading country was led by a “dictator” in half the interviews and a democratically elected government” in the other half. The attacker sometimes had aggressive motives—it invaded “to get more power and resources”—and sometimes invaded “because of a long-standing historical feud.” Half the participants were informed that the attacker had a “strong military,” such that “it would have taken a major effort for the United States to help push them out,” and the other half were told that the attacker had a “weak military,” which the United States could have repelled without major effort. Finally, a victory by the attacking country would either “hurt” or “not affect” the safety and economy of the United States.
Having read the background information, participants learned how the U.S. president handled the situation. Each respondent received a vignette in which the president followed one of five strategies:
[1] Stay out: “The U.S. president said the United States would stay out of the conflict. The attacking country continued to invade. In the end, the U.S. president did not send troops, and the attacking country took over its neighbor.”
[2] Back down from verbal threat: “The U.S. president said that if the attack continued, the U.S. military would push out the invaders. The attacking country continued to invade. In the end, the U.S. president did not send troops, and the attacking country took over its neighbor.”
[3] Back down from show of force: “The U.S. president said that if the attack continued, the U.S. military would push out the invaders. He sent troops to the region and prepared them for war. The attacking country continued to invade. In the end, the U.S. president did not send our troops into battle, and the attacking country took over its neighbor.”
[4] Back down from use of force with no casualties: “The U.S. president said that if the attack continued, the U.S. military would push out the invaders. He sent troops to the region and prepared them for war. The attacking country continued to invade. The president then ordered U.S. troops to destroy one of the invader’s military bases. U.S. troops destroyed the base, and no Americans died in the operation. The invasion still continued. In the end, the U.S. president did not order more military action, and the attacking country took over its neighbor.”
[5] Back down from use of force with American casualties: “The U.S. president said that if the attack continued, the U.S. military would push out the invaders. He sent troops to the region and prepared them for war. The attacking country continued to invade. The president then ordered U.S. troops to destroy one of the invader’s military bases. U.S. troops destroyed the base, and 20 Americans died in the operation. The invasion still continued. In the end, the U.S. president did not order more military action, and the attacking country took over its neighbor.”
At the end of the experiment, each respondent received a set of bullet points that recapitulated the scenario, and then indicated to what degree they approved of the way the U.S. president handled the situation.
Experiment 2: This experiment followed the same pattern as Experiment 1, but introduced a new feature: Senate support versus opposition.
When the president stayed out, half the respondents were told that “Most U.S. senators publicly supported the president’s decision; they, too, wanted the United States to stay out of the conflict.” The remaining respondents were told that “Most U.S. senators publicly opposed the president’s decision; they wanted the U.S. military to push out the invaders.
When the president escalated, half the respondents were told that “Most U.S. senators publicly supported the president’s decision; they, too, wanted the U.S. military to push out the invaders.” The remaining respondents were told that “Most U.S. senators publicly opposed the president’s decision; they wanted the United States to stay out of the conflict.”
Presidential approval: “Do you approve, disapprove, or neither approve nor disapprove of the way the U.S. president handled the situation?” Respondents who approved or disapproved were asked whether they held their view very strongly, or only somewhat strongly. Those who answered “neither” where prompted: “Do you lean toward approving of the way the U.S. president handled the situation, lean toward disapproving, or don’t you lean either way?”
Audience costs exist across a wide range of conditions and increase with the level of escalation. The costs are evident throughout the population, and especially among politically active citizens who have the greatest potential to shape government policy.
These findings demonstrate the promise of using experiments to address central questions in the field of international relations. These findings also have important substantive implications: they supply behavioral microfoundations for theories of signaling and commitment in world affairs. It is widely assumed that domestic audiences contribute to credibility by punishing leaders who say one thing but do another. I confirm that citizens respond this way, a discovery that was far from preordained. If citizens had focused on foreign policy outcomes rather than processes, or regarded bluffing as a reasonable strategy, or rewarded leaders for trying before conceding, or cared little about their country’s reputation, audience costs would not have emerged. By showing that audience costs arise consistently across a wide range of conditions, this article advances our understanding of signaling and commitment in both international security and political
Respondents were also asked about their political party affiliation and to what extent they agreed or disagreed with the following statements: “The use of military force only makes problems worse” and “The United States needs to play an active role in solving conflicts around the world.”
Michael Tomz, “Domestic Audience Costs in International Relations: An Experimental Approach,” International Organization 61, no. 4 (Fall 2007): 821–40.