Download data and study materials from OSF
Geoffrey P.R. Wallace
Sample size: 2929
Field period: 9/26/2010-12/21/2011
Recent research on international law suggests states use international legal agreements as a commitment device to increase the credibility of their promises by raising the reputational consequences of violations. Testing this reputational mechanism for compliance has been complicated by selection effects both in a state’s decision to join an agreement in the first place, as well as its likely anticipation of the consequences of subsequent noncompliance. This project uses an experiment embedded in a national survey to estimate the effect of a legalized commitment on a country’s reputation, which is measured by the ability of the country to garner support from foreign actors for future cooperative agreements.
More legalized commitments are expected to have a greater impact on a country’s reputation compared to a situation where the country made no prior formal promise. This study examines one type of reputational consequence, which is operationalized as the willingness of a domestic actor (in this case the U.S. public) to support or oppose future cooperation with a foreign country based on the latter’s compliance behavior with regards to different types of legalized commitments. Drawing on the existing international relations literature as well as the broader study of reputation, the hypotheses are as follows.
1) Complying with/violating a legalized commitment by a foreign state increases/decreases public support for future cooperative agreements with that state.
2) The more legalized the commitment by a foreign state, the greater the increase/decrease in public support for future cooperative agreements with that state based on its compliance/violation behavior.
3) The impact of complying with/violating a legalized commitment will be greater for agreements involving high politics issue areas than low politics issue areas.
4) The impact of complying with/violating a legalized commitment will be greater for future cooperative agreements dealing with the same issue area as the original agreement compared to unrelated issue areas.
All respondents were presented with a scenario where a foreign country faces a situation where it needs to make a foreign policy decision. Respondents are randomly assigned to different additional facts regarding the scenario according to the following three treatments.
1) Type of scenario (binary): The treatment group was given a military scenario, where the foreign country needed to decide whether or not to use military force to take control of a contested island where natural resources were recently discovered. The control group was given an economic scenario, where the foreign country was facing economic difficulties domestically and needed to decide whether or not to raise trade barriers against imports.
2) Type of legal commitment (trichotomous): This treatment focused on the existence and level of legalization in a previous commitment made by the foreign country. The first treatment group was given a hard legalization prompt, where the country had signed a formal international treaty not to undertake the relevant policy option based on the type of scenario (for the military scenario, use force to take the island; for the economic scenario, raise trade barriers). The second treatment group was given a soft legalization prompt, where the country had participated in the creation of regional guidelines not to undertake the relevant policy option. Finally, the control group was told the country had not previously made any legal obligation refusing to undertake the relevant policy option.
3) Violation (binary): The treatment group was told the foreign country eventually decided to undertake the relevant policy option. The control group was told the foreign country eventually decided to undertake the relevant policy option.
The use of two binary treatments and one trichotomous treatment implied a 2x3x2 factorial design with 12 total experimental groups.
After being presented with the scenario, respondents were asked two questions regarding the degree to which they would support the U.S. government signing a future agreement with the foreign country, where one question concerned a military agreement and the other a trade agreement. For each respondent the first question asked was based on the scenario they faced (for instance, the trade question was asked first for respondents receiving the economic scenario, followed by the military question). Response options for both questions were based on a five-point Likert scale ranging from strongly support to strongly oppose.
The findings are generally consistent with the expectations that legalized commitments increase the reputational consequences of compliance decisions. Across both military and economic scenarios, respondents were much less likely to support future U.S. cooperation with a foreign country that violated a previous legalized commitment compared to a similar foreign country that engaged in the same action but had not made a previous legal promise. On the other hand, foreign states achieved greater benefits by complying with legalized commitments compared to engaging in similar compliant behavior absent a prior legal promise, though the reputational gains of compliance were smaller than the costs of violation. This pair of findings suggests that while states can improve their reputation through complying with legalized promises, the primary reputational function of an international agreement is to serve as a commitment device that increases the ex post costs of noncompliance. Additional results show that the reputational consequences of international law are conditioned by an agreement’s level of legalization and the type of issue area. The findings also suggest possible spillover effects from a country’s commitments and compliance behavior in one issue area for its reputation across other issue areas.
By offering a specific test of a reputational mechanism, this study builds on a series of earlier surveys that found international law and legalization had a significant impact on public attitudes toward foreign policy. This study was thus intended to assess the possible role of reputational concerns in explaining the overall effect of legalized commitments.