Current Research Projects

Researchers interested in collecting survey data from national samples often consider three possible ways of doing so: (1) telephone interviewing of RDD samples, (2) internet data collection from non-probability samples of volunteers, and (3) internet data collection from probability samples recruited via RDD.  In order to help inform such choices, a single questionnaire was designed and administered by each of eight survey firms (one doing RDD telephone interviewing, one doing internet data collection from a probability sample, and six doing internet data collection from non-probability samples; each sample approximately 1,000 people).  The firm that conducted internet data collection from a probability sample and one firm that collected data from volunteer respondents (SPSS) were told in advance that results would be compared across firms, and the remaining firms were not told in advance that such comparisons would be made.  A set of preliminary results were presented at the 2005 annual meeting of the American Association for Public Opinion Research (AAPOR).  Questions about this study can be directed to Professor Douglas Rivers at Stanford University (rivers@stanford.edu).

For years, researchers have been interested in what determines citizens?N decisions about which candidates to vote for in presidential elections.  These decisions have intrigued political scientists and social psychologists alike, and their research has led to the discovery of numerous elements that seem to play causal roles: party identification, performance of the incumbent, the health of the national economy, candidates?N stances on important national issues, perceptions of candidates?N personalities, and more.  Although correlations of various political attitudes and beliefs with candidate preferences are well-documented and assumed to influence impact on candidate preferences, social psychology (and cognitive consistency theories in particular) suggests a very different interpretation of these correlations: rationalization.  

We are investigating the theory that people form candidate preferences and thereafter change related political attitudes and beliefs so they become more consistent with their candidate choice.  Using all available National Election Study data, we have found evidence of pervasive rationalization in every instance possible to analyze.  We have found evidence of rationalization in ideological self-identifications, party identification, and stances on abortion and other national policy issues. We are now attempting to employ panel data to yield strong evidence about the causes of candidate choices, eliminating the contaminating influence of rationalization.

Survey researchers are generally mindful that balancing the wording of a question can alter the distributions of answers obtained. However, researchers who choose to use balanced questions can choose among multiple ways to achieve this aim. A u?Lfully balancedi^ question involves fully restating the competing point of view, whereas a u?Lminimally balancedi^ question simply uses the words u?Lor noti^ or a phrase of that sort to briefly acknowledge a second viewpoint.

In two studies using national sample survey data, we compared the distributions and concurrent validity of responses across fully and minimally balanced questions. We also explored whether the impact of full balancing varied with respondent education, a variable that has been shown in prior studies to regulate the magnitude of various response effects. Across these studies, minimally balanced and fully balanced questions resulted in similar distributions of responses of equivalent validity, and this pattern did not vary with respondent education.

A third study examined the distributions of responses to factual knowledge questions using a sample of college undergraduates. Participants provided responses to either fully balanced or minimally balanced questions that were worded either in a conversationally conventional way (e.g., u?Lgreater or less thani^) or in a non-conventional way (e.g., u?Lless than or greater thani^).  The latter approach has been shown in other research to disrupt processing and reduce data quality, and we found here that the unconventional question wording yielded fewer correct answers from respondents.  When the unconventional wording was used, full balancing enhanced answer accuracy over what was obtained with minimal balancing.  But when a question is worded in a way consistent with conversational conventions, full balancing did not offer an advantage over minimal balancing in terms of response accuracy. 

Therefore, when questions are worded in conversationally conventional ways, the practical benefits of minimal balancing give that approach a relative advantage over full balancing.  However, if researchers were inclined to violate conversational conventions, fully balanced items may offer an advantage.

In the social psychological literature on attitude change, there has been an ongoing debate regarding how being personally involved in an issue impacts attitude change.  One perspective has been that involvement in an issue can be separated into two types of involvementA?None based on self-interest and one based on value-relevance.  It was hypothesized that these two constructs would have separate effects on attitude change.  The other perspective is that involvement need not be separated into its more specific components because they will all lead to the same effect on attitude change.  No previous studies have provided an adequate framework from which to test these hypotheses.

An experimental framework was created in which to test the effects of self-interest and value-relevance on attitude change.  In the study, participants were manipulated to connect the issue of comprehensive graduation exams to either their self-interest or their values before they read either a strong or weak counter-attitudinal message regarding the issue, which consistent of appeals to both self-interest and values. 

The findings provide preliminary support for the idea that value-relevance interacts with message strength in the same manner as self-interest to affect attitude change.  In addition, high value-relevance also led to a greater resistance to attitude change than high self-interest when messages were weak, indicating that separate effects may also be plausible.

Currently, this study is being re-run with new self-interest and value-relevance manipulations as well as more clear definitions of self-interest and values. The results of this study will hopefully elucidate the earlier findings while providing greater insight on how involvement is best conceptualized as a variable in attitude change. 

With their response rates declining and costs rising, telephone surveys are increasingly difficult to conduct.  At the same time, Internet data collection is emerging as a viable alternative, in two forms.  Some firms are distributing computer equipment to national samples recruited through RDD calling, and other firms are attracting volunteer respondents and then building panels of those individuals with some demographic characteristics distributed as they are in the nation.  Most firms assemble panels of respondents who provide data on a regular basis.

Given the obvious practical advantages of Internet-based data collection, it seems worthwhile to conduct object tests of this relatively new method in direct comparison with the dominant alternative methodology: telephone interviewing.  To do so, we commissioned a set of side-by-side surveys using a single questionnaire to gauge public opinion and voting intentions regarding the 2000 U.S. Presidential Election from national samples of American adults.

Data were collected by three houses: The Ohio State University Center for Survey Research (CSR), Knowledge Networks (KN), and Harris Interactive (HI).  The CSR did RDD telephone interviewing.  KN recruited respondents via RDD telephone interviews and equipped them with WebTV, which then permitted Internet data collection.  HI respondents joined a panel after seeing and responding to invitations to participate in regular surveys; the invitation appeared on the Excite search engine web page and in various other places as well.  These respondents also completed Internet surveys.

This study suggests that Internet-based data collection represents a viable approach to conducting representative sample surveys.  Internet-based data collection compromises sample representativeness, more so when respondents volunteer rather than being recruited by RDD methods.  But Internet data collection improves the accuracy of the reports respondents provide over that rendered by telephone interviews.

During the last two decades, numerous surveys have been done to assess the monetary value that people place on the existence of various natural resources that have been damaged by human actions. For example, the Exxon Valdez oil tanker damaged ecosystems in Prince William Sound, Alaska, and surveys later sought to ascertain the value of those ecosystems for litigation. This sort of survey research has been called "contingent valuation".

A heated debate has been raging in the contingent valuation literature about how best to measure these monetary values. Some scholars argue that it is best to ask respondents closed-ended questions about whether they would be willing to pay a specified amount of money to prevent the same sort of damage from happening again in the future. Different respondents are randomly assigned to be asked about different amounts, and everyone?Ns answers are used to assess the public?Ns total willingness to pay. Other scholars argue that this approach may bias estimates of willingness to pay by anchoring people's thinking on the particular values offered. Therefore, these scholars suggest, it is preferable to ask open-ended questions about how much respondents would be willing to pay.

We initiated a project assessing the impact of anchoring on responses to these sorts of closed ended questions. We found that although offering values close to a person?Ns true willingness to pay can in fact pull his or her answers in the direction of the anchor, anchors that are far from people?Ns true willingness to pay have no such biasing impact. Furthermore, as long as anchors are spread across a wide range of dollar values, the particular values chosen do not seem to bias estimated total willingness to pay. Thus, the open-ended and closed-ended questioning approaches perform comparably. This finding enhances our understanding of the comparative functioning of open-ended and closed-ended questions more generally.

Thousands of experimental studies have compared the effectiveness of questionnaire items written in different ways, yet these studies have never been brought together in a single review. We are now completing a book doing just that. We will draw upon this literature to recommend to people when to use open-ended vs. closed-ended questions, when to use rating scales vs. ranking tasks, how many points to put on rating scales and how to label the points verbally, how the order of response choices influences answers, whether to offer "don?Nt know" response options, how to word and order questions, whether to ask people to recall their attitudes at prior times, and whether to ask people to explain the causes of their thinking and actions. The result of our efforts is an empirically-validated set of recommendations about how to maximize the reliability and validity of data collected via questionnaires.

A great deal of research has explored the psychological origins of citizen activism intended to influence government policy in democratic societies.  Although various scholars have speculated that one motivator of such behavior is the perception of a credible threat of policy change in an undesirable direction, this hypothesis has never been directly tested using data on individual citizens and their perceptions.  Our research tests the hypothesis that policy change threat leads to activism.  More specifically, we are testing the effect of a citizen?Ns belief that a piece of legislation that he/she does not want to see passed i` whether for health or safety reasons, or just personal preference i` might actually become law.

We suspect that such threats will be more likely to inspire activism among some individuals than others, one determining factor being the personal importance of the issue to the individual. To a citizen who attaches a great deal of importance to an issue, an undesirable policy change would be personally devastating.  But no matter how much significance people attach to an issue, they cannot be immediately and vigorously responsive to a threat unless they have the available resources in terms of time and/or money.  Resources have, of course, been recognized as very important determinants of activism.  But in all past research, resources have been statistically treated as having direct effects on activism, as if simply having more money or more free time, in and of themselves, directly inspire political action.  Although this research implies that the effect of resources on activism depends on certain conditions, it is surprising that this hypothesis has never been explicitly tested, which we have now done.

We have conducted a series of studies to test the following hypotheses: 1) that policy change threat leads to political activism; 2) that the effect of policy change threat on activism is moderated by personal importance; and 3) that the effect of policy change threat on activism is moderated by income.  These studies have employed a variety of methods, including nationally representative telephone surveys and a field experiment, and have examined threat in a variety of context, including attitudes towards the environment, abortion, and presidential candidates.

The results of these studies have been quite compelling i` in all cases, the threat of an undesirable policy change motivated activism aimed at preventing the change.  In addition, we have found that personal importance exacerbates the effect of threat on activism, as does income, when the type of activism examined is financial contributions (an act for which income is a necessary resource).   We are currently conducting a nationally representative telephone survey experiment and a laboratory experiment to extend our findings and show additional causal evidence of the effect of threat on activism.

Satisficing theory proposes a number of survey features and individual differences that are likely to make satisficing more or less likely.  For example, individuals who have not given much thought to an issue are theorized to be especially likely to satisfice; similarly, individuals lower in education or who don't like to think may tend to satisfice.  On the other hand, individuals who are specifically instructed to try to answer questions carefully and accurately may more likely to provide optimal answers.  We are currently subjecting these proposed moderators of satisficing to experimental tests, using both student and national samples.  In particular, these experiments will shed light on the ways in which various interpersonal and situational factors interact to influence survey responding.

Currently assisting in the development of several national-level survey questionnaires for a NASA program designed to track changes in US aviation safety. This project has applied research on the relationship between the organization of events in memory and recall of those memories to survey instrument design. More specifically, this research has used focus groups and individual interviews with pilots and air traffic controllers to identify the safety-related issues that inform the content of the questionnaire as well as preliminary information on the organization of related events, identification of a key cognitive organization scheme used by pilots?N in their memories of safety-related events, assessments of pilots?N abilities to accurately recall events over time, and an experimental demonstration of the effectiveness of using memory cues matching the pilots?N organization scheme to enhance accurate recall for safety-related events.

Surveys that forecast election outcomes have implications for campaign strategies, financial contributions, political analysis in the mass media and academia, as well as actual electoral turnout. However, techniques for improving the accuracy of election forecasting in polling organizations are often proprietary and not amenable to comparative evaluation. With the intention of moving the field toward viewing these techniques as an appropriate terrain for scientific investigation, we investigated how election forecasting would be affected by (a) elimination of respondents who are not likely to vote, (b) allocation of undecided respondents to candidates or referendum positions, (c) weighting of samples for representativeness, (d) addition of random responses, (e) controlling for candidate name order effects. 

Using data from the 1997-1999 Buckeye State Polls collected by the Center for Survey Research at The Ohio State University, we found that (a) although better forecasting is achieved when using a combination of filters than separate filters to eliminate non-voters, there is a limit on the number of filters that should be applied. Filtering down to 50% of the samples is optimal for forecasting candidate races, while forecasting of referendum benefit from using only 10-20% of the original samples. (b) Random allocation of undecided respondents to candidates or issue positions improved the forecasting potential of the pre-election surveys for both candidate races and referenda. (c) Although weighting of the unfiltered samples did not consistently improve forecasting, substantial improvement was attained on referenda forecasting by weighting the samples after the optimal number of filters have been applied. (d) Addition of random responses did not improve forecasting. (e) Candidate name order effects emerged on the surveys, with recency effects most pronounced for less publicized races and among respondents with less education. Based on these results, we were able to provide a set of recommendations on how to improve election forecasting using empirically-validated techniques.