Recently Completed Research Projects

In recent years, there has been wide-spread speculation about the possibility that response rates for national surveys have been low and are dropping due to increasing respondent reluctance to be interviewed.  This concern is accompanied by additional worry that low and dropping response rates are associated with decreased representativeness of survey samples and therefore reduced accuracy.

We initiated a project to attempt to better understand current response rates in the best and most visible surveys being done of nationally representative populations by telephone via Random Digit Dialing.  To this end, we approached the nation's leading news media polling organizations and the nation's leading survey research firms that do large-scale telephone surveys for agencies of the federal government.  All of the organizations we approached agreed to provide to us full disposition codes for recent national RDD telephone surveys, answers to a series of questions about how the surveys were conducted, and unweighted distributions of demographic variables for the respondents who completed interviews.

We found that response rates for the news media surveys were lower than those for the government contractors and that there was considerable variability in these response rates, with some very low and others quite high.  Observed response rates were correlated strongly with refusal rates and more weakly with contact rates.  Various aspects of survey procedure were associated with higher response rates, as would be expected, including longer field periods, shorter questionnaires, the payment of incentives, sending of advance letters, and more.

Most importantly, the unweighted demographics of the survey were compared to data on the nation gathered via the U.S. Census Bureau's Current Population Survey (an authoritative benchmark).  The survey samples were remarkably similar to the nation in terms of age, race, gender, education, and income.  Higher response rate surveys manifested slightly less error than lower response rate surveys, but these differences were quite small.

In recent decades, survey research throughout the world has shifted from emphasizing in-person interviewing of block-listed samples to random digit dialing samples interviewed by telephone.  In this paper, we propose three hypotheses about how this shift may bring with it changes in the psychology of the survey response, involving survey satisficing, enhanced social desirability response bias, and compromised sample representativeness among the most socially vulnerable segments of populations.  We report tests of these hypotheses using data from three national mode experiments.  As expected, RDD-telephone samples were less representative of the population and more significantly under-represented the most socially vulnerable segments.  Furthermore, telephone respondents were more likely to satisfice (as evidenced by no-opinion responding, non-differentiation, acquiescence, and interview length), less cooperative and engaged in the interview, and more likely to express dissatisfaction with the length of the interview.  Telephone respondents were also more suspicious about the interview and more likely to present themselves in socially desirable ways than were face-to-face respondents.  These findings shed light on the nature of the survey response process, on the costs and benefits associated with particular survey modes, and on the nature of social interaction generally.

Many researchers have argued that citizens combine information about political candidates by simply subtracting the number of unfavorable beliefs they have about a candidate from the number of favorable beliefs they have about the candidate.  This describes a symmetric linear process.  It is symmetric because favorable and unfavorable beliefs have the same magnitude of impact on attitudes.  It is linear because as beliefs are added, they have the same amount of impact as earlier beliefs.  For example, five beliefs have five times as much impact is one belief.  In addition, according to a symmetric linear model (SLM), citizens who have no favorable or unfavorable beliefs about a candidate have neutral attitudes toward him or her.

Work in psychology adopting a behavioral adaptive perspective suggests a number of amendments to the SLM.  According to this perspective, human cognitive and behavioral processes develop because they facilitate survival and reproduction in a potentially hostile world.  Approaching any new object with favorable expectations is worthwhile, because it could be food or could facilitate acquisition of food.  However, vigilantly scanning for ally signs of danger all object might pose is also important, so that harmful objects call be avoided.  In the absence of any information about all object, then, attitudes toward it should be slightly positive.  And people should be especially attentive to the first information they receive about all object in order to form an accurate first impression.  Then if the object appears to pose not immediate threat, vigilance can taper off, so that the impact of each additional piece of information acquired about the object may diminish.  However, because one must vigilantly scan for signs of danger in all object, unfavorable information should have more impact than favorable information and vigilance to additional unfavorable information should not taper off to the same degree as attention to additional favorable information.

The model we propose, the asymmetric nonlinear model (ANM) is based on this approach and makes three predictions about attitudes about political candidates that differ from those of the SLM. First, citizens who leave no favorable or unfavorable beliefs about a candidate should have slightly positive attitudes toward him or her. Second, the information should have less impact as the amount of previously acquired information increases. And third, unfavorable information should have greater impact than favorable information and/or as the amount of previously acquired information increases, the impact of unfavorable information should decrease slower than the impact of favorable information.

We compared the SLM and the ANM using National Election Study (NES) data from presidential elections from 1972 to 1996.  Cross-sectional NES data showed that the ANM describes attitudes toward presidential candidates and political parties better than the SLM among respondents high and low in political involvement (measured using education, political knowledge, time of voting decision, and whether or not respondents voted).  Longitudinal NES data (collected before and after presidential elections between 1980 and 1996) showed that the ANM outperforms the SLM in describing the impact of beliefs on changes over time in attitudes toward presidential candidates.  And the ANM revealed that voter turnout is enhanced by a stronger preference for one preferred candidate, as long as at least one candidate is dislike, whereas the SLM failed to detect this effect.  Thus, the ANM appears to be superior to the SLM and the ANM has important implications for understanding the impact of election campaigns on citizens' preferences and actions.

A great deal of evidence suggests that survey respondents?N answers to closed-ended questions can be influenced by the order in which those choices are presented. However, the impact of order depends upon whether the choices are presented visually or orally. Under visual presentation conditions, people are inclined to select the first response options they encounter, and whereas under oral presentation conditions, people are inclined to select the choices they encounter last.

Given that these order effects appear quite consistently in surveys, we were interested in whether they would appear in real elections. When they enter voting booths, citizens encounter candidates?N names visually, either written on paper or written on a voting machine. Findings from survey research therefore suggest that people may be inclined to selected names toward the top of the list. To test this idea, we collected actual voting returns for the 1992 elections in three large counties in Ohio. Precincts are randomly assigned to receive different orders of candidate names, so we were able to analyze these data as if they resulted from an experiment. And indeed, people were inclined to vote for candidates whose names appeared toward the top of the ballot. These effects were much more common in races about which voters knew less and where partisan affiliations of the candidates were not listed on the ballot.

In 2001-2002, we conducted another test of these effects.  This time, we analyzed data from the 2000 general election for the entire state of Ohio, as well as the states of North Dakota and California.  In all three states, portions of the state (such as precincts or counties) were assigned to receive different orders of candidate names for all races we analyzed, so we were able to analyze these data as if they resulted from an experiment.  Again, name order effects were found in many of the races we analyzed.  A side effect of this research project was our discovery of how widely varied name order laws are in the U.S.  We are currently in the process of writing a paper detailing the laws governing candidate name order throughout the U.S.

A number of theories posit that people's attitudes become stronger as they get older, though they disagree on exactly how and when this might occur. Using data from national and regional surveys of adults, we have found that peoples political attitudes are especially open to change between ages 18-25, become more resistant to change immediately thereafter, and become more open to change at the end of the life-cycle. Other manifestations of attitude strength (e.g., the personal importance of attitudes, the confidence with which they are held, and the amount of knowledge people feel they have) also show this same surge and decline.

We have recently expanded this program of research in a number of ways.  First, we are exploring the generalizability of our results to other attitude domains.  Because our research, like virtually all of the investigations that have preceded it, has focussed on attitudes toward social and political issues, it is not clear whether the observed pattern of openness to attitude change is unique to social and political attitudes, or whether it describes age-related fluctuation in openness to change more generally.

Second, in addition to resistance to attitude change, we are exploring changes over the life span in some of the other defining qualities of attitude strength. Specifically, we are assessing changes over the life span n the degree to which attitudes (1) motivate and guide behavior and (2) direct information processing.

Finally, we are moving beyond a simple description of the relation between age and openness to attitude change to explore the causal underpinnings of this relation.  Specifically, we are testing several possible mediators of the relation between age and openness to change, including (1) changes in the size, composition, and frequency of contact with  eople's social networks, (2) changes in the frequency of role transitions and new social identifications, (3) changes in the nature of people's self concept, and (4) changes in cognitive functioning over the life span.

This program of research promises to enrich our understanding of the aging process and refine our appreciation of the adult life cycle.  Equally important, however, this research will contribute to a broader understanding of the social and psychological factors that determine susceptibility to attitude change in general.

A great deal of literature has shown that the news media have the ability to influence peoples political judgments. One particular media effects is agenda-setting, the notion that by paying attention to a particular national problem, the media can induce people to cite it as the most important national problem. A second media effect is priming, the idea that prolonged focus on a political issue can lead Americans to derive their overall evaluations of their President's job performance primarily from his handling of that issue.

We tested a widely-held assumption about the cognitive mechanism responsible for these effects: accessibility. In short, scholars have presumed that media attention to an issue makes attitudes and beliefs about that issue especially accessible, which leads them to select the issue as the country's most important and leads them to place weight on it when evaluating presidential performance. However, our laboratory studies clearly refuted these hypotheses by showing that although news media to a problem did increase the accessibility of relevant attitudes and knowledge, this increase in accessibility did not mediate either effect. Furthermore, agenda-setting did not mediate priming; that is, considering a problem to be the nations most important did not lead people to place greater weight on it when evaluating presidential performance. These findings challenge prevailing wisdom about these news media effects and encourage future research seeking to identifying the mechanisms that are in fact at work.

Research in linguistics suggests that conventions govern the order in which words are listed in sentences during everyday conversations.  We examine one such convention, that when listing two terms, one positive and the other negative, it is conventional to list the positive one first (e.g., like or dislike, for or against, support or oppose).  Specifically, we examine whether, when asking a question to gauge a person's attitude it is conventional to offer the positive or affirmative response choice first, and the negative response choice second.

We found that in everyday conversation it is conventional to offer the positive or affirmative response option first.  We have found that violating conversational conventions can sometimes reduce the data quality of responses to attitude questions.  When the two options are presented in the unconventional order, expectations are violated, people are surprised and distracted, so responses are made more slowly and with more error.  These effects are most apparent among respondents with the least cognitive skills, those with low GPA's or little formal education.

If there is a convention regarding the order in which response alternatives to such questions should be offered, one might presume that researchers would never violate it, so the problems caused by violating the convention would never occur.  However, there is a reason why researchers may violate the convention: response order effects.  A great deal of research has found that the order in which response choices are offered can influence the distribution of answers to closed-ended questions, sometimes advantaging alternatives presented first, and other times advantaging alternatives presented last.  In order to minimize the impact of such response order effects on response distributions, some questionnaire design experts have advised that response order be systematically rotated across respondents and at least on major survey firm, the Gallup Organization, routinely rotates response alternatives in order to estimate and control for response order effects.

In the past, the only apparent costs of such rotation have been that it increases the complexity and expense of the survey and introduces a source of systematic measurement error that must then be modeled in multivariate statistical analyses. However, our research suggests that presenting responses in the unconventional order makes respondents' cognitive tasks more difficult and reduces data quality. Consequently, the best solution may be to use only the conventional response order and take steps to eliminate response order effects by enhancing respondent's motivation to thoughtfully answer survey questions.

Attitude Importance and Attitude Accessibility
(with George Bizer)

Some scholars have argued that people use attitude accessibility as a heuristic with which to infer attitude importance, whereas others have argued that importance causes accessibility. Through a series of experiments, we have examined the relation between these constructs.  We failed to find an effect of accessibility on importance, whereas we did find effects of importance on accessibility.  These findings have helped us to better understand the relation between these two constructs and, perhaps more importantly, the underlying structure of attitude strength in general.  Specifically, it appears that importance and accessibility represent distinct constructs, and some apparent effects of importance may be mediated by accessibility.

In September, 1995, the international community of scientists who study the environment announced that they had come to a new consensus that global warming has been occurring as the result of human activities and that it will have very significant and costly consequences for the world unless some steps are taken to slow its development. This new consensus was reported to Americans via television news programs and in newspapers, but these two media carried slightly different messages. Whereas the television messages simply acknowledged the new scientific consensus, newspaper stories acknowledged that a minority of scientists disagreed with this position, and newspaper stories published in October and November, 1995, were especially skeptical.

In December, 1995, we conducted a telephone survey of a representative sample of Ohio adults to study the diffusion and impact of this information. And in short, we found that people formed their beliefs about whether or not global warming is real using both news media information and their own personal experiences. Television exposure did indeed encourage people to believe more in the existence of global warming, whereas newspaper exposure discouraged such a belief. But these media effects occurred only among people who were highly trusting of scientists to provide accurate information. People who were distrusting of scientists based their assessments of the existence of global warming on their own first-hand observations of changes in temperature and air pollution levels in recent years. Those who thought temperatures had gotten warmer and who thought pollution had increased were especially likely to believe in global warming.

We also examined the origins of people's attitudes toward global warming. Although most people thought global warming would be negative, some felt it would be neither positive nor negative, and a few actually thought it would be positive overall. And these attitudes were apparently driven by people's beliefs about impact on factors immediately relevant to people's daily lives: food, water, and shelter. People who believed global warming would hurt food and water supplies and would flood coastal living areas held negative attitudes. In contrast, global warming?Ns perceived impact on the beauty of natural scenery, on processes of plant and animal species extinction, on animal migration, and the like were inconsequential. Therefore, it appears that people's attitudes were driven by their beliefs about the immediate material interests of society.

This survey project also allowed us to explore some general issues in the attitude literature. For example, we examined whether four dimensions of attitude strength (attitude importance, prior thought, certainty, and perceived knowledge) are all reflections of a single underlying construct. And although a factor analysis of them yielded a single factor, they were correlated quite differently with demographic variables, psychological antecedents, and a measure of the magnitude of the false consensus effect. This evidence reinforces the general conclusion that attitude strength is not a unitary construct.

In September and October, 1997, we conducted a telephone survey or a representative sample of adults.  In December, 1997 through February, 1997, we re-interviewed a portion of those interviewed in September and October, as well as an additional representative sample of adults who had not previously been interviewed.  Between these two sets of interviews, the White House Conference on Global Climate Change occurred, and hundreds of stories on global warming were broadcast on television and radio, and published in newspapers and magazines across the country.  Our goal was to re-examine our findings from the Ohio survey with a national sample and to study how this media coverage changed public beliefs and attitudes about global warming.

On the surface, American public opinion about global warming did not seem to change in response to media coverage of the issues.  However, changes did occur when party identification was considered.  Strong Democrats moved in the direction of the message coming from the White House (i.e., that global warming would happen, that it would be bad, and that something should be done about it) while strong Republicans moved in the opposite direction.  So even though overall attitudes did not change, opinions polarized along party lines.  In addition to this partisan polarization, the media attention led the public to do more thinking about the issue of global warming and to be more certain of their opinions about global warming.  People were also able to report their opinion about global warming more quickly during the second set of interviews, one indicator that people's opinions about global warming were more crystallized after the media campaign.

Respondents' reports of behavioral frequencies have implications for important issues spanning the spectrum of unemployment rates, medical epidemiology, neighborhood and community evaluations, transport infrastructure, crime rates, consumer behavior, and government health resource allocation. Despite numerous assumptions about the relative strengths and weaknesses of questions asking about the past week vs. a typical week, there is a lack of empirical evidence comparing the performance of these two question forms. One previous study revealed no significant difference between past week and typical week measures, but those analyses treated variances in these two question forms as if they were the same. Using more appropriate analysis techniques, we compared the validity of "typical" week and "past" week reports using data from the 1989 National Election Pilot Study, in which respondents were randomly assigned to report TV news and newspaper exposure during either a typical week or the past week. The predictive validity of the measures was assessed using objective tests of current events and political knowledge, as well as self-report assessments of political knowledge. The typical week question form proved to be consistently superior, especially among the most educated respondents. We encourage further attempts to replicate the current findings in other domains of behavioral frequencies.

Satisficing theory suggests that respondents may sometimes choose the first satisfactory response alternative they consider, rather than carefully considering all the response alternatives.  This theory predicts that respondents are most likely to satisfice when they are unable and/or unmotivated to think carefully about a question and when the question is difficult to answer.  When questions are presented orally, respondents typically cannot start thinking about the response alternatives until all have been read, so they more fully process response alternatives read last.  This process typically leads to recency effects when questions are presented orally.  In a meta-analysis of 212 dichotomous response order experiments in telephone surveys conducted by the Gallup Organization between 1995 and 1998, we are testing the impact of respondents ability, respondent motivation, and task difficulty on the likelihood and magnitudes of response order effects.  In addition, we are exploring a new hypothesis, that the order in which response options are considered can be affected by the linguistic structure of the question.