2016-2017 Winter Colloquia

ABSTRACT: Should experts always practice what they preach? When an expert displays exemplary behavior, individuals who fear negative devaluation sometimes anticipate that this expert will look down on them. As a result, displays of excellence can paradoxically turn off the very people they are trying to inspire. Seven studies document this in the medical domain, showing that individuals who are overweight or obese and concerned about their weight avoid physicians who advertise their fitness, for fear that these doctors will judge them negatively. Several studies test ways to prevent individuals who are overweight and obese from fearing devaluation from fit physicians, and explore how having some unhealthy habits exposed as a physician can reduce fear of devaluation. This work demonstrates that it is critical to take into account ego-defensive processes when attempting to lead by example.

ABSTRACT: : Scientific advancement is largely an incremental process in which new discoveries are built on the shoulders of giants, the validity of the former hinging in part on the validity of the latter. In a democratic society in which scientific discoveries are a public good funded by the public, public appreciation for and trust in how scientists conduct scientific work is a critical element of scientific advancement.

Social scientists have made great strides in improving how research is done to achieve greater validity. A major purpose of social science research is to transport the behavioral outcomes relations, findings obtained from specific studies to the population as a whole. Various scientific practices, such as using representative samples or replicating the findings by own or other scientists, permit or aid the transportability. Past research suggested that people do not appreciate the merits of these scientific practices in increasing generalizability. For example, people seemed oblivious to the potential problems of sample selection bias and sample size used in the studies, and this phenomenon may be driven by representative heuristic people employ in judgment and decision-making.

The present study aims to devise novel approaches for social scientists to communicate to the public random sampling, replication and other scientifically sound practices, and ascertain how the public may respond to these practices. Of particular interest is to explore how much people believe the finding of a single social science research study.

This workshop is to explore some of the following factors that may determine how much people believe a finding:

1. Random vs. not random sampling (random sampling has to be described effectively)
2. Size of sample
3. Heterogeneity of sample (college students vs. general public)
4. Whether other scientists want to replicate the study
5. Whether other scientists have begun to try to replicate the study
6. Whether replication attempts by the same scientist produced the same results
7. Whether replication attempts by other scientists produced the same results
8. Whether the finding is surprising/counter-intuitive
9. Whether the finding threatens the respondent
10. Whether there is a compelling story of what the mechanism of the effect might be
11. Whether there is evidence of what the mechanism of the effect is

ABSTRACT: The reproducibility of scientific findings has become an issue of growing concern in science. Disciplines including medicine, psychology, genetics, and biology researchers have been repeatedly challenged by findings that are not as robust as they initially appeared. Shrinking effects and outright failures of replication raise questions not only about the specific findings they challenge, but more generally about the confidence that we can have in published results that have yet to be verified independently.

Furthermore, meta-analyses, which aggregate the findings of numerous related studies into a single analysis, indicate that many effects in biology, medicine, and psychology have declined with repeated replication. Surveys of the scientific practices of scientists suggest that a sizable proportion of researchers engage in practices that elevate the risk of false positive results. Statistical techniques have identified patterns of results in scientific studies that suggest they may be “too good to be true.” Replication projects, some still currently underway, are investigating the degree to which existing paradigms can be independently replicated. This project aims to investigate some of the potential causes of the decline effect.

The proposed investigation in Deciphering the Decline Effect will not only robustly assess the merits of the individual hypotheses explored in each study, but also more broadly confront the nature of the scientific process itself. In particular, the collective outcome of the proposed studies will address a host of conventional accounts that have been hypothesized to affect the replicability of scientific findings, including: false positive effects, selective reporting, publication bias, and changes in procedure or sampling. Furthermore, if decline effects emerge despite researchers best efforts to control them, this might suggest the possibility of an unconventional mechanism that may have been masked by the lack of transparency of the scientific process.

ABSTRACT: Latent semantic analysis of text data may provide additional information on voting behavior under threat. We analyzed text related to the British EU referendum campaign to study the relationship between threat, belongingness needs, and choosing the status quo. Experimental data was collected from a British Internet Campaign Panel (Prolific) conducted a couple of weeks before the EU referendum in June 2016. The respondents were treated with one of two threat scenarios by thinking about their own death or a dentist visit, and reported their vote intention in the EU referendum. Clustering of the latent semantic representation of respondents’ word descriptions of the potentially threatening event revealed a significant relationship between threat, anxiety and voting for the status quo. Threat amplified the relationship between rejection sensitive characteristics and voting behavior. The paper demonstrates that latent semantic word cluster analysis is a useful tool, which complements traditional statistic on numerical data. The findings add knowledge to the impact of threat and individual level characteristics on voting in a referendum, and the relationship between risk aversions and voting for or against the status quo.

ABSTRACT: In many communication contexts, behavior modification is a desired outcome. At the individual-level, behavior change is often the goal of persuasive messages, such as when a doctor attempts to get a patient to stop smoking. Similarly, attempts at behavior modification are common in the domain of mass communication. Advertisements, such as those used by politicians to mobilize their supporters or demobilize their opponents’ supporters or those used by firms attempting to get consumers to purchase a product or service, are frequently aimed at altering recipient behavior. Prior research suggests that even minimal exposure or communication may result in attitude change (Zajonc 1968) and behavior change (Sherman 1980). The link between answering questions about a target behavior and subsequent performance of that behavior by survey respondents has been one important and reliable effect discovered in the psychology literature. While most work on this question-behavior link has focused on behaviors that will take place farther into the future such as voting, purchasing, or engaging in physical activity, earlier work in survey methodology by Charles Cannell and his colleagues indicated that asking respondents to commit to providing accurate responses at the beginning of an interview can indeed produce higher quality data in that survey. However, the method developed by Cannell and his colleagues has not been widely adopted nor has it been systematically tested in self-administered survey modes such as mail or web-based surveys. In these contexts, there is reason to believe that the results found by Cannell may not replicate. For example, commitments made verbally to an interviewer may prove to be much more effective than those made in a web or mail survey. Additional research is needed to address these gaps in the extant literature. Using a series of experiments, this project will evaluate the effects of respondent commitments to provide high-quality data on a variety of indicators of survey data quality and on the strength of experimental treatment effects for substantive research questions. This project will leverage data from surveys on a variety of commercial and political topics that utilize student samples or samples from opt-in web panels. While these data sources cannot be taken to be representative of the general population, they do provide a diverse set of individuals on which to assess the experimental treatment effects.

ABSTRACT: The ANES (American National Election Studies) Time Series surveys have been conducted every 2 years from 1948 to 2008, using a combination of fresh cross sections and panels, including a pre-election and a post-election component each survey. Many of these survey datasets are unweighted. In this project, we are building one of the most comprehensive and high-quality sets of weights for every ANES Times Series dataset dating from 1948 to 2008. For each ANES year, we build base weights from the ANES dataset to account for probability of selection, defaulting to the NEHM (number of eligible household members), adding adjustments for (if needed) panel attrition and sampling bias. Then, demographic and election variables are processed and raked with our base weights, through a post-stratification weighting methodology described in “Computing Weights for American National Election Study Survey Data” by Debell and Krosnick (2009), implemented through the ‘anesrake’ R package, maintained by Josh Pasek. The final raked weights, for pre-election and post-election, are tested against CPS (Current Population Survey) benchmarks as well as election results. The R code, final weights, and result comparison tables are maintained for each ANES year. Raked variables achieve much higher accuracy to the benchmarks while unraked variables have also shown improved accuracy over the unraked data.

ABSTRACT: Dr. Mick Smyer will report on the process of the development and implementation of Graying Green: Climate Action for an Aging World. In the presentation, Dr. Smyer will include commentary on the role of social science survey data in the design process, and the value of combining social science methods with design methods.

ABSTRACT: Politicians are widely perceived to lose significance upon leaving office. Yet media accounts often highlight politicians’ legacies as a source of influence that endures even after they retire. The article assesses these contrasting views by investigating the substance, endurance, and significance of legacies. We develop a theoretical account of legacies and their relevance to contemporary politics, distinguishing between “hard legacies”- concrete and enduring policy achievements – and “soft” legacies”- memories enshrined in the public’s consciousness. We ground our theoretical account empirically by testing a series of observable implications using data from online discussion forums, thousands of former politicians’ Wikipedia pages, original surveys of both citizens and political elites, and a randomized experiment. We find that establishing a lasting legacy is a key motivation of public officials. More generally, our findings provide substantial evidence that legacies influence contemporary policy debates long after a leader steps down.