Academic Research

Automated text analysis methods have made possible the classification of large corpora of text by measures such as topic and tone. Here, we provide a guide to help researchers navigate the consequential decisions they need to make before any measure can be produced from the text. We consider, both theoretically and empirically, the effects of such choices using as a running example efforts to measure the tone of New York Times coverage of the economy. We show that two reasonable approaches to corpus selection yield radically different corpora and we advocate for the use of keyword searches rather than predefined subject categories provided by news archives. We demonstrate the benefits of coding using article segments instead of sentences as units of analysis. We show that, given a fixed number of codings, it is better to increase the number of unique documents coded rather than the number of coders for each document. Finally, we find that supervised machine learning algorithms outperform dictionaries on a number of criteria. Overall, we intend this guide to serve as a reminder to analysts that thoughtfulness and human validation are key to text-as-data methods, particularly in an age when it is all too easy to computationally classify texts without attending to the methodological choices therein.

The COVID-19 pandemic represents a massive global health crisis. Because the crisis requires large-scale behaviour change and places significant psychological burdens on individuals, insights from the social and behavioural sciences can be used to help align human behaviour with the recommendations of epidemiologists and public health experts. Here we discuss evidence from a selection of research topics relevant to pandemics, including work on navigating threats, social and cultural influences on behaviour, science communication, moral decision-making, leadership, and stress and coping. In each section, we note the nature and quality of prior research, including uncertainty and unsettled issues. We identify several insights for effective response to the COVID-19 pandemic and highlight important gaps researchers should move quickly to fill in the coming weeks and months.

“Clickbait” headlines designed to entice people to click are frequently used by both legitimate and less-than-legitimate news sources. Contemporary clickbait headlines tend to use emotional partisan appeals, raising concerns about their impact on consumers of online news. This article reports the results of a pair of experiments with different sets of subject pools: one conducted using Facebook ads that explicitly target people with a high preference for clickbait, the other using a sample recruited from Amazon’s Mechanical Turk. We estimate subjects’ individual-level preference for clickbait, and randomly assign sets of subjects to read either clickbait or traditional headlines. Findings show that older people and non-Democrats have a higher “preference for clickbait,” but reading clickbait headlines does not drive affective polarization, information retention, or trust in media.

For nearly a century social scientists have sought to understand left–right ideological differences in values, motives, and thinking styles. Much progress has been made, but — as in other areas of research — this work has been criticized for relying on small and statistically unrepresentative samples and the use of reactive, self-report measures that lack ecological validity. In an effort to overcome these limitations, we employed automated text analytic methods to investigate the spontaneous, naturally occurring use of language in nearly 25,000 Twitter users. We derived 27 hypotheses from the literature on political psychology and tested them using 32 individual dictionaries. In 23 cases, we observed significant differences in the linguistic styles of liberals and conservatives. For instance, liberals used more language that conveyed benevolence, whereas conservatives used more language pertaining to threat, power, tradition, resistance to change, certainty, security, anger, anxiety, and negative emotion in general. In 17 cases, there were also significant effects of ideological extremity. For instance, moderates used more benevolent language, whereas extremists used more language pertaining to inhibition, tentativeness, affiliation, resistance to change, certainty, security, anger, anxiety, negative affect, swear words, and death-related language. These research methods, which are easily adaptable, open up new and unprecedented opportunities for conducting unobtrusive research in psycholinguistics and political psychology with large and diverse samples.

Michael Jordan supposedly justified his decision to stay out of politics by noting that Republicans buy sneakers too. In the social media era, the name of the game for celebrities is engagement with fans. So why then do celebrities risk talking about politics on social media, which is likely to antagonize a portion of their fan base? With this question in mind, we analyze approximately 220,000 tweets from 83 celebrities who chose to endorse a presidential candidate in the 2016 U.S. presidential election campaign to assess whether there is a cost — defined in terms of engagement on Twitter — for celebrities who discuss presidential candidates. We also examine whether celebrities behave similarly to other campaign surrogates in being more likely to take on the “attack dog” role by going negative more often than going positive. More specifically, we document how often celebrities of distinct political preferences tweet about Donald Trump, Bernie Sanders, and Hillary Clinton, and we show that followers of opinionated celebrities do not withhold engagement when entertainers become politically mobilized and do indeed often go negative. Interestingly, in some cases political content from celebrities actually turns out to be more popular than typical lifestyle tweets.

Are legislators responsive to the priorities of the public? Research demonstrates a strong correspondence between the issues about which the public cares and the issues addressed by politicians, but conclusive evidence about who leads whom in setting the political agenda has yet to be uncovered. We answer this question with fine-grained temporal analyses of Twitter messages by legislators and the public during the 113th U.S. Congress. After employing an unsupervised method that classifies tweets sent by legislators and citizens into topics, we use vector autoregression models to explore whose priorities more strongly predict the relationship between citizens and politicians. We find that legislators are more likely to follow, than to lead, discussion of public issues, results that hold even after controlling for the agenda-setting effects of the media. We also find, however, that legislators are more likely to be responsive to their supporters than to the general public.

Pinning down the role of social ties in the decision to protest has been notoriously elusive largely due to data limitations. Social media and their global use by protesters offer an unprecedented opportunity to observe real-time social ties and online behavior, though often without an attendant measure of real-world behavior. We collect data on Twitter activity during the 2015 Charlie Hebdo protest in Paris, which, unusually, record real-world protest attendance and network structure measured beyond egocentric networks. We devise a test of social theories of protest that hold that participation depends on exposure to others' intentions and network position determines exposure. Our findings are strongly consistent with these theories, showing that protesters are significantly more connected to one another via direct, indirect, triadic, and reciprocated ties than comparable nonprotesters. These results offer the first large-scale empirical support for the claim that social network structure has consequences for protest participation.

Computational propaganda and the use of automated accounts in social media have recently become the focus of public attention, with alleged Russian government activities abroad provoking particularly widespread interest. However, even in the Russian domestic context, where anecdotal evidence of state activity online goes back almost a decade, no public systematic attempt has been made to dissect the population of Russian social media bots by their political orientation. We address this gap by developing a deep neural network classifier that separates pro-regime, anti-regime, and neutral Russian Twitter bots. Our method relies on supervised machine learning and a new large set of labeled accounts, rather than externally obtained account affiliations or orientation of elites. We also illustrate the use of our method by applying it to bots operating in Russian political Twitter from 2015 to 2017 and show that both pro- and anti-Kremlin bots had a substantial presence on Twitter.

A major point of debate in the study of the Internet and politics is the extent to which social media platforms encourage citizens to inhabit online “bubbles” or “echo chambers,” exposed primarily to ideologically congenial political information. To investigate this question, we link a representative survey of Americans with data from respondents’ public Twitter accounts (N = 1,496). We then quantify the ideological distributions of users’ online political and media environments by merging validated estimates of user ideology with the full set of accounts followed by our survey respondents (N = 642,345) and the available tweets posted by those accounts (N ~ 1.2 billion). We study the extent to which liberals and conservatives encounter counter-attitudinal messages in two distinct ways: (a) by the accounts they follow and (b) by the tweets they receive from those accounts, either directly or indirectly (via retweets). More than a third of respondents do not follow any media sources, but among those who do, we find a substantial amount of overlap (51%) in the ideological distributions of accounts followed by users on opposite ends of the political spectrum. At the same time, however, we find asymmetries in individuals’ willingness to venture into cross-cutting spaces, with conservatives more likely to follow media and political accounts classified as left-leaning than the reverse. Finally, we argue that such choices are likely tempered by online news watching behavior.

Social scientific models of protest activity emphasize instrumental motives associated with rational self-interest and beliefs about group efficacy and symbolic motives associated with social identification and anger at perceived injustice. Ideological processes are typically neglected, despite the fact that protest movements occur in a sociopolitical context in which some people are motivated to maintain the status quo, whereas others are motivated to challenge it. To investigate the role of ideology and other social psychological processes in protest participation, we used manual and machine-learning methods to analyze the contents of 23,810 tweets sent on the day of the May Day 2012 Occupy Wall Street demonstration along with an additional 664,937 tweets (sent by 8,244 unique users) during the 2-week lead-up to the demonstration. Results revealed that social identification and liberal ideology were significant independent predictors of protest participation. The effect of social identification was mediated by the expression of collective efficacy, justice concerns, ideological themes, and positive emotion. The effect of liberalism was mediated by the expression of ideological themes, but conservatives were more likely to express ideological backlash against Occupy Wall Street than liberals were to express ideological support for the movement or demonstration. The expression of self-interest and anger was either negatively related or unrelated to protest participation. This work illustrates the promise (and challenge) of using automated methods to analyze new, ecologically valid data sources for studying protest activity and its motivational underpinnings — thereby informing strategic campaigns that employ collective action tactics. 

So-called “fake news” has renewed concerns about the prevalence and effects of misinformation in political campaigns. Given the potential for widespread dissemination of this material, we examine the individual-level characteristics associated with sharing false articles during the 2016 U.S. presidential campaign. To do so, we uniquely link an original survey with respondents’ sharing activity as recorded in Facebook profile data. First and foremost, we find that sharing this content was a relatively rare activity. Conservatives were more likely to share articles from fake news domains, which in 2016 were largely pro-Trump in orientation, than liberals or moderates. We also find a strong age effect, which persists after controlling for partisanship and ideology: On average, users over 65 shared nearly seven times as many articles from fake news domains as the youngest age group.

How accurate are survey-based measures of social media use, in particular about political topics? We answer this question by linking original survey data collected during the U.S. 2016 election campaign with respondents’ observed social media activity. We use supervised machine learning to classify whether these Twitter and Facebook account data are content related to politics. We then benchmark our survey measures on frequency of posting about politics and the number of political figures followed. We find that, on average, our self-reported survey measures tend to correlate with observed social media activity. At the same time, we also find a worrying amount of individual-level discrepancy and problems related to extreme outliers. Our recommendations are twofold. The first is for survey questions about social media use to provide respondents with options covering a wider range of activity, especially in the long tail. The second is for survey questions to include specific content and anchors defining what it means for a post to be “about politics.”

We introduce a novel classification of strategies employed by autocrats to combat online opposition generally, and opposition on social media in particular. Our classification distinguishes both online from offline responses and censorship from engaging in opinion formation. For each of the three options — offline action, technical restrictions on access to content, and online engagement — we provide a detailed account for the evolution of Russian government strategy since 2000. To illustrate the feasibility of researching online engagement, we construct and assess tools for detecting the activity of political "bots," or algorithmically controlled accounts, on Russian political Twitter, and test these methods on a large dataset of politically relevant Twitter data from Russia gathered over a year and a half.

As non-democratic regimes have adapted to the proliferation of social media, they have begun actively engaging with Twitter to enhance regime resilience. Using data taken from the Twitter accounts of Venezuelan legislators during the 2014 anti-Maduro protests in Venezuela, we fit a topic model on the text of the tweets and analyze patterns in hashtag use by the two coalitions. We argue that the regime’s best strategy in the face of an existential threat like the narrative developed by La Salida and promoted on Twitter was to advance many competing narratives that addressed issues unrelated to the opposition’s criticism. Our results show that the two coalitions pursued different rhetorical strategies in keeping with our predictions about managing the conflict advanced by the protesters. This article extends the literature on social media use during protests by focusing on active engagement with social media on the part of the regime. This approach corroborates and expands on recent research on inferring regime strategies from propaganda and censorship.

The following report is intended to provide an overview of the current state of the literature on the relationship between social media; political polarization; and political “disinformation,” a term used to encompass a wide range of types of information about politics found online, including “fake news,” rumors, deliberately factually incorrect information, inadvertently factually incorrect information, politically slanted information, and “hyperpartisan” news. The review of the literature is provided in six separate sections, each of which can be read individually but that cumulatively are intended to provide an overview of what is known—and unknown—about the relationship between social media, political polarization, and disinformation. The report concludes by identifying key gaps in our understanding of these phenomena and the data that are needed to address them.

It is often claimed that social media platforms such as Facebook and Twitter are profoundly shaping political participation, especially when it comes to protest behavior. Whether or not this is the case, the analysis of “Big Data” generated by social media usage offers unprecedented opportunities to observe complex, dynamic effects associated with large-scale collective action and social movements. In this article, we summarize evidence from studies of protest movements in the United States, Spain, Turkey, and Ukraine demonstrating that: (1) Social media platforms facilitate the exchange of information that is vital to the coordination of protest activities, such as news about transportation, turnout, police presence, violence, medical services, and legal support; (2) in addition, social media platforms facilitate the exchange of emotional and motivational contents in support of and opposition to protest activity, including messages emphasizing anger, social identification, group efficacy, and concerns about fairness, justice, and deprivation as well as explicitly ideological themes; and (3) structural characteristics of online social networks, which may differ as a function of political ideology, have important implications for information exposure and the success or failure of organizational efforts. Next, we issue a brief call for future research on a topic that is understudied but fundamental to appreciating the role of social media in facilitating political participation, namely friendship. In closing, we liken the situation confronted by researchers who are harvesting vast quantities of social media data to that of systems biologists in the early days of genome sequencing.

Automated and semiautomated Twitter accounts, bots, have recently gained significant public attention due to their potential interference in the political realm. In this study, we develop a methodology for detecting bots on Twitter using an ensemble of classifiers and apply it to study bot activity within political discussions in the Russian Twittersphere. We focus on the interval from February 2014 to December 2015, an especially consequential period in Russian politics. Among accounts actively Tweeting about Russian politics, we find that on the majority of days, the proportion of Tweets produced by bots exceeds 50%. We reveal bot characteristics that distinguish them from humans in this corpus, and find that the software platform used for Tweeting is among the best predictors of bots. Finally, we find suggestive evidence that one prominent activity that bots were involved in on Russian political Twitter is the spread of news stories and promotion of media who produce them.

We analyzed Twitter language to explore hypotheses derived from moral foundations theory, which suggests that liberals and conservatives prioritize different values. In Study 1, we captured 11 million tweets from nearly 25,000 U.S. residents and observed that liberals expressed fairness concerns more often than conservatives, whereas conservatives were more likely to express concerns about group loyalty, authority, and purity. Increasing political sophistication exacerbated ideological differences in authority and group loyalty. At low levels of sophistication, liberals used more harm language, but at high levels of sophistication conservatives referenced harm more often. In Study 2, we analyzed 59,000 tweets from 388 members of the U.S. Congress. Liberal legislators used more fairness- and harm-related words, whereas conservative legislators used more authority-related words. Unexpectedly, liberal legislators used more language pertaining to group loyalty and purity. Follow-up analyses suggest that liberals and conservatives in Congress use similar words to emphasize different policy priorities.

How successful is the Islamic State’s online strategy? To what extent does the organization achieve its goals of attracting a global audience, broadcasting its military successes, and marketing the Caliphate? Using Twitter and YouTube search data, we assess how suspected ISIS accounts, sympathizers, and opponents behave across two social media platforms, offering key insights into the successes and limitations of ISIS’ information warfare strategy. Analyzing the tweet content and metadata from 16,364 suspected ISIS accounts, we find that a core network of ISIS Twitter users are producing linguistically diverse narratives, touting battlefield victories and depicting utopian life in the Caliphate. Furthermore, a dataset of over 70 million tweets, as well as analysis of YouTube search data, indicates that although pro-ISIS content spreads globally and remains on message, it is far less prolific than anti-ISIS content. However, this anti-ISIS content is not necessarily anti-extremist or aligned with Western policy goals.

How can one technology—social media—simultaneously give rise to hopes for liberation in authoritarian regimes, be used for repression by these same regimes, and be harnessed by antisystem actors in democracy? We present a simple framework for reconciling these contradictory developments based on two propositions: 1) that social media give voice to those previously excluded from political discussion by traditional media, and 2) that although social media democratize access to information, the platforms themselves are neither inherently democratic nor nondemocratic, but represent a tool political actors can use for a variety of goals, including, paradoxically, illiberal goals.

This chapter examines the use of social networking sites such as Twitter in measuring public opinion. It first considers the opportunities and challenges that are involved in conducting public opinion surveys using social media data. Three challenges are discussed: identifying political opinion, representativeness of social media users, and aggregating from individual responses to public opinion. The chapter outlines some of the strategies for overcoming these challenges and proceeds by highlighting some of the novel uses for social media that have fewer direct analogs in traditional survey work. Finally, it suggests new directions for a research agenda in using social media for public opinion work.

Political debate concerning moralized issues is increasingly common in online social networks. However, moral psychology has yet to incorporate the study of social networks to investigate processes by which some moral ideas spread more rapidly or broadly than others. Here, we show that the expression of moral emotion is key for the spread of moral and political ideas in online social networks, a process we call “moral contagion.” Using a large sample of social media communications about three polarizing moral/political issues (n = 563,312), we observed that the presence of moral-emotional words in messages increased their diffusion by a factor of 20% for each additional word. Furthermore, we found that moral contagion was bounded by group membership; moral-emotional language increased diffusion more strongly within liberal and conservative networks, and less between them. Our results highlight the importance of emotion in the social transmission of moral ideas and also demonstrate the utility of social network methods for studying morality. These findings offer insights into how people are exposed to moral and political ideas through social networks, thus expanding models of social influence and group polarization as people become increasingly immersed in social media networks.

As more than a billion people had done previously, on November 21, 2013, Ukrainian journalist and activist Mustafa Nayem wrote a Facebook post; this post, however, would have a much larger impact on subsequent political developments than most that had preceded it. Frustrated with President Viktor Yanukovych’s decision not to sign a long-promised association agreement with the European Union, Nayem asked others who shared his frustration to comment on his post. Even more importantly, Nayem wrote that if the post received at least 1,000 comments from people willing to join him, they should all go to Independence Square to protest. And indeed they did: starting with just a few thousand people, the protests would swell to be the largest since Ukraine’s independence, particularly after police used force against protesters at the end of November 2013. Eventually, these protests led to the resignation of the government, the exile of the former president, and indirectly to the secession of Crimea and the ongoing conflict in the eastern part of the country.