How Language Shapes Belief in Misinformation: A Study Among Multilinguals in Ukraine

This study investigates how the language in which misinformation is presented influences its credibility among multilingual Ukrainians, particularly examining whether misinformation in a less preferred or secondary language is evaluated differently.

Abstract

Scholarship has identified key determinants of people’s belief in misinformation predominantly from English-language contexts. However, multilingual citizens often consume news media in multiple languages. We study how the language of consumption affects belief in misinformation and true news articles in multilingual environments. We suggest that language may pass on specific cues affecting how bilinguals evaluate information. In a ten-week survey experiment with bilingual adults in Ukraine, we measured if subjects evaluating information in their less-preferred language were less likely to believe it. We find those who prefer Ukrainian are less likely to believe both false and true stories written in Russian by approximately 0.2 standard deviation units. Conversely, those who prefer Russian show increased belief in false stories in Ukrainian, though this effect is less robust. A secondary digital media literacy intervention does not increase discernment as it reduces belief in both true and false stories equally.

Background

Our cumulative knowledge about belief in misinformation predominantly comes from surveying English-speaking Americans about misinformation written in English from American media sources. However, the global media environment is multilingual. Half of the global population uses two or more languages or dialects in their daily life and, therefore, likely consumes media, including misinformation, in multiple languages from both within and outside of their borders. Given both the potentially troubling consequences for democracy and the rise of multilingual citizens who consume news in different languages in a single media market, it is imperative to develop a more comprehensive understanding of how news consumers assess the veracity of news in different languages. In this registered report, we conducted a study that addresses the broad research question: Are individuals more or less susceptible to misinformation in their non-preferred language? We conduct the study in Ukraine, where pro-Kremlin disinformation is rife and Ukrainians read in both Ukrainian and Russian languages.

Study

We conducted a 10-week survey experiment in Ukraine from May 15 to July 19, 2024 to test our hypotheses. Ukraine is an ideal case to test our hypotheses because most of its citizens are bilingual news consumers in Ukrainian and Russian. Each week, we used quota sampling to ensure geographic, ideological, and linguistic variation. We develop a transparent, replicable, and pre-registered news article selection process that identifies popular false/misleading, and true articles within 24 hours of their publication and subsequently distributes the full articles for evaluation to respondents in Ukraine. Our respondents evaluate these popular articles within 48-96 hours of publication. For each of the study's 10 weeks, we collect and distribute a new group of five articles for each respondent to evaluate in randomized order. We then test whether respondents evaluate stories differently, depending on whether they consume the story in their preferred language.

Results

Exposing subjects to misinformation in their less-preferred language reduces belief in these stories, but the data tell a much more nuanced story. When we estimate the effects separately for those who prefer Russian and Ukrainian, we find that Ukrainian-preferring respondents reading stories in Russian reduce their belief in misinformation significantly, while Russian-preferring respondents increase their ratings of misinformation if the story is presented in Ukrainian. However, exposing respondents to stories in their less-preferred language also reduces their belief in true stories in equal measure, meaning the effect on discernment overall is substantively null. Our findings suggest that language serves as a credibility cue in multilingual environments, but these effects work asymmetrically depending on majority versus minority language status and do not improve people's ability to distinguish truth from falsehood.