Evaluating Echo Chambers, Rabbit Holes, and Radicalization Pathways on YouTube

Background

YouTube’s recommendation system has played a prominent role in public and scholarly debates about online political information. As digital platforms have become increasingly central conduits for accessing political content, concerns have grown that recommendation algorithms may narrow what people see, reinforce existing political views, or guide users toward more extreme material. These concerns are especially important because recommendation systems shape not only what content users encounter but also what they are encouraged to watch next.

Studying these questions is difficult because user behavior and platform recommendations are closely intertwined. Watch-history data show what users ultimately viewed, but it cannot determine whether they arrived there because of YouTube’s recommendation algorithm or because of their own preferences. Other approaches, such as anonymous scraping or artificial accounts, can miss the personalized recommendations that real users receive. This study addresses those limitations by using an audit design with real YouTube users logged into their own accounts, allowing the authors to examine personalized recommendations while controlling how users navigated the platform.

Study

The authors conducted an original audit of YouTube recommendations shown to real users. The authors recruited 1,639 U.S.-based YouTube users in fall 2020 and asked them to complete a “traversal task” while logged into their own YouTube accounts. Participants installed a browser plug-in that recorded the videos YouTube recommended to them during the task, allowing the authors to observe personalized recommendations generated for real users rather than anonymous or artificial accounts.

The “traversal task” proceeded as follows. Each participant was randomly assigned a starting video from a set of 24 seed videos balanced across ideology and political and nonpolitical topics. Participants were then assigned to one of two conditions. In the “preference” condition, users clicked on whichever recommended video they found most interesting. In the “audit” condition, users followed a fixed rule, such as always clicking the first (or second, or fourth) recommendation. This design allowed the authors to compare what happens when users express their own preferences with what happens when user choice is constrained, helping separate the effects of YouTube’s recommendation algorithm from users’ own choices.

To evaluate the ideological content of recommendations, the authors used a method developed in related work to estimate the ideology of YouTube videos based on video metadata. They then analyzed how the ideology of recommendations changed across users and over successive traversal steps. Echo chambers were measured by whether recommendations differed by users’ partisanship or watch history. Rabbit holes were measured by whether recommendations were shaped by the ideology of the current or recently watched video. Radicalization pathways were measured by whether recommendations became both more ideologically extreme and narrower over time. 

Results

The study finds limited evidence that YouTube’s recommendation algorithm independently pushes users into ideological echo chambers. Republicans were shown somewhat more conservative recommendations than non-Republicans, but this difference appeared only when users were allowed to choose the recommended videos they found most interesting. In the audit condition, where users followed assigned traversal rules, the difference disappeared. This suggests that mild ideological echo chambers on YouTube were primarily driven by user behavior rather than by the recommendation algorithm alone.

The authors find strong evidence of content rabbit holes. Across both the preference and audit conditions, the ideology of the video a user was currently watching strongly predicted the ideology of the videos YouTube recommended next. The influence of previously watched videos faded quickly, with the two most recently watched videos mattering most for later recommendations. This suggests that YouTube’s recommendation system prioritized content similar to what users were currently watching, creating evolving pathways of related content.

The study does not find evidence that these rabbit holes produced radicalization pathways on average. Instead of steering Democrats and Republicans toward increasingly extreme content aligned with their own partisanship, the algorithm narrowed the range of recommendations while moving users toward moderately conservative content over time, regardless of users’ partisanship. 

The authors caution that these findings reflect YouTube recommendations in fall 2020, during a single-session audit with a convenience sample of users. They also note that user behavior and recommendation systems are closely intertwined: the algorithm may not independently produce radicalization pathways, but it can still respond to and reinforce what users appear to want. The results suggest that YouTube’s recommendation algorithm alone cannot fully explain radicalization risks on the platform, because user behavior and the broader availability of content also shape what users encounter.