Survey Professionalism: New Evidence from Web Browsing Data
To assess the prevalence of professional responders to surveys, we analyzed three samples from the U.S. Our findings provide key insights into the scale and effects of survey professionalism on online panels.
Citation
Von Hohenberg, Bernard, Tiago Ventura, Jonathan Nagler, Ericka Menchen-Trevino, and Magdalena Wojcieszak. "Survey Professionalism: New Evidence from Web Browsing Data." OSF, (2024). https://osf.io/preprints/osf/jxdc8
Date Posted
Aug 30, 2024
Authors
- Bernhard Clemm von Hohenberg,
- Tiago Ventura,
- Jonathan Nagler,
- Ericka Menchen-Trevino,
- Magdalena Wojcieszak
Area of Study
Tags
Abstract
Online panels have become an important resource for research in political science, but the financial compensation involved incentivizes respondents to become “survey professionals”, which raises concerns about data quality. We provide evidence on survey professionalism using behavioral web browsing data from three U.S. samples, recruited via Lucid, YouGov, and Facebook (total n = 3,886). Survey professionalism is common but varies across samples: By our most conservative measure, we identify 1.7% of respondents on Facebook, 7.9% of respondents on YouGov, and 34.3% of respondents on Lucid as survey professionals. However, evidence that professionals lower data quality is limited: they do not systematically differ demographically or politically from non-professionals and do not respond more randomly—although they are somewhat more likely to speed, to straightline, and to take questionnaires repeatedly. While concerns are warranted, we conclude that survey professionals do not, by and large, distort inferences of research based on online panels.