Data Science Methodology
Our experts produce new methodologies to further understand how social media affects politics and democracy. From developing and deploying code, CSMaP researchers create new ways to quantify social media interactions and its effects.
Academic Research
-
Working Paper
Web Scraping for Research: Legal, Ethical, Institutional, and Scientific Considerations
Working Paper, December 2024
Scientists across disciplines often use data from the internet to conduct research, generating valuable insights about human behavior. However, as generative AI relying on massive text corpora becomes increasingly valuable, platforms have greatly restricted access to data through official channels. As a result, researchers will likely engage in more web scraping to collect data, introducing new challenges and concerns for researchers. This paper proposes a comprehensive framework for web scraping in social science research for U.S.-based researchers, examining the legal, ethical, institutional, and scientific factors that researchers should consider when scraping the web. We present an overview of the current regulatory environment impacting when and how researchers can access, collect, store, and share data via scraping. We then provide researchers with recommendations to conduct scraping in a scientifically legitimate and ethical manner. We aim to equip researchers with the relevant information to mitigate risks and maximize the impact of their research amidst this evolving data access landscape.
-
Journal Article
News Sharing on Social Media: Mapping the Ideology of News Media, Politicians, and the Mass Public
Political Analysis, 2024
Reports & Analysis
-
Analysis
Are Influence Campaigns Trolling Your Social Media Feeds?
Now, there are ways to find out. New data shows that machine learning can identify content created by online political influence operations.
October 13, 2020
News & Commentary
-
News
2024 Year in Review: Our Research & Impact
A look at our top articles, events, and more from the past year.
December 18, 2024
-
Policy
Feedback on the EU's Digital Services Act
The European Commission's Digital Services Act is a critical step towards supporting data access for independent research. We submitted comments on this legislation, advocating for structures and mechanisms that would ensure secure and standardized data sharing.
December 9, 2024