Speaker
Description
Computational methods are in full swing in communication science. Part of their promise is to make communication research more reproducible. However, how this plays out in practice has not been systematically studied. We verify the reproducibility of the entire cohort of 30 substantive and methods papers published in the journal Computational Communication Research (CCR), the official journal of the International Communication Association Computational Methods Division with a focus on transparency and hence a high rate of voluntary Open Science participation in the field. Among these CCR papers, we are not able to verify the computational reproducibility of 16 papers as no data and/or code were shared. For the remaining 14 papers, we attempt to execute the code shared by the original authors in a standardized containerized computational environment. We encounter a variety of issues that preclude us from reproducing the original findings, where incomplete sharing of data or code is the most common issue. In the end, we could at least partially reproduce the findings in only 6 papers (20%). Based on our findings, we discuss strategies for researchers to correct for this disheartening state of computational reproducibility. We emphasize the lack of computational reproducibility as a socio-technical challenge and at least for this case the social part was the main culprit. To speak to research software engineers, we warn against the "Technological Solutionism" approach to support computational reproducibility.
| I want to participate in the youngRSE prize | no |
|---|
