European Business Schools Librarian's Group
HEC Research Papers Series,
HEC Paris
No 1467: Computational Reproducibility in Finance: Evidence from 1,000 Tests
Christophe Pérignon, Olivier Akmansoy, Christophe Hurlin, Anna Dreber, Felix Holzmeister, Juergen Huber, Magnus Johannesson, Michael Kirchler, Albert J. Menkveld, Michael Razen,
et al (1 additional author(s))
Utz Weitzel
Additional contact information
Christophe Pérignon: HEC Paris
Olivier Akmansoy: CNRS
Christophe Hurlin: University of Orleans
Anna Dreber: Stockholm School of Economics - Department of Economics
Felix Holzmeister: University of Innsbruck - Department of Economics
Juergen Huber: University of Innsbruck
Magnus Johannesson: Stockholm School of Economics - Department of Economics
Michael Kirchler: University of Innsbruck
Albert J. Menkveld: Vrije Universiteit Amsterdam
Michael Razen: University of Innsbruck
Utz Weitzel: VU University Amsterdam
Abstract: We analyze the computational reproducibility of more than 1,000 empirical answers to six research questions in finance provided by 168 international research teams. Running the original researchers’ code on the same raw data regenerates exactly the same results only 52% of the time. Reproducibility is higher for researchers with better coding skills and for those exerting more effort. It is lower for more technical research questions, more complex code, and for results lying in the tails of the results distribution. Neither researcher seniority, nor peer-review ratings appear to be related to the level of reproducibility. Moreover, researchers exhibit strong overconfidence when assessing the reproducibility of their own research. We provide guidelines for finance researchers and discuss several implementable reproducibility policies for academic journals.
Keywords: computational reproducibility; open science; credibility of research; multi-analyst study; data-availability policy; scientific publishing
61 pages, April 6, 2022
Full text files
papers.cfm?abstract_id=4064172 HTML file Full text
Questions (including download problems) about the papers in this series should be directed to Antoine Haldemann ()
Report other problems with accessing this service to Sune Karlsson ().
RePEc:ebg:heccah:1467This page generated on 2024-09-13 22:19:53.