There is evidence that the results of a significant number of published scientific studies are not reproducible, i.e. the same result cannot be obtained in a repeat study using the same methods. “Simulations suggest that this could in part be due to ‘cultural evolutionary processes,’” says Dr. Oliver Braganza of the Institute of Experimental Epileptology and Cognition Research at the University Hospital Bonn (UKB). A member of the ImmunoSensation2 Cluster of Excellence, the transdisciplinary research areas Life & Health, Modelling and Individuals & Societies as well as the University of Bonn Center for Science and Thought, Dr. Braganza has conducted simulations to determine for example what sample size tends to maximize the researcher’s number of publications. Many studies with small sample sizes, it was revealed, can lead to more publications, which can be beneficial for a scientific career.
“Cultural evolutionary processes are likely to play a role in situations where our actions are influenced by cultural factors, such as field norms,” Dr. Braganza explains. “The decision as to what size a sample should be is often not fully determinable on scientific grounds, reflecting instead, to a degree, field norms or successful scientific precedents.” In this view, the standards or techniques that maximize success in the scientific selection process will be ‘passed on’ from professors to doctoral students. “The result would be that non-replicable results accumulate in the literature,” Dr. Braganza emphasizes, that “this process could occur without bad intent or even knowledge of individual researchers”.
Such simulation studies have not yet been experimentally corroborated, however. “That is unfortunate because experiments remain the gold-standard to establish causal relationships, and human behavior is generally more complex than corresponding model assumptions”. The researchers working on the project propose a paradigm which systematically links empirical study, simulation and experimentation. Dr. Braganza: “Such an approach will allow us to test predictions resulting from simulations on real human subjects in a controlled ‘laboratory context’, and systematically study the effectiveness of interventions.” The objective is to find ways to improve current scientific evaluation and reward systems, for instance to promote more reproducible research.