On the Replication Crisis

“Standing on the shoulders of giants” – this phrase, attributed to Bernard of Chartres and prominently used by Isaac Newton, emphasises the role of past discoveries in ongoing scientific research. 


For modern research, where communication is predominately done through scientific publications, a high reliability of published findings is instrumental.

Yet surprisingly little is known about how reliable findings in scientific publications actually are.

In 2005, John Ioannidis argued in an influential essay that a large fraction of published findings are false – a claim that has been supported by recent systematic replication studies in several research fields. The Replication Project: Psychology (RPP), for instance, found that only 36 of 100 replicated studies came to the same conclusions as the originals. 

Key factors which contribute to a low rate of replicability include low power, publication bias, flexible experimental or statistical designs that permit ‘p-hacking’, and low pre-study odds for the tested hypotheses – i.e. the testing of ‘unlikely’ hypothesis. 


While the notion that science is facing a ‘replication crisis’ has been contested, it is becoming clear that a deeper understanding of factors that influence the reliability of published research is important.

It informs the design of conventions and policies in scientific research, but also the use of research findings by decision makers. Our engagement with DARPA’s SCORE programme is driven by our desire to contribute to this knowledge.

Contribute to the discussion...

This site uses Akismet to reduce spam. Learn how your comment data is processed.

May we send you invitations to future research projects? 

Add your email address to our low-volume distribution list.

Note: We cannot re-use participant lists for new recruiting. Please opt-in (above) for occasional announcements about related future studies.

We’re sorry to see you go! Please visit our social media sites.

This site uses cookies to provide you with a better browsing experience.

Visit our Privacy Policy for more information.