Tackling the Reproducibility Crisis

In recent years, the issue of reproducibility has gained significant attention across various fields of research. The failure to replicate the results of published studies has prompted the scientific community to explore new ways of ensuring that computational methods and data are transparent, understandable, and accessible. Reproducibility is especially crucial in fields like bioinformatics, where large datasets and complex computational models are increasingly common.

To tackle this challenge, our team has developed ENCORE (ENhancing COmputational REproducibility), a practical tool that aims to significantly enhance the transparency and reproducibility of computational research.

Why ENCORE?

Reproducibility in computational research is complicated by several factors, such as undocumented manual processing steps, unavailable or outdated software, changes in public repositories, and a lack of comprehensive documentation. ENCORE integrates all project components into a standardized File System Structure (sFSS) to combat these challenges. This structure simplifies documenting and sharing projects, making it easier for other researchers to replicate the work.

The Development and Evolution of ENCORE

The development of ENCORE was a meticulous and extensive process that involved refining, testing, and internal evaluations over several years. It was initially introduced to our research group in 2020, and since then, it has been instrumental in creating over 20 ENCORE projects and structuring more than 50 data analysis projects for external collaborations.

One of the primary challenges we encountered in adopting ENCORE was ensuring consistent compliance with its guidelines across all projects. Upon initial evaluation, it became evident that while most projects had adopted the sFSS structure, the documentation's level of detail varied, leading to difficulties reproducing some projects. To address this, we simplified the file system structure, reduced the number of pre-defined files, and enhanced documentation guidelines.

The development process also involved continuous feedback and iteration. For instance, early versions of ENCORE required manual effort to link data, code, and results. However, we overcame this obstacle by creating an HTML-based sFSS Navigator. This tool significantly simplified the project directory, helping researchers navigate and identify key components with ease.

The Road Ahead: Expanding ENCORE’s Impact

We have already witnessed the significant impact of ENCORE within our group, and we strongly believe its potential can be maximized if embraced by a wider audience. To facilitate widespread adoption, we have released ENCORE on GitHub for free, allowing researchers to access and integrate the template into their projects. Furthermore, we are committed to providing comprehensive documentation and best practices for utilizing ENCORE, which will undoubtedly facilitate its widespread adoption.

Moreover, we are confident that ENCORE can play a pivotal role in advancing the transparency and credibility of scientific research on a larger scale. By establishing a standardized method for computational reproducibility, ENCORE aligns with the growing trend toward open science and open data. Looking ahead, we are eager to collaborate with other research groups to enhance ENCORE further and tailor it to the specific needs of various scientific communities.