Mistakes happen, catch them if you can.

While scientists strive for perfection, mistakes happen. It is important to identify and correct errors as soon as possible so they do not perpetuate in the scientific literature.

Published in Microbiology

Mistakes happen, catch them if you can.
Like

Share this post

Choose a social network to share with, or copy the URL to share elsewhere

This is a representation of how your post may appear on social media. The actual post will vary between social networks

Explore the Research

BioMed Central
BioMed Central BioMed Central

A framework for assessing 16S rRNA marker-gene survey data analysis methods using mixtures. - Microbiome

Background There are a variety of bioinformatic pipelines and downstream analysis methods for analyzing 16S rRNA marker-gene surveys. However, appropriate assessment datasets and metrics are needed as there is limited guidance to decide between available analysis methods. Mixtures of environmental samples are useful for assessing analysis methods as one can evaluate methods based on calculated expected values using unmixed sample measurements and the mixture design. Previous studies have used mixtures of environmental samples to assess other sequencing methods such as RNAseq. But no studies have used mixtures of environmental to assess 16S rRNA sequencing. Results We developed a framework for assessing 16S rRNA sequencing analysis methods which utilizes a novel two-sample titration mixture dataset and metrics to evaluate qualitative and quantitative characteristics of count tables. Our qualitative assessment evaluates feature presence/absence exploiting features only present in unmixed samples or titrations by testing if random sampling can account for their observed relative abundance. Our quantitative assessment evaluates feature relative and differential abundance by comparing observed and expected values. We demonstrated the framework by evaluating count tables generated with three commonly used bioinformatic pipelines: (i) DADA2 a sequence inference method, (ii) Mothur a de novo clustering method, and (iii) QIIME an open-reference clustering method. The qualitative assessment results indicated that the majority of Mothur and QIIME features only present in unmixed samples or titrations were accounted for by random sampling alone, but this was not the case for DADA2 features. Combined with count table sparsity (proportion of zero-valued cells in a count table), these results indicate DADA2 has a higher false-negative rate whereas Mothur and QIIME have higher false-positive rates. The quantitative assessment results indicated the observed relative abundance and differential abundance values were consistent with expected values for all three pipelines. Conclusions We developed a novel framework for assessing 16S rRNA marker-gene survey methods and demonstrated the framework by evaluating count tables generated with three bioinformatic pipelines. This framework is a valuable community resource for assessing 16S rRNA marker-gene survey bioinformatic methods and will help scientists identify appropriate analysis methods for their marker-gene surveys.

In this Behind the Paper post, we look at how we identified and corrected an error while publishing our paper, A framework for assessing 16S rRNA marker-gene survey data analysis methods using mixtures (Olson et al., 2020). During peer-review, we discovered an error in our sample labels when generating a figure requested by a reviewer. Diligent documentation, developing our analysis in a reproducible manner, and the peer review process allowed us to quickly identify, verify, and correct the error. We hope that the lessons learned here will help other scientists prevent, identify, and correct similar errors.

We described our initial mixture and sequencing experimental design in a study protocol document. We developed the initial protocol to minimize the impact of potential sample cross-contamination and propagation of pipetting biases (Fig. 1A). However, the execution of the protocol was not practical. We revised the protocol based on feedback from the co-authors performing the laboratory experiments. Unfortunately, the sample sheet was not updated to reflect these changes. Due to our collaborator’s diligent documentation of the laboratory procedure, we have photographic evidence of the actual sample layout (Fig 1B).

A reviewer requested a PCA analysis of all the samples and titrations to show that the titrations were behaving as expected (Fig. 2A). We generated similar figures during our initial exploratory data analysis but only on individual samples and the expected titration trends were observed. When we plotted the titration series for all five individuals on a single plot, we identified an error in our sample sheet. The plot revealed that the unmixed post-exposure samples were incorrectly grouping for four of the five individuals. From the plot, we hypothesized that we assigned the unmixed post-exposure samples to the wrong individuals.  Our hypothesized sample IDs were consistent with the plate layout photo taken in the lab. (Fig. 1B). We corrected the sample IDs and re-generated the PCA analysis, resulting in the expected trend with samples grouping by individual and titration (Fig. 2B). By developing our analysis pipeline in a reproducible manner under version control we quickly reproduced the study results and evaluated the impact of the error on the study conclusions. 

Figure 1: Titration series (A) experimental design plate layout and (B) image of the titration series plate layout for the titration series taken after sample mixing and prior to the initial 16S rRNA PCR.

 

Figure 2: PCA of benchmarking dataset with color indicating titration and shape is individual or subject for the (A) uncorrected sample labels and (B) corrected sample labels. Before sample label correction the unmixed pre-treatment samples, titration 20, are grouped with the incorrect individual for four of the five individuals, E01JH0038 was not effected by the sample sheet error. Whereas, after sample correction the pre-treatment samples were correctly grouped.

It is our hope that documenting and reporting this error will help prevent other researchers from making these same mistakes. While the peer-reviewer process succeeded in identifying the error prior to publication, diligent documentation and a reproducible analysis pipeline allowed us to quickly correct the error and re-run the full analysis pipeline. Let this experience serve as a reminder that mistakes happen in science, and so, it is important to consider preventative measures as a part of any initial study design, perform analyses like PCA that can identify potential errors, and include sample label verification in the protocol.

Reference

Olson, N.D., Kumar, M.S., Li, S. et al. A framework for assessing 16S rRNA marker-gene survey data analysis methods using mixtures.. Microbiome 8, 35 (2020). https://doi.org/10.1186/s40168...

Please sign in or register for FREE

If you are a registered user on Research Communities by Springer Nature, please sign in

Follow the Topic

Microbiology
Life Sciences > Biological Sciences > Microbiology
  • Microbiome Microbiome

    This journal hopes to integrate researchers with common scientific objectives across a broad cross-section of sub-disciplines within microbial ecology. It covers studies of microbiomes colonizing humans, animals, plants or the environment, both built and natural or manipulated, as in agriculture.

Related Collections

With Collections, you can get published faster and increase your visibility.

Harnessing plant microbiomes to improve performance and mechanistic understanding

This is a Cross-Journal Collection with Microbiome, Environmental Microbiome, npj Science of Plants, and npj Biofilms and Microbiomes. Please click here to see the collection page for npj Science of Plants and npj Biofilms and Microbiomes.

Modern agriculture needs to sustainably increase crop productivity while preserving ecosystem health. As soil degradation, climate variability, and diminishing input efficiency continue to threaten agricultural outputs, there is a pressing need to enhance plant performance through ecologically-sound strategies. In this context, plant-associated microbiomes represent a powerful, yet underexploited, resource to improve plant vigor, nutrient acquisition, stress resilience, and overall productivity.

The plant microbiome—comprising bacteria, fungi, and other microorganisms inhabiting the rhizosphere, endosphere, and phyllosphere—plays a fundamental role in shaping plant physiology and development. Increasing evidence demonstrates that beneficial microbes mediate key processes such as nutrient solubilization and uptake, hormonal regulation, photosynthetic efficiency, and systemic resistance to (a)biotic stresses. However, to fully harness these capabilities, a mechanistic understanding of the molecular dialogues and functional traits underpinning plant-microbe interactions is essential.

Recent advances in multi-omics technologies, synthetic biology, and high-throughput functional screening have accelerated our ability to dissect these interactions at molecular, cellular, and system levels. Yet, significant challenges remain in translating these mechanistic insights into robust microbiome-based applications for agriculture. Core knowledge gaps include identifying microbial functions that are conserved across environments and hosts, understanding the signaling networks and metabolic exchanges between partners, and predicting microbiome assembly and stability under field conditions.

This Research Topic welcomes Original Research, Reviews, Perspectives, and Meta-analyses that delve into the functional and mechanistic basis of plant-microbiome interactions. We are particularly interested in contributions that integrate molecular microbiology, systems biology, plant physiology, and computational modeling to unravel the mechanisms by which microbial communities enhance plant performance and/or mechanisms employed by plant hosts to assemble beneficial microbiomes. Studies ranging from controlled experimental systems to applied field trials are encouraged, especially those aiming to bridge the gap between fundamental understanding and translational outcomes such as microbial consortia, engineered strains, or microbiome-informed management practices.

Ultimately, this collection aims to advance our ability to rationally design and apply microbiome-based strategies by deepening our mechanistic insight into how plants select beneficial microbiomes and in turn how microbes shape plant health and productivity.

This collection is open for submissions from all authors on the condition that the manuscript falls within both the scope of the collection and the journal it is submitted to.

All submissions in this collection undergo the relevant journal’s standard peer review process. Similarly, all manuscripts authored by a Guest Editor(s) will be handled by the Editor-in-Chief of the relevant journal. As an open access publication, participating journals levy an article processing fee (Microbiome, Environmental Microbiome). We recognize that many key stakeholders may not have access to such resources and are committed to supporting participation in this issue wherever resources are a barrier. For more information about what support may be available, please visit OA funding and support, or email OAfundingpolicy@springernature.com or the Editor-in-Chief of the journal where the article is being submitted.

Collection policies for Microbiome and Environmental Microbiome:

Please refer to this page. Please only submit to one journal, but note authors have the option to transfer to another participating journal following the editors’ recommendation.

Collection policies for npj Science of Plants and npj Biofilms and Microbiomes:

Please refer to npj's Collection policies page for full details.

Publishing Model: Open Access

Deadline: Jun 01, 2026

Microbiome and Reproductive Health

Microbiome is calling for submissions to our Collection on Microbiome and Reproductive Health.

Our understanding of the intricate relationship between the microbiome and reproductive health holds profound translational implications for fertility, pregnancy, and reproductive disorders. To truly advance this field, it is essential to move beyond descriptive and associative studies and focus on mechanistic research that uncovers the functional underpinnings of the host–microbiome interface. Such studies can reveal how microbial communities influence reproductive physiology, including hormonal regulation, immune responses, and overall reproductive health.

Recent advances have highlighted the role of specific bacterial populations in both male and female fertility, as well as their impact on pregnancy outcomes. For example, the vaginal microbiome has been linked to preterm birth, while emerging evidence suggests that gut microbiota may modulate reproductive hormone levels. These insights underscore the need for research that explores how and why these microbial influences occur.

Looking ahead, the potential for breakthroughs is immense. Mechanistic studies have the power to drive the development of microbiome-based therapies that address infertility, improve pregnancy outcomes, and reduce the risk of reproductive diseases. Incorporating microbiome analysis into reproductive health assessments could transform clinical practice and, by deepening our understanding of host–microbiome mechanisms, lay the groundwork for personalized medicine in gynecology and obstetrics.

We invite researchers to contribute to this Special Collection on Microbiome and Reproductive Health. Submissions should emphasize functional and mechanistic insights into the host–microbiome relationship. Topics of interest include, but are not limited to:

- Microbiome and infertility

- Vaginal microbiome and pregnancy outcomes

- Gut microbiota and reproductive hormones

- Microbial influences on menstrual health

- Live biotherapeutics and reproductive health interventions

- Microbiome alterations as drivers of reproductive disorders

- Environmental factors shaping the microbiome

- Intergenerational microbiome transmission

This Collection supports and amplifies research related to SDG 3, Good Health and Well-Being.

All submissions in this collection undergo the journal’s standard peer review process. As an open access publication, this journal levies an article processing fee (details here). We recognize that many key stakeholders may not have access to such resources and are committed to supporting participation in this issue wherever resources are a barrier. For more information about what support may be available, please visit OA funding and support, or email OAfundingpolicy@springernature.com or the Editor-in-Chief.

Publishing Model: Open Access

Deadline: Jun 16, 2026