I worked on signal transduction in breast cancer for my Ph.D. thesis study. At that time, all I cared about was the bands in gels, either Western or PCR. I used nude mice to test the gene therapy I developed. The answer was always tumor growth delayed or not.
In 2005, I started my postdoc research to generate syngeneic mouse models of melanoma progression for preclinical studies. To my horror, the tumor growth pattern in these mice was so heterogeneous that I did not even know how to measure the results. Over time, I found this was just the nature of such models, and some techniques like Kaplan-Meier analysis were useful to give them measurable, comparable results.
The diversity of tumor growth pattern still bothered me. However, after looking the outcomes of so many results, I noticed that, although I could not predict how tumor will grow in a specific mouse, the distribution of growth pattern of tumor in mouse cohort was very consistent. For example, tumors will reach endpoint within one month in one model, and 2-3 months in another; or, there are always 20% of tumor reaching endpoints in the earliest time point, 50% in the middle, and 30% in the latest time points. One interesting observation is that tumor growth rate is like an intrinsic feature of a model, no matter how diverse its growth pattern in a mouse cohort.
I read recent comments on the reproducibility of preclinical studies; many focused on the super strict control in everything, which would drive everyone nut until no meaningful job could be accomplished. In contrast, I hardly see any discussion on the reproducibility of heterogeneous pattern itself. To me, that's the real, natural feature of a model in regard of biological complexity. It's impossible to put every factor under control. However, it's possible to understand the critical factors in reproducing the diverse pattern.
It's not a new idea. Someone had explained this very well: while one cannot foresee the actions of a particular individual, the laws of statistics as applied to large groups of people could predict the general flow of future events. You can understand the complexity of biology through the power of cohorts.
By the way, that wise guy's name is Issac Asimov. He mentioned the idea in a book with the title, "Foundation".
Modeling, in a sense, is a type of sampling from the real world. For example, when testing a drug in a mouse model, you choose a mammalian body plan to represent billion of other body plans to test the effect of a compound. When running a clinical trial, you sample a small cohort to represent the whole Hunan population.
Since it is sampling, there is inherent variation. Even for a specific model built on inbred mice, variation exists among individual mice due to style difference in diet, microbiome, hormone level, social interaction, etc. For clinical trials, the origins of variation are even more complicated. All these complexity are overwhelming. The current conversation focuses on comprehensive search for all variables. Such approach simply intimidate researchers not to take reproducibility in study seriously.
These variations are difficult to control. However, that does not mean it's impossible to interpret the results. First of all, since it is a sampling process, each study is a game of number. The big the number is, the more powerful the explanation is. Second, making conclusion from comparison within the group is very important, because it can minimize lots of environmental variations. For the second point, we have to apply the concept, "dimensionless constant".
Dimensionless constant refers to the quality that remains the same in different scale, such as smaller and larger population. For example, the gene expression difference among tumors of patient cohort is similar to that among cell population within a tumor. Such difference is likely caused by the same kind of driving factors.
This approach allows analysis of even single experiment. For example, tumors in a mouse model respond diversely. Studying the driver of such diversity can give us important clue of the cause of therapeutic resistance, assuming it is a dimensionless factor that also exist in patient cohorts.
Recently I learned about "DNA activation" from a seminar: siRNA is not always silencing gene expression. In some cases, the dsRNA-bound Ago protein can bind to the promoter of a gene to activate its transcription. This novel mechanism has been used to design a therapy targeting macrophages in a clinical trial of liver cancer.
It's a fantastic discovery showing great promise in basic and clinical research. After the seminar, I recalled that I have been seeing this in several siRNA screening experiments. However, every time I only considered the increased gene expression by the siRNA as another failure of siRNA design or experimental variation, never really thought about it could be the opposite effect.
For so many times we are locked in the current paradigm. When seeing new results, we are either ignorant or afraid of reviewer's criticism. I wonder how many new discoveries we have missed because we intentionally ignore negative results. In publishing or perishing we bow.