Sources of variation in 'omics experiments
Metabolomics and proteomics are essentially comparison technologies, applied to populations. In a well-designed experiment, differences between populations are identified which can be ascribed to the phenomenon being studied (e.g. diseased state in comparison to healthy state). The primary challenge in these types of experiment is to discriminate between variation arising from the subject of the study and other coincidental variation. The aim is to produce a well-matched case-control study of sufficient statistical power (essentially, number of subjects) to allow meaningful conclusions to be extracted.
Coincidental variation can arise from two sources
Technical variation can be caused by variable extraction efficiency or “instrument drift” during the analytical procedure; or by pre-analytical factors, such as varying background from different batches or sources of consumables used for sample collection; or by interferences from inappropriate sample collection or handling. Technical variation can usually be controlled quite well when an experiment has been designed from scratch, for a specimen type we have previously studied (serum, soft tissue), and with a limited number of scientists collecting the material using pre-validated consumables to a defined method over a limited time period.
In contrast, “rare samples” collected when available over extended periods, archived material collected for other purposes, or material derived from several collections or several experiments, will almost certainly result in uncontrolled variation being introduced with the potential to confound or completely invalidate the study. An important distinction should therefore be drawn between a designed experiment and an assembled experiment – ‘omics studies based on sample sets assembled from existing material have a high probability of failure due to confounding effects.
Biological variation arises from a wide variety of sources including genetic variation, environmental factors and behavioural factors. Proper case-control matching (e.g. for age and gender) and adequate sample numbers to allow “typical” ranges (and outlier subjects) to be defined are key. Comprehensive “metadata” can be vital to avoid misinterpretation of findings (by allowing bias or stratification of the sample set to be identified), but needs to be rigorously studied prior to commencing the analysis and used to decide whether to proceed. “Interesting” single samples attached to an experiment are not interpretable – do the metabolite or protein differences represent consequences of a unique disease state or of that individual’s idiosyncratic diet (etc.). Animal experiments are generally less problematic than human studies due to the high levels of control possible (although even in animal studies, we have seen a population clearly discriminated by which cage a subject was housed in (see PCA plot)). Other workers have demonstrated instability in the performance of statistical significance tests when sample numbers fall below 10, and so we would recommend a minimum of 12-15 subjects per class for an animal experiment. Human studies are considerably more problematic, due to very high levels of inter-subject variability in genetic background, diet, medical intervention and lifestyle/behavioural factors, and at present we would regard all human studies as high risk.
Due to technical constraints, our current metabolomics capability is to conduct single batch experiments of ca 50 subjects. Although it has been asserted that multiple batches can be assembled into a single data set, the current evidence is that this remains problematic. A strategy based on using higher throughput targeted methods in larger human populations to test animal-model discovery metabolomics might prove more practical than using our current untargeted methods in multi-batch experiments.
Citrate plasma, EDTA plasma and gel separator blood collection tubes are known to be problematic due to contamination and/or interference with our analytical methods. Certain common chemicals present in the laboratory, such as polyethylene glycol (PEG), will also cause problems. There is some evidence that supplier and batch may affect the suitability of blood collection tubes. Human aorta is mechanically incompatible with our current extraction method.