The Limitations of the Cochran and Grubbs Outlier Tests in Round Robin Testing

Large round robin tests are often performed by oil companies in order to evaluate the repeatability and reproducibility of the methods used to control the quality of their products. It is very important to identify the laboratories that present statistically non-coherent results (outliers) in order to avoid an unjustified overestimation of the results variability. These round robin tests may involve more than 30 laboratories with an associated risk of more than two laboratories considered as outliers. In this presentation, the classical statistical tests of outliers detection (Cochran and Grubbs’ tests) described in the ISO normative documents used to analyze the round robin tests, are reviewed by using practical examples. We illustrate the difficulties of identifying multiple outliers in the situations of masking effect (2). In that case, there is no statistically grounded justification for the removal of these multiple outliers. Furthermore, these tests were not originally designed to be applied iteratively for the successive removal of outliers. Finally, some simple new algorithms derived from the Student and Fisher statistics are presented and evaluated on the basis of Monte Carlo simulations and true results. Both of these algorithms attempt to provide a simple test which uses known critical values and can detect multiple outliers.