Control chart procedures have always stressed that any significant or unusual occurrence must be investigated and explained in attempting to bring a process into control; i.e., to have a process with only common cause or error variation in it. As indicated in Montgomery (1996) any significant value indicates that a change has occurred in the process. Whether the change is for the better or for the worse is irrelevant. All are to be investigated. For p-charts, c-charts and R-charts Gitlow, Oppenheim and Oppenheim (1995-chapter 5) state that values below the lower control limit are “good” values in the sense that they indicate that there may have been an improvement in the process. If such is the case and the cause can be identified then making the change part of the process results in a permanent improvement in quality. While the “small” value may represent simply a chance occurrence all other possibilities are to be eliminated before that conclusion is reached.
This philosophy does not seem to be in use in the field of experimental design and statistical analysis in general, particularly in the various tests associated with linear models. All of the F-ratios in linear models with fixed effects are constructed essentially as the mean square (MS) for the effect of interest divided by an estimate of the error variance. If the null hypothesis is true and all assumptions underlying the procedure are satisfied then the F-ratio is expected to be near 1.0. If the null hypothesis is false and all assumptions satisfied then the mean square for the effect of interest contains both an estimate of the error variance and a sum of squared terms attributable to the effect of interest. If the effects are random then the E(MS) for the effect of interest includes the variance for that effect plus a linear combination of variances for various interactions and the error term. The F-ratio then compares the effect’s MS to a MS whose E[MS] is the linear combination of the variances of the various interactions and the error term. Again, if Ho is true the variance of the effect of interest is zero and the F-ratio is expected to be 1.00 while if Ha is true the ratio is expected to exceed 1.00. In a model with mixed effects each term’s E[MS] must be considered individually in determining the F-ratio. In all cases, the only values indicating rejection of the hull hypothesis, or supporting the alternative, are large ones. Values for the F-ratio that are less than 1.0 simply lead to non-rejection of the null hypothesis and generally are not investigated any further, regardless of their actual magnitude. This paper suggests that a significantly small value for the F-ratio should be investigated further to determine if an explanation can be identified.