Unusually Small F-Statistic in Analysis of Variance and Regression Analysis: A Warning in Design of Experiments and Regression

Unusually Small F-Statistic in Analysis of Variance and Regression Analysis: A Warning in Design of Experiments and Regression

Ceyhun Ozgur (Valparaiso University, Valparaiso, IN, USA)
Copyright: © 2016 |Pages: 15
DOI: 10.4018/IJBAN.2016070103
OnDemand PDF Download:


All textbooks and articles dealing with classical tests in the context of linear models stress the implications of a significantly large F-ratio since it indicates that the mean square for whatever effect is being evaluated contains significantly more than just error variation. In general, though, with one minor exception, all texts and articles, to the authors' knowledge, ignore the implications of an F-ratio that is significantly smaller than one would expect due to chance alone. Why this is so difficult to explain since such an occurrence is similar to a range value falling below the lower limit on a control chart for variation or a p-value falling below the lower limit on a control chart for proportion defective. In both of those cases the small value represents an unusual and significant occurrence and, if valid, a process change that indicates an improvement. Therefore, it behooves the quality manager to determine what that change is in order to have it continue. In the case of a significantly small F-ratio some problem may be indicated that requires the designer of the experiment to identify it, and to take “corrective action.” While graphical procedures are available for helping to identify some of the possible problems that are discussed they are somewhat subjective when deciding if one is looking at an actual effect; e.g., interaction, or whether the result is merely due to random variation. A significantly small F-ratio can be used to support conclusions based on the graphical procedures by providing a level of statistical significance as well as serving as a warning flag or warning that problems may exist in the design and/or analysis.
Article Preview


Control chart procedures have always stressed that any significant or unusual occurrence must be investigated and explained in attempting to bring a process into control; i.e., to have a process with only common cause or error variation in it. As indicated in Montgomery (1996) any significant value indicates that a change has occurred in the process. Whether the change is for the better or for the worse is irrelevant. All are to be investigated. For p-charts, c-charts and R-charts Gitlow, Oppenheim and Oppenheim (1995-chapter 5) state that values below the lower control limit are “good” values in the sense that they indicate that there may have been an improvement in the process. If such is the case and the cause can be identified then making the change part of the process results in a permanent improvement in quality. While the “small” value may represent simply a chance occurrence all other possibilities are to be eliminated before that conclusion is reached.

This philosophy does not seem to be in use in the field of experimental design and statistical analysis in general, particularly in the various tests associated with linear models. All of the F-ratios in linear models with fixed effects are constructed essentially as the mean square (MS) for the effect of interest divided by an estimate of the error variance. If the null hypothesis is true and all assumptions underlying the procedure are satisfied then the F-ratio is expected to be near 1.0. If the null hypothesis is false and all assumptions satisfied then the mean square for the effect of interest contains both an estimate of the error variance and a sum of squared terms attributable to the effect of interest. If the effects are random then the E(MS) for the effect of interest includes the variance for that effect plus a linear combination of variances for various interactions and the error term. The F-ratio then compares the effect’s MS to a MS whose E[MS] is the linear combination of the variances of the various interactions and the error term. Again, if Ho is true the variance of the effect of interest is zero and the F-ratio is expected to be 1.00 while if Ha is true the ratio is expected to exceed 1.00. In a model with mixed effects each term’s E[MS] must be considered individually in determining the F-ratio. In all cases, the only values indicating rejection of the hull hypothesis, or supporting the alternative, are large ones. Values for the F-ratio that are less than 1.0 simply lead to non-rejection of the null hypothesis and generally are not investigated any further, regardless of their actual magnitude. This paper suggests that a significantly small value for the F-ratio should be investigated further to determine if an explanation can be identified.

Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 4: 4 Issues (2017)
Volume 3: 4 Issues (2016)
Volume 2: 4 Issues (2015)
Volume 1: 4 Issues (2014)
View Complete Journal Contents Listing