Selective elimination of residual error can be used when applying Harrington's ANOVA-PCA in order to improve the capabilities of the method. ANOVA-PCA is sometimes unable to discriminate between levels of a factor when sources of high residual variability are present. In some cases this variability is not random, possesses some structure and is large enough to be responsible for the first principal components calculated by the PCA step in the ANOVA-PCA. This fact sometimes makes it impossible for the interesting variance to be in the first two PCA components. By using the proposed selective residuals elimination procedure, one may improve the ability of the method to detect significant factors as well as have an understanding of the different kinds of residual variance present in the data. Two datasets are used to show how the method is used in order to iteratively detect variance associated with the factors even when it is not initially visible. A permutation method is used to confirm that the observed significance of the factors was not accidental.