next up previous contents
Next: Least Median of Squares Up: Robust Estimation Previous: Regression Diagnostics



One popular robust technique is the so-called M-estimators. Let tex2html_wrap_inline3528 be the residual of the tex2html_wrap_inline3562 datum, the difference between the tex2html_wrap_inline3562 observation and its fitted value. The standard least-squares method tries to minimize tex2html_wrap_inline3566 , which is unstable if there are outliers present in the data. Outlying data give an effect so strong in the minimization that the parameters thus estimated are distorted. The M-estimators try to reduce the effect of outliers by replacing the squared residuals tex2html_wrap_inline3568 by another function of the residuals, yielding


where tex2html_wrap_inline3570 is a symmetric, positive-definite function with a unique minimum at zero, and is chosen to be less increasing than square. Instead of solving directly this problem, we can implement it as an iterated reweighted least-squares one. Now let us see how.

Let tex2html_wrap_inline3572 be the parameter vector to be estimated. The M-estimator of tex2html_wrap_inline2527 based on the function tex2html_wrap_inline3576 is the vector tex2html_wrap_inline2527 which is the solution of the following m equations:


where the derivative tex2html_wrap_inline3584 is called the influence function. If now we define a weight function


then Equation (29) becomes


This is exactly the system of equations that we obtain if we solve the following iterated reweighted least-squares problem


where the superscript tex2html_wrap_inline3588 indicates the iteration number. The weight tex2html_wrap_inline3590 should be recomputed after each iteration in order to be used in the next iteration.

The influence function tex2html_wrap_inline3592 measures the influence of a datum on the value of the parameter estimate. For example, for the least-squares with tex2html_wrap_inline3594 , the influence function is tex2html_wrap_inline3596 , that is, the influence of a datum on the estimate increases linearly with the size of its error, which confirms the non-robusteness of the least-squares estimate. When an estimator is robust, it may be inferred that the influence of any single observation (datum) is insufficient to yield any significant offset [18]. There are several constraints that a robust M-estimator should meet:

Table 1 lists a few commonly used influence functions. They are graphically depicted in Fig. 4. Note that not all these functions satisfy the above requirements.

Table 1: A few commonly used M-estimators

Figure 4: Graphic representations of a few common M-estimators

Briefly we give a few indications of these functions:

There still exist many other tex2html_wrap_inline3570 -functions, such as Andrew's cosine wave function. Another commonly used function is the following tri-weight one:


where tex2html_wrap_inline3147 is some estimated standard deviation of errors.

It seems difficult to select a tex2html_wrap_inline3570 -function for general use without being rather arbitrary. Following Rey [18], for the location (or regression) problems, the best choice is the tex2html_wrap_inline3754 in spite of its theoretical non-robustness: they are quasi-robust. However, it suffers from its computational difficulties. The second best function is ``Fair'', which can yield nicely converging computational procedures. Eventually comes the Huber's function (either original or modified form). All these functions do not eliminate completely the influence of large gross errors.

The four last functions do not guarantee unicity, but reduce considerably, or even eliminate completely, the influence of large gross errors. As proposed by Huber [7], one can start the iteration process with a convex tex2html_wrap_inline3570 -function, iterate until convergence, and then apply a few iterations with one of those non-convex functions to eliminate the effect of large errors.

next up previous contents
Next: Least Median of Squares Up: Robust Estimation Previous: Regression Diagnostics

Zhengyou Zhang
Thu Feb 8 11:42:20 MET 1996