Leverage Effect

The term "leverage" is commonly used for an undesirable effect which is experienced with regression analysis (as well as with other methods). It basically means that a single data point which is located well outside the bulk of the data (an "outlier") has an overproportional effect on the resulting regression curve.

The origin of this effect can be found in the method of least squares. As the regression line is determined by minimizing the sum of squared residuals, a value far off the trend line of the data has much more influence on the results as the "correct" data points. This effect may become so strong that the regression line completely "tilts".

Depending on the number of the samples and the distance of the outlier from the rest of the data, this effect may completely corrupt a regression model which would be quite good in the absence of any outliers. Click the figure to start the interactive example.