Statistics

Least-Squares Regression Line

A student compares vertical gaps between data points and a line.

Least-Squares Regression Line definition

The least-squares regression line minimizes the sum of squared vertical differences between observed and predicted responses.

In Spanish
recta de regresión por mínimos cuadrados
Topic
Two-variable data

Least-Squares Regression Line in a picture

Least-Squares Regression Line diagramThe dashed segments represent vertical residuals. Squaring them gives the quantity used to fit the line.
The dashed segments represent vertical residuals. Squaring them gives the quantity used to fit the line.

How it works

For each observation, the residual is y − ŷ. Square the residuals and add them to measure the line’s error. With an intercept and varying x-values, the fitted line passes through (mean x, mean y). Its slope equals the sum of cross-products of centered values divided by the sum of squared centered x-values.

Worked examples

Residuals are −1, 2, and −1. Find the squared error sum.

1 + 4 + 1 = 6.

For (1, 2), (2, 4), (3, 6), find the least-squares line.

ŷ = 2x gives zero residuals, the smallest possible sum.

The means are x̄ = 4 and ȳ = 11, and the fitted slope is 2. Find the intercept.

a = 11 − 2(4) = 3.

Common mistake

Measure residuals vertically, in the response variable’s units. Ordinary least squares does not minimize perpendicular distances.

Quick check

Why square residuals?

Show answer

Positive and negative errors cannot cancel after squaring.

What is the smallest possible squared error sum?

Show answer

Zero.

Which mean point lies on a least-squares line fitted with an intercept?

Show answer

(x̄, ȳ).

Sources

Flashcard

Least-Squares Regression Line

Reveal the definition

The least-squares regression line minimizes the sum of squared vertical differences between observed and predicted responses.

Effortless Math. “Least-Squares Regression Line.” Math Dictionary.