"God does not care about our mathematical difficulties. He integrates empirically." Einstein

Normal Equation

The normal equation solves without iterating for the minima of the cost function. As a result it returns the $\theta$ where the cost function is at its minimum.

Comparison with Gradient Descent

Normal Equation

• No need to choose $\alpha$
• Don’t need to iterate
• Need to compute $(X^TX)^{-1}$
• Slow if $n$ is very large

Gradient Descent

• Need to choose $\alpha$
• Needs many iterations
• Works well even when $n$ is large