Sunmin.Kim
statistics · ISLR 4.8.3 · 2026-10-17

Why QDA Boundaries Curve

While working through ISLR exercise 4.8.3 for my ML class, I realized LDA quietly makes one assumption: every class shares the same variance. This note is me convincing myself, with pictures, why dropping that single assumption bends the decision boundary.

The question

One feature X, K classes. Class k generates data from a normal distribution N(μk, σ²k). The LDA derivation in ISLR 4.4 assumes σ²₁ = … = σ²K. Drop that assumption and show the Bayes classifier is quadratic, not linear.

1. Equal variances: one crossing, so a line

Say class A and class B have bell curves of the same width. The classification rule is simple: at any x, the taller curve wins. Equal widths means the two curves cross exactly once, and that crossing is the boundary. One boundary point means everything left of it is A and everything right of it is B. That is a linear classifier — LDA.

Curves of equal width meet only once. Left of the crossing A is always taller; right of it B is.

2. Different variances: the boundary splits in two

Now make A narrow and B wide. Drag the slider below and watch the boundary.

The key scene is the far left. As x moves left, away from A's mean, the narrow A curve crashes to zero quickly, but the wide B curve decays slowly. So at some point B becomes taller again. Result: two boundary points, and the regions split as B | A | B. No single point (no line) can draw that shape. An equation with up to two roots is a quadratic.

Drag σB down to 1.00 to match A's width: the left boundary escapes off-screen and only one boundary remains. That moment is exactly LDA.

3. The same story in math

The Bayes classifier picks the class with the largest πk·fk(x). Taking logs does not change which one is largest, so we can expand each class's "score" δk(x) = log(πk fk(x)). Plugging in the normal density:

δk(x) = −1⁄2σ²k · x² + μk⁄σ²k · x − μ²k⁄2σ²k − log σk + log πk

Everything else is a constant. The whole argument lives in one place: the x² coefficient, −1/(2σ²k).

Figure 3 shows exactly this. Plot the score difference δA(x) − δB(x): with equal variances it is a straight line crossing zero once; with different variances it is a parabola crossing zero twice.

Classify as A wherever δA−δB > 0. The dashed line (equal variances) crosses zero once; the red parabola (different variances) crosses twice. Zero crossings are boundary points.

4. One-line takeaway

Set σ²k = σ² and the x² term vanishes, recovering LDA's linear discriminant. LDA is a special case of QDA, and the "equal variances" assumption is the one thing that was keeping the boundary straight.

What I'll remember: when someone says "QDA is more flexible than LDA," the entire difference is a single subscript on σ.