Clustering: k-means
No labels, no loss you can verify — just structure hiding in the data. The algorithm that finds it, and the reasons it'll find the wrong structure.
Everything so far had answers attached — that's what "supervised" means. Drop them. A pile of unlabeled customer records still has structure: which users resemble which. Clustering is the family of methods that finds structure nobody labeled, and k-means is its workhorse.
The loop in one breath
Guess k centers. Assign every point to its nearest center. Move each center to the average of its members. Repeat — assignments stop changing when the centers settle. That's it.
Unlabeled points; the loop writes its own labels
inertia: —
three blobs: k=3 recovers them; k=4 invents a split you'll accidentally believe
Crosses = centers, dragged to member averages each iteration; iterations stop when no assignment changes. Reseed to watch a different start settle differently — same data, different local optimum. Inertia is the “loss”, but unlike chapter 1’s losses, no one can tell you the numbers are right about anything: no labels, no ground truth, only structure you now own responsibility for.
Reading the loop honestly
- It minimizes something real: total squared distance from points to their center (the "inertia" readout). It is gradient-descent's coordinate descent cousin — each step provably goes downhill, so it always converges.
- To a local optimum. Re-run it — the artifact's rescan button — and different starting centers can mean settling on a visibly worse split. Real practice runs many restarts and keeps the best, exactly like gradient descent with multiple random initializations.
- k is yours to guess, and it can be wrong. There's no bell — only trade-off curves (elbow method, silhouette) that suggest. Two plausible clusterings of one dataset is a normal crossing, not a malfunction: "how many kinds of customers do we have?" is a business judgment the algorithm will happily express either way.
The failures the geometry dictates
k-means assumes spherical, similarly-sized clusters — it's a metric distance around average-points, so it cannot carve two moons, rings, or one-small-inside-one-big (see the moons preset). It also bends to unit scaling like every distance method (chapter 3's warning, mandatory here). When shapes are weird, alternatives (DBSCAN, spectral) carve by connectivity or graph structure instead.
Illustrative vs real
Real clustering runs k-means++ seeding, many restarts, scaler fit per CV fold, and — increasingly — represents items by embeddings and clusters those, which is this lesson's endpoint.
Where next: PCA — throwing away dimensions you never had labels for.