Anomaly detection
The rare, the weird, the never-seen-in-training. You can't learn from examples you don't have — so model 'normal' and flag the edges.
Anomalies are the classification problem that can't be solved as one: you don't have labeled fraud examples until you've been defrauded, and each new fraud looks like the last ones didn't. The classical move: model what normal looks like from abundant clean data, then flag whatever looks unlike it. One-class by construction.
The guard who's never met a criminal
Train only on normal days. Anything that looks unlike a normal day — too far out, too sparse-around, too statistically-compressed — gets flagged. You never needed examples of crime, only a comfortable definition of calm.
Click to place an event; score = how unlike “the ring” it is
flagged: 3/3
Accent score rings = accepted as normal-ish, orange = flagged. Lower the threshold and review queue explodes — the same precision/recall invoice as the metrics chapter.
The model never saw anomalies — there are none in its training set by design. “Normal” is the dashed band; score is distance, output is a flag, the human decision is the threshold. One-class at the start, precision/recall dial at both ends, and an unfixable drift hazard: block flagged fraud and flagged fraud stops arriving to teach you anything new.
The toolbox, honestly graded
- Statistical fences (z-scores, percentiles): one line of code and defensible to auditors. Assumes roughly normal distributions; breaks on multi-modal data (two populations stacked on top of each other produce "no anomalies" in the middle and false corners at the edges).
- Density methods (LOF, the artifact's percentile ring): flag points whose neighborhoods are unusually empty. Better on irregular shapes, harder to explain in a sentence.
- Isolation Forest: a random forest whose trees actively cut the space apart — anomalies get isolated in a few cuts, normal points take many (the forest's average path length is the anomaly score). The one method that scales like trees do.
- Reconstruction methods (autoencoders — the deep-bad cousin): train to reconstruct normal data; anomalous inputs reconstruct badly. This is the route that scales to the LLM book's territory: perplexity gating, trained on normal text, flags out-of-distribution prompts.
The unfixable data problem
What you flag, someone reviews. Threshold at 1%, you flood reviewers with 99% noise — chapter 3's precision/recall dial lives here too, and its cost figures still belong to people. Worse: feedback loops. Block the flagged traffic and your "normal" training data gets quietly cleaner — the anomaly detector starts grading its own homework (test set drift, now with legal liability), which is why production systems keep replaying and re-validating rules against human-labeled outcomes.
Illustrative vs real
One ring of "normal" and manual outliers here. Real elastic systems run thousands of monitors; the inevitable SRE experience is that the hardest anomalies are not system outliers but process outliers — watch the "everything is 30% slower since Tuesday" shape.
Where next: embeddings and similarity — the representation layer underneath all this.