Sliding filters
A convolution is one small pattern, stamped across the whole image at every position. It runs vision because the same edge matters everywhere.
A dense layer looks at an image only by flattening it: pixel (3, 5) and pixel (200, 5) are just two coordinates with no idea they're both part of "somewhere in the sky". Convolutions instead slide a small weight grid — a kernel, typically 3×3 — across every position of the image, applying the same weights everywhere:
Why weight-sharing fits vision
An edge detector that works in the top-left corner should work in the bottom-right corner — cat fur doesn't switch properties at pixel coordinates. Weight-sharing encodes exactly that: one pattern, stamped everywhere. The consequences are drastic: a 3×3 kernel sees all positions with 9 parameters; a dense layer over a 224×224 image burns 224×224 per unit for the same job.
Hover the output to see which input patch a given stamp aligns with — one kernel, one output map.
click a cell to cycle −1 → +1
input (8×8)
output (6×6)
The kernel stamps the same 3×3 weights at every position: each output cell is ⟨kernel, patch⟩ at that spot. The horizontal edge detector lights a bright line exactly where the diagonal crosses each row.
Real correlation arithmetic: out = Σ kernel·input over each 3×3 window. Illustrative: single-channel, no padding/bias/ activation, one layer — real conv nets stack dozens of these, in parallel channels, and everything you see here is what each of those kernels *is doing*.
What kernels learn
Kernels aren't chosen; they're discovered by gradient descent. Feed a convnet images and the first-layer kernels come out looking like classic hand-crafted filters — edge detectors of various orientations, color blobs, a "stroke detector". Later layers take those outputs as input, so second-layer kernels detect corner-or-orient double-strokes, and so on up a hierarchy: edges → textures → parts → objects. Nobody programs that ladder; it's what stacked folds want to learn when the input is locally repetitive.
The three knobs
Convolutions add exactly three design choices to a network:
- Kernel size — 3×3 dominated because "two 3×3 layers see the same span as one 5×5, with fewer parameters and a nonlinearity in between".
- Stride — move the stamp more than one pixel at a time; downsampling built into the operation itself.
- Channels — each kernel maps a multi-channel input (RGB, or the entire previous layer's 64 outputs) to one output channel; layers are banks of them.
Illustrative vs real
The artifact runs an actual correlation: an editable kernel over a fixed grayscale patch, convolved live. Missed here on purpose: channel stacking, padding edges, and bias/activation per output — none change the core idea that one little pattern explains the whole output map.
Where next: pooling-and-hierarchies — those output maps only grow from here; the trick is shrinking them while throwing away nothing that matters.
This lesson has exercises attached — predicting convolution output sizes and edge responses — launching once the exercises layer ships.