Vectors: data as arrows

Every machine-learning model ultimately sees the world as vectors — lists of numbers. An image is a vector of pixel intensities (millions of them). A word in an embedding table is a vector of a few thousand coordinates. A student's record might be [attendance, cgpa, credits]. Learning is geometry on these lists.

Agreement between two vectors #

The single most important operation on vectors is the inner product (dot product): multiply matching entries and sum. It answers "how much do these two point the same way?" Drag the vector tips below and watch the dot product, norms and angle update together.

Three things to notice while you play:

1. The dot product is a projection. Its value is the length of a's shadow on b, times ‖b‖. When the vectors are perpendicular the shadow is zero.
2. The norm is a special case. a · a = ‖a‖², so the dot product already contains the idea of length.
3. Only the angle and lengths matter. Rotate both vectors together and the dot product does not change.

The vocabulary, precisely #

For vectors $a, b \in \mathbb{R}^n$:

• Addition $a + b$ adds entrywise — the parallelogram diagonal.
• Scalar multiplication $\alpha a$ stretches (or flips) the arrow.
• Inner product $a \cdot b = \sum_{i=1}^n a_i b_i$.
• Euclidean norm $\lVert a \rVert = \sqrt{a \cdot a}$, the arrow's length.
• Distance $\lVert a - b \rVert$ — the norm of the difference is the distance between the two points.
• Angle $\cos\theta = \dfrac{a \cdot b}{\lVert a \rVert\,\lVert b \rVert}$. When $\theta = 90^\circ$ the vectors are orthogonal ($a \cdot b = 0$).

Because $\lVert a \rVert$ and $\lVert b \rVert$ are fixed once the vectors are, similarity in high dimensions is almost always reported as cosine similarity $\cos\theta$ — the norm-free part of the dot product. This is why retrieval systems compare embeddings by cosine, not by raw distance: two paragraphs of different lengths can point the same way.

Independence, basis, and why dimension counts #

A set of vectors is linearly independent if none of them can be written as a combination of the others — every added arrow contributes a genuinely new direction. A basis is a minimal independent set that can reach every point in the space; the number of vectors in it is the dimension.

This is the fact behind dimensionality reduction: if your 768-dimensional embeddings actually live near a 40-dimensional subspace, then only 40 basis directions carry signal. Finding those directions is exactly what PCA and the SVD (later this chapter) do.

What it isn't #

Vectors of raw numbers are not automatically comparable. [2 m, 3 kg] and [200 cm, 3 kg] describe nearly the same thing but have very different norms — so "closest by distance" can be an artifact of units, not meaning. The fix (feature scaling) is a practice-chapter topic. The geometric operations are exact; the conclusions you draw from them depend on the coordinates being meaningful.

Check yourself

Eduspheria wiki · Mathematics for AI, Linear algebra

0 / 5 answered

1. 1Let a = (3, 4). What is ‖a‖?

   Check Hint Show answerNumeric answer
2. 2Two non-zero vectors have a · b = 0. What does that mean geometrically?

   A.One has zero lengthB.They are parallelC.They are identicalD.They are orthogonal

   Check Show answerMultiple choice
3. 3Which quantity is the norm-free part of the dot product, used to compare embeddings of different lengths?

   Check Show answerShort answer
4. 4A matrix–vector product with an m×n matrix costs O(mn).

   TrueFalse

   Check Show answerTrue / false
5. 5An embedding is 768-dimensional but lives near a 40-dimensional subspace. How many basis directions carry its signal?

   Check Show answerNumeric answer

Where next: matrices — the operators that move vectors around the space.