Matrices: transformations and data

A spreadsheet of students against subjects is a matrix. So is the weight table of a neural network layer, the covariance of a dataset, and a grayscale image. All of these really are just rectangular arrays of numbers, but the reading that unlocks linear algebra is different: a matrix is a function that takes a vector in and returns a vector out, moving the entire space at once.

Drag the four entries below and watch the grid deform. The two solid arrows are the columns — the destinations of the basis vectors $(1,0)$ and $(0,1)$.

The two readings of a matrix #

A matrix $A \in \mathbb{R}^{m\times n}$ can be read two ways, and fluency means being able to switch between them:

• Rows as measurements. Each of the $m$ rows is a weighting of the $n$ inputs. Multiplying $A x$ computes $m$ inner products — a score per row. This is the reading of a classifier: row $i$ scores how much the input looks like class $i$.
• Columns as directions. The $n$ columns are vectors in $\mathbb{R}^m$. The product $A x$ is a weighted sum of those columns, with weights $x_1, \dots, x_n$. This is the reading that explains span and rank.

$A x = x_1\, a_{:,1} + x_2\, a_{:,2} + \cdots + x_n\, a_{:,n}.$

The matrix–vector product costs $O(mn)$: $m$ dot products, each over $n$ entries. A matrix–matrix product $AB$ costs $O(mnk)$ because each of the $k$ columns of $B$ is transformed separately — composition is just applying two maps in sequence.

Rank, span, and the column space #

The set of all outputs $A x$ as $x$ ranges over every input is the column space (or range) of $A$. Its dimension is the rank. Rank is the amount of genuinely independent information the map can carry:

• An $m\times n$ matrix has rank at most $\min(m, n)$.
• Rank $= n$ means distinct inputs give distinct outputs (the columns are independent; no information is collapsed).
• Rank $< n$ means some non-zero $x$ is sent to zero. That input is in the null space, and it is unrecoverable.

This is why a determinant of zero matters. The determinant is the signed area (or volume) by which the map scales space; zero means the whole plane has been squashed onto a line. Volumes collapse, inverse maps stop existing, and a model built on such a matrix is not identifying its parameters.

Special matrices worth recognizing #

• Identity $I$ — leaves every vector unchanged; the map that does nothing.
• Diagonal — stretches each axis independently; cheap to invert and multiply.
• Symmetric $A = A^\top$ — the eigenvectors come out perpendicular with real eigenvalues, which is why covariance matrices are so well behaved.
• Orthogonal $Q^\top Q = I$ — a rotation or reflection; it preserves lengths and angles, so it never amplifies error.
• Positive definite — for every non-zero $x$, $x^\top A x > 0$; these are the matrices that define genuine squared distances.

What matrices are not #

Multiplying numbers is commutative; multiplying matrices is not. $AB$ generally differs from $BA$, and that is not a technicality — it says that applying one transformation and then another depends on the order. A rotation followed by a stretch is a different map from the stretch followed by the rotation.

Check yourself

Eduspheria wiki · Mathematics for AI, Linear algebra

0 / 5 answered

1. 1What is the determinant of M = [[2, 1], [1, 2]]?

   Check Hint Show answerNumeric answer
2. 2The columns of a matrix are best described as which of these?

   A.The eigenvalues of the matrixB.The images of the basis vectorsC.The null space of the matrixD.The rows of the inverse matrix

   Check Show answerMultiple choice
3. 3What is the name of the subspace containing every vector of the form A·x?

   Check Show answerShort answer
4. 4If det A = 0, then A is not invertible.

   TrueFalse

   Check Show answerTrue / false
5. 5A 4×3 matrix has rank 2. How many dimensions does its null space have?

   Check Hint Show answerNumeric answer

Where next: linear systems — when the equations $Ax = b$ have a solution, and what to do when they do not.