Unstructured data
Text, images and audio do not arrive as columns. Encoding them into vectors is the bridge between raw content and every model downstream.
A support ticket, an X-ray, a call recording, a scanned invoice. Most of the data an organisation accumulates is unstructured, and none of it has columns. Models cannot consume raw text or pixels directly; they consume vectors. The job of an encoder is to map unstructured input into a numeric space where distance means something — so that "similar" in the vector space corresponds to "similar" for the task.
The artifact starts with two deliberately old-fashioned encoders — a bag-of-words term-frequency vector for text and a flattened pixel vector for images — because they expose the pipeline every modern encoder still follows: tokenise or patch, count or measure, vectorise, and compare by cosine.
Raw input → a numeric vector → a similarity score
Sparse vector over a 30-term vocabulary:
Cosine similarity to corpus
These are honest 20th-century encoders — bag-of-words term frequency and a raw pixel vector — not learned embeddings, and the toy corpus is tiny. The unglamorous truth is that an embedding model is still just this: unstructured input mapped to a vector, with similarity read off a cosine. The model changes; the pipeline does not.
An encoder is a lossy map with a purpose
No vector captures everything about a paragraph or a photograph. An encoder is good when the information it discards is the information your task does not need. Bag-of-words throws away order and keeps vocabulary; a convolutional network keeps local texture and builds up to objects. The choice of encoder is a choice of what to throw away.
Text: from tokens to vectors
The classical pipeline is tokenise → build a vocabulary → count → weight. The simplest representation is a term-frequency vector, one dimension per vocabulary word, and similarity is cosine:
Two refinements carry most of the weight. TF-IDF down-weights terms that appear in many documents, since a word everywhere is a word that discriminates nothing:
And dense embeddings replace the sparse count vector with a learned low-dimensional one — word2vec, GloVe, or the contextual vectors from a transformer — so that synonyms end up near each other instead of orthogonal, as they are in the one-hot world. The vocabulary is fixed at encoding time, which is why subword tokenisation exists: it lets a model represent a word it never saw in training by composing its pieces.
Images and audio: grids and windows
An image is a grid of intensities, so the raw encoder is just a flattened vector. The problem is dimensionality and invariance: a 224×224 colour image is 150,528 numbers, and every pixel-level similarity is broken by a one-pixel shift. Convolutional encoders fix both by sliding a small learned filter across the image — weight sharing cuts the parameter count and builds in translation equivariance — and stacking layers composes edges into textures into objects. The penultimate layer's activations become the embedding used for retrieval and transfer. Audio follows the same recipe on a spectrogram, or a sequence model processes frames directly.
The bridge to everything else
Once content is a vector, the rest of this domain applies unchanged: features for a model, similarity for search and recommendations, and — in the multimodal case — a shared space where a caption and an image can be compared directly. Retrieval-augmented generation, semantic search, and zero-shot classification are all built on this one move: put the unstructured thing into the same vector space as the query.
Careful
Unstructured data brings governance problems structured data does not. Text and images contain personally identifying information that is hard to detect and easy to leak; scraped corpora carry licence and copyright constraints; and an encoder trained on historical data inherits its biases. Token counts also translate directly into cost and latency, so encoding choices are budget choices. Treat the encoder as part of the data supply chain, with the same quality and compliance scrutiny.
Illustrative vs real
The text encoder in the artifact is a term-frequency bag of words over a four-document corpus, and the image encoder is a 25-pixel binary grid matched against three prototypes. Neither understands meaning — the point is the shape of the pipeline. A production encoder is a pretrained transformer or convolutional network with millions of parameters, but it emits a vector and is compared by cosine exactly like these.
Check yourself
Eduspheria wiki · Data, MLOps & Deployment, Data engineering
0 / 4 answered
From the assignment paper
Modeled on NITJ AI-505, Assignment/Quiz
0 / 6 answered
Where next: the ML lifecycle — how data, models and operations fit into one loop that runs for years.