Wiki
Intro10 min read

Image formation and the pinhole camera

How a 3-D scene becomes a 2-D array of pixels — the projection, focal length and field of view that every vision system has to invert.

A photograph is a shadow caught in a box. Light leaves every point of the scene in every direction; a camera with a small opening lets through a single ray per scene point, and wherever that ray strikes the sensor is the pixel value we record. The whole of computer vision is then an attempt to run this process backwards — to recover something about the three-dimensional world from a flat grid of numbers.

The simplest and still the most useful model of that process is the pinhole.

One ray per point

Put a pinhole between the scene and a wall and each scene point contributes exactly one ray to exactly one wall location. That one-to-one mapping is what makes an image sharp. Widen the hole and many rays from one point land in many places — the image blurs. A lens is the trick that collects many rays per point and bends them back together.

A point at height YY and depth ZZ in front of the hole lands at signed height y=−f Y/Zy = -f\,Y/Z on an image plane a distance ff behind it. Move the sliders and the picture, and the numbers, update together.

Move the object back and forth, make it taller, change the focal length — the image height follows f·Y / Z exactly.

image planepinholeobject YimageZ = distancef
image height
0.520 m
magnification
-0.400×
field of view
64.0°
inverted?
yes

f < Z: the image is minified, as in ordinary photography.

The ray from the object tip crosses the pinhole and lands below the axis, so the image is flipped: the signed projection is y = −f·Y/Z. Illustrative: an ideal pinhole with no lens, no focus blur and a unit sensor half-width; a real camera adds aperture, lens distortion and a pixel grid.

From a diagram to a matrix

For a point (X,Y,Z)(X, Y, Z) in the camera's coordinate frame, the perspective projection to pixel coordinates (u,v)(u, v) is

u=fxXZ+cx,v=fyYZ+cy.u = f_x \frac{X}{Z} + c_x, \qquad v = f_y \frac{Y}{Z} + c_y .

The focal lengths fx,fyf_x, f_y (in pixels) and the principal point (cx,cy)(c_x, c_y) are the intrinsic parameters of the camera. Stacked into homogeneous coordinates this becomes a single matrix multiplication,

[uv1]  ∝  [fx0cx0fycy001]⏟K[XYZ].\begin{bmatrix} u \\ v \\ 1 \end{bmatrix} \;\propto\; \underbrace{\begin{bmatrix} f_x & 0 & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{bmatrix}}_{K} \begin{bmatrix} X \\ Y \\ Z \end{bmatrix}.

The symbol ∝\propto is doing real work: the projection is defined only up to scale, because any point along the ray projects to the same pixel. Adding a rotation RR and translation tt that place the world relative to the camera gives the full camera matrix P=K [R∣t]P = K\,[R \mid t].

Field of view follows directly from ff and the sensor half-width ss: FOV=2arctan⁡(s/f)\text{FOV} = 2\arctan(s/f). A short focal length is a wide-angle lens; a long one is a telephoto — the same trade the sliders expose.

What the model throws away

The pinhole is exact for a world with no lenses and no depth. Real cameras bend straight lines near the frame edge (radial distortion), blur out-of-focus points (depth of field), and never integrate exactly one ray per point. Vision pipelines either calibrate these effects away or model them explicitly.

Projection is not invertible

Many scene points map to one pixel, so you can never recover (X,Y,Z)(X, Y, Z) from (u,v)(u, v) alone. Vision recovers relative structure and motion from multiple views; absolute scale needs another cue, a second camera, or a known object size. Any claim of "3-D reconstruction from a single image" is really a learned prior, not geometry.

Illustrative vs real

The widget uses a unit sensor and an ideal pinhole with no lens. A real camera adds a lens with finite aperture, radial and tangential distortion, a Bayer-filtered pixel grid and noise. The projection equation above is exact for the idealised camera and remains the first-order model that every calibration, stereo and structure-from-motion system starts from.

Check yourself

Eduspheria wiki · Applied AI, Classical computer vision

0 / 5 answered

  1. 1An object of height 2 m sits 8 m from a pinhole camera with focal length 10 mm. What is the image height in mm?
    mm
    Numeric answer
  2. 2Which quantity belongs to the intrinsic matrix K rather than the extrinsics?
    Multiple choice
  3. 3Doubling the focal length while keeping the sensor size fixed narrows the field of view.
    True / false
  4. 4Because many 3-D points project to the same pixel, what does a single image lose that multiple views help recover?
    Short answer
  5. 5A camera has f = 50 mm and sensor half-width 18 mm. What is the horizontal field of view in degrees?
    °
    Numeric answer

From the exam paper

Modeled on NITJ AI-502, End-Sem May 2025

0 / 5 answered

  1. 1An 8-pixel grayscale patch has intensities [0, 1, 1, 2, 3, 3, 3, 4]. In the normalized histogram, what is the probability of intensity 3?
    Numeric answer
  2. 2For the same patch [0, 1, 1, 2, 3, 3, 3, 4], what is the normalized probability of intensity 1?
    Numeric answer
  3. 3Which intensity level is most probable in the patch [0, 1, 1, 2, 3, 3, 3, 4]?
    Multiple choice
  4. 4The normalized histogram probabilities of the patch sum to 1.0.
    True / false
  5. 5Which lens distortion is caused by misalignment between the lens and the image sensor rather than by the lens shape?
    Short answer

Where next: with images formed, we look for points that are stable enough to recognise again in another view — corners and the descriptors built around them.