Wiki
Core13 min read

Medical imaging in 2-D and 3-D

CT, MRI and their neighbours: Hounsfield units, window/level display, tri-planar viewing, and the 2-D and 3-D networks that segment anatomy.

Medical images are not photographs. A CT scan is a three-dimensional map of X-ray attenuation, an MRI is a map of proton behaviour in a magnetic field, and both arrive as stacks of slices whose intensity has physical meaning. Reading them well — and building AI that reads them well — depends on understanding that meaning before any model touches a pixel.

The most important fact is that the numbers are quantitative. In CT, each voxel is in Hounsfield units (HU): roughly −1000-1000 for air, around 00 for water, +40+40 to +80+80 for soft tissue, and upwards of +400+400 for bone. Diagnosis is a matter of which narrow band of that scale you choose to display.

Window and level

A display has 256 grey levels; CT spans thousands of HU. Windowing maps a chosen HU range — centre (level) and width (window) — onto the full display range, clipping everything outside. Change the window and a different tissue becomes visible in the same slice. Radiologists switch windows constantly, and good AI preprocessing does too.

Scrub the slices, change the window, and turn on a maximum-intensity projection.

Scrub through orthogonal slices of a 20³ volume, change the window/level, or switch on a maximum-intensity projection.

axial (slice)

20×20

coronal

20×20

sagittal

20×20

Real voxel sampling and linear window mapping on a procedural phantom. Clinical viewers read DICOM volumes in Hounsfield units and offer the same axial/coronal/sagittal tri-planar and MIP views; 3-D CNNs such as V-Net and U-Net operate on exactly this voxel grid.

From slices to volumes

A CT or MRI study is a volume V(x,y,z)V(x, y, z), and the same anatomy can be viewed along three orthogonal planes: axial (top-down slices), coronal (front-facing) and sagittal (side). Clinical viewers show all three simultaneously, because a lesion is often obvious in one plane and invisible in another. A maximum-intensity projection (MIP) collapses the volume along one axis, taking the brightest voxel per ray — useful for vessels and bone.

display(v)=clamp⁡ ⁣(v−(level−w2)w, 0, 1).\text{display}(v) = \operatorname{clamp}\!\left(\frac{v - (\text{level} - \tfrac{w}{2})}{w},\, 0,\, 1\right).

2-D and 3-D architectures

Early medical CNNs treated each slice as an independent image and classified it. That ignores through-plane continuity, so the field moved to volumetric models:

  • 2-D segmentation, slice by slice (U-Net): an encoder that downsamples to capture context and a decoder with skip connections that recovers resolution, trained per slice. Cheap, but a lesion spanning slices may be split.
  • 3-D segmentation (V-Net, 3-D U-Net): 3-D convolutions over the whole volume, trained with a Dice overlap loss suited to the severe class imbalance of small structures. Expensive in memory, but anatomically consistent.

A middle path processes thick 2.5-D slabs, or treats one axis as channels, trading memory for partial context.

The practical difficulties

  • Class imbalance. A tumour may be a few thousand voxels in tens of millions; loss functions and sampling must compensate.
  • Small datasets. Expert annotation is scarce and expensive, so transfer learning, self-supervision and heavy augmentation are the norm.
  • Domain shift. Scanner vendor, protocol and even day-to-day acquisition change the intensity statistics; a model trained at one hospital can fail at another.
  • Registration. Comparing scans over time, or fusing CT with MRI, requires spatially aligning volumes — a 3-D version of the matching problem from the first chapter.

A model is not a diagnosis

Medical AI is a high-stakes, regulated activity. Sensitivity and specificity must be reported at a clinically chosen operating point on a representative dataset, with subgroup performance shown; a single accuracy or AUC figure is not evidence of safety. Deployment requires prospective validation, monitoring for drift and a clear human-in-the-loop protocol.

Illustrative vs real

The widget renders a procedural 20³ phantom — an ellipsoidal lesion and a bone column — with genuine voxel sampling and linear window mapping. Clinical data is DICOM with anisotropic spacing, orientation metadata and Hounsfield calibration, and real models consume much larger volumes at multiple scales. The windowing arithmetic and the tri-planar / MIP reasoning are exactly as shown.

Check yourself

Eduspheria wiki · Applied AI, Medical AI

0 / 5 answered

  1. 1Windowing uses level 40 and width 80. What is the lower bound of the displayed HU range?
    Numeric answer
  2. 2What does a maximum-intensity projection along the z axis display for each (x, y)?
    Multiple choice
  3. 3V-Net performs 3-D convolutions over a volume rather than processing independent slices.
    True / false
  4. 4What physical unit does CT assign to each voxel to make intensities comparable across scanners?
    Short answer
  5. 5Which is the most serious practical obstacle to deploying a segmentation model across hospitals?
    Multiple choice

From the exam paper

Modeled on NITJ AI-607, End-Sem May 2025

0 / 5 answered

  1. 1Why might a clinician choose MRI over CT for imaging soft tissue?
    Multiple choice
  2. 2CT is based on X-ray attenuation whereas MRI uses strong magnetic fields and radiofrequency pulses.
    True / false
  3. 3In a radiograph, why do bones appear bright (white) while air appears dark?
    Multiple choice
  4. 4Which modality combines many X-ray projections taken from different angles and reconstructs cross-sectional images, for example by filtered back-projection?
    Short answer
  5. 5Because MRI uses ionizing radiation, it is usually avoided in pregnant patients.
    True / false

Where next: out of the scanner and onto the body — wearables stream physiology continuously, and the IoMT turns those streams into clinical alerts.