Steering and applications
From reading to controlling: activation steering, activation patching, model edits — and the safety work this enables.
Probing reads; the final lesson of this chapter touches. Once a feature exists as a clean direction, three moves open up: adding it to activations (steering), swapping activations between runs (patching), and editing weights (model editing). This is interpretability graduating from microscope to — sometimes — control panel.
Start here
If the model's behavior is mediated by identifiable directions in its activations, then at inference time you can turn a knob on those directions — no training, no prompt engineering: subtract the "sycophancy" direction and watch agreement-bias drop.
Steering: inference-time behavior control
Add a scaled feature vector to the residual stream (every layer, or a chosen layer) during generation:
Try the dose–response:
Add α × (formal-register vector) to every residual stream, then read the output
user: Can you fix the bug in this function?
α = 0: the model's default register — casual, direct.
One vector, dose-dependent effect, no retraining — the essence of activation steering. Continuations are curated for the demonstration; in a real model the vector comes from a SAE feature or the mean activation difference over a feature-relevant corpus, and dose–response is measured, not guessed.
Where does come from? Two standard recipes: a SAE feature's decoder direction (the previous lesson's output), or difference-in-means — record activations on prompts where the feature is present vs absent, subtract the means. Then tune : small values shift style, large ones degrade coherence (the artifact's overshoot regime) — steering is dose-dependent, not binary.
The honest limits: effects bleed into adjacent behaviors (one group's "reduced sycophancy" was another group's "blunter, less helpful answers"); the right layer matters as much as the vector; and strong steering can push the model off-distribution, where nothing is predictable.
Patching: causal localization
Activation patching is probing with a causal twist: run the model on input A and input B, then transplant one activation from the A-run into the B-run and see how much the output moves. This is the standard method for where is the answer computed questions, and it produced the most famous result in model editing:
- ROME (Rank-One Model Editing), Meng et al. (2022) first localized a factual association ("Eiffel Tower is in Paris") via causal tracing — running input A/B patching across layers to find where the fact lives (a single MLP layer, in a mid-range GPT model) — then showed in the same work how to edit the weights at that site with a rank-one update: rewrite "Paris" → "Rome" for the subject's entity vector, and the model's answers change downstream — without touching anything else.
The ROME debates matter as much as the result: later work showed such edits often break other facts sharing the subject (localization yes, surgical editing harder), which is the recurring lesson of this chapter — reading is robust, controlling is fragile.
What it's for
The applications are the reason to fund the science:
- Auditing safety claims. Is a model's refusal real or a fragile persona? Does the "we removed capability X" claim hold when you search the weights for X's features? Behavior testing can't distinguish removed from suppressed — internals can.
- Unlearning with verification. Deleting hazardous knowledge (CBW synthesis, etc.) and proving the feature is gone, not just masked by a guardrail prompt.
- Deception/monitoring. Features that fire on deceptive reasoning-during-honest-appearing-output are candidates for runtime honesty monitors.
- Debugging capability. "Why does the model fail at decades math?" is answerable: find the century-conversion feature, watch where the chain breaks.
Careful
Interpretability's standing gaps: clean results cluster in toy settings and narrow features; industrial-scale SAEs are expensive and partial; and interventions don't yet compose (steer on two features and interactions surprise you). The field's trajectory is real — treat current capabilities as diagnostic, not yet as a control plane.
This closes the wiki's full arc: the network compresses its corpus into superposed features; sparse autoencoders recover a readable shadow of them; and every recovered feature is a place where training, monitoring, and auditing can grip — the internals half of the safety story whose systems half the frontier chapter's final lesson covers.
If you want to go back and read the frontier chapter's limits-and- safety lesson with these internals in mind, start at limits, safety, and what's next.