Prompting and reasoning
How to steer a trained model — and why asking it to think out loud actually changes what it can do.
The model is fixed at inference time — no weights move. Everything a user gets is steered through the context: instructions, examples, format constraints. Prompting is this steering, and reasoning modes are the surprising discovery that came out of it.
Start here
The context window is the model's only workspace at inference time. Prompting = engineering what goes into it. Chain-of-thought = discovering that letting the model write its workspace into the output makes hard problems tractable.
From instructions to reasoning
Prompting has roughly four maturity levels, each unlocked by scale:
- Zero-shot — just ask. "Summarize this text." Works at all, works better at scale.
- Few-shot — show 2–5 examples inside the prompt; the model infers the pattern and continues it. This is in-context learning: no weight change, yet the model picks up a task it was never trained on.
- Instruction quality — constraints ("answer in JSON", "cite sources", "ask before assuming") measurably change behavior.
- Chain-of-thought — "think step by step" or few-shot worked examples that demonstrate reasoning.
Chain-of-thought: computation via tokens
The key observation, made explicit:
Same question, two decoding strategies
A pen costs $7. You buy 3 pens and pay with a $25 bill. What change do you get?
Why it works: every token generated buys another forward pass — writing intermediate steps gives the model "working memory in tokens." Illustrative example; the direct-answer failure is common on multi-step problems, not guaranteed every time.
The mechanism is not mystical: each token the model generates conditions the next forward pass. Writing "3 × $7 = 21" puts the correct intermediate result in the context where the next step can read it. Without it, the model must compute several operations in a single pass — and errors compound silently. Reasoning-mode models (the next lesson) train this behavior deliberately.
Two honest caveats:
- CoT helps multi-step problems (arithmetic, logic, planning), not everything — single-fact retrieval often gets slower and no better.
- The steps aren't always the real mechanism. The written reasoning usually reflects the computation, but models can produce fluent steps toward a wrong answer. CoT is scaffolding, not proof.
Prompting that actually ships
Practitioner patterns worth knowing:
- Put the role, the task, the constraints, and the output format in that order. Models follow structure better than prose.
- Show one worked example rather than describing one.
- Decompose — "first list the facts, then answer" beats "answer."
- Ask for uncertainty — "if unsure, say so" measurably reduces confident nonsense, though it never eliminates it (the limits lesson).
- System vs. user turns matter — product prompts set durable behavior; user messages steer per-request. Post-training tuned the model to respect that distinction.
Note
Prompting sits at the boundary between using a model and engineering a system. The next lessons push further along that boundary — training the thinking itself (reasoning models), giving the model knowledge it doesn't have (retrieval), and actions it can't take alone (tools and agents).
Next: reasoning models — training the thinking itself, not just asking for it.