Reasoning models, explained
"Thinking" models work by generating intermediate text before the final answer. That sounds trivial and turns out to be the mechanism — here's why writing more tokens buys more accuracy, what it costs, and when it's wasted.
By Jackdaw Team
The short version: A reasoning model "thinks" by writing out intermediate steps before its answer, which lets it spend more computation on the problem. That buys real accuracy where the work is genuinely hard — and is wasted cost and latency where it isn't.
A "reasoning" model — one that "thinks before it answers" — is one of the more important shifts in how these systems are used, and one of the most over-mystified. The mechanism is simple: thinking is the model writing out intermediate text before its final answer. Why that helps, what it costs, and when it's a waste are the parts worth understanding.
A transformer does a fixed amount of work per token
Start with a constraint. When a model processes a sequence and produces one token, it runs a fixed-size computation — the same number of layers, the same number of operations, every time. A model can't "think harder" about a single token; the amount of compute per token is baked into its architecture. For an easy next word, that's plenty. For a hard problem — a multi-step math question, a tricky logic puzzle, a subtle piece of code — the answer requires more sequential reasoning than fits inside one forward pass. The model is being asked to jump straight to a conclusion that genuinely needs several steps of work, with only one step's worth of computation available.
There's a way around the constraint, and it's the whole idea: let the model produce more tokens. Because generation is autoregressive — each token the model writes becomes part of the input for the next one — the model can use its own output as a scratchpad. It writes step one, then reads step one back in to compute step two, then reads both back to compute step three. Each intermediate token is another fixed chunk of computation, and stringing them together lets the model spend far more total compute on the problem, and decompose it into pieces each small enough to fit in one forward pass.
That's what "thinking" is. The model isn't accessing a deeper faculty. It's converting a hard one-shot problem into a sequence of easy steps, using the page as external working memory, the same way you'd never multiply two four-digit numbers in your head but can do it easily on paper.
From a prompt trick to a trained-in skill
This started as a prompting discovery: around 2022, people found that simply adding "let's think step by step," or showing a few worked examples, made models dramatically better at reasoning tasks. Same model, same weights — just given room to write out intermediate steps before answering. The phenomenon got a name, chain-of-thought, and it was clearly leaving capability on the table to not use it.
The newer reasoning models bake this in. Instead of relying on you to prompt for it, they're trained — typically with reinforcement learning that rewards reaching the correct final answer — to generate long, useful chains of reasoning on their own: trying approaches, checking their own work, backtracking when a path fails. The visible result is a model that, handed a hard problem, produces a substantial stretch of internal reasoning and then a final answer. The dial you control is usually exposed as an "effort" or "thinking" setting: how much of this internal work to do.
More thinking buys accuracy — up to a point
Here's the surprising part: you can trade compute for accuracy at inference time, not just at training time. Let the model think longer — generate more reasoning tokens — and its accuracy on hard problems climbs. This is "test-time compute," and it's a genuinely different lever: historically you made models smarter by training bigger ones, an expensive, one-time investment. Now you can also make a model effectively smarter on a given hard problem by letting it spend more tokens thinking about it, on demand.
The curve has the shape you'd expect. On hard problems, more thinking helps, with diminishing returns — the first chunk of reasoning buys a lot, each additional chunk buys less, and eventually the model is "overthinking," second-guessing a correct answer or wandering. The sweet spot depends on the problem's difficulty.
What it costs
Thinking isn't free, and the bill comes in two forms.
Tokens. Those reasoning steps are generated tokens, and they're billed — usually at output-token rates, which are the expensive ones. A reasoning model can produce many times more internal reasoning than visible answer, so a question with a one-sentence answer might cost as if you'd asked for several pages. For a single hard query that's a bargain. Run it across a high-volume workload and it's the difference between a sustainable cost and an alarming one.
Latency. Generating a long reasoning chain takes time — a heavily-thinking model can take seconds to minutes on a hard problem. That's fine for an analysis you'll read later; it's unacceptable for an interactive loop where a human is waiting.
When it's wasted
The flip side: on easy problems, more thinking does nothing useful and costs you both ways. Classifying a support ticket, extracting a field from a form, rephrasing a sentence, looking up something stated in the prompt — these don't need multi-step reasoning, and forcing it on them adds latency and token cost for no gain. Worse, on genuinely simple tasks heavy reasoning can occasionally hurt: the model talks itself out of an obvious correct answer.
So the skill is matching effort to difficulty. Turn thinking up for the hard, high-stakes, latency-tolerant problems where accuracy is worth the spend. Turn it down or off for the routine, high-volume, latency-sensitive ones. The mistake is picking one setting for everything — usually "maximum," because it scored best on a benchmark — and then paying reasoning-model prices and latency on a pile of work that a fast, cheap model would have nailed instantly.
A caveat worth keeping
One honest complication: the reasoning a model writes is not guaranteed to be why it reached its answer. Research has repeatedly found that a model's stated chain of thought can be a plausible-sounding rationalization that doesn't faithfully reflect the actual computation — the model can reach an answer for one reason and write up a different, tidier-looking justification. This matters if you're tempted to trust the reasoning trace as an explanation you can audit. It's a real artifact of work, and it demonstrably improves accuracy, but it's not a transparent window into the model's "mind." Treat it as a scratchpad that helps the model think, not as a confession of how it actually thought.
The plain takeaway: reasoning models give you a knob that trades money and time for accuracy on hard problems. That's genuinely valuable and genuinely new. It is also, like every knob, something to set deliberately per task — not a free upgrade to leave on max.