Reasoning Models and Test-Time Compute: What the Evidence Supports

The new scaling axis

For years, the default way to improve a model was to train a bigger one. Reasoning models added a second axis: spend more computation at inference time. Generate intermediate steps, sample multiple candidates, search over possibilities or let a model trained with reinforcement learning think for longer before answering.

The idea is not new. Chain-of-thought prompting showed that intermediate reasoning steps improve performance on arithmetic, commonsense and symbolic tasks. What changed is that labs began training models to use that computation more effectively, and serving systems began to expose it as a controllable budget.

Chain-of-thought and self-consistency

Wei et al. showed that prompting a model to produce step-by-step reasoning improved performance on several benchmarks. Kojima et al. found that even a simple instruction such as "think step by step" could elicit reasoning in models that were not explicitly trained for it.

Self-consistency (Wang et al.) extended the idea by sampling multiple reasoning paths and selecting the most consistent answer. The gain comes from reducing variance: if several independent paths agree, the answer is more likely to be correct. The cost is obvious: more samples mean more tokens, more latency and more money.

The practical lesson is that reasoning quality is not a property of the prompt alone. It is a property of the sampling strategy and the budget.

Search, verifiers and process supervision

A more structured approach is to treat reasoning as search. Tree of Thoughts (Yao et al.) explores multiple partial solutions and evaluates them. Cobbe et al. trained verifiers to rank candidate solutions. Lightman et al. showed that process supervision, which rewards correct intermediate steps rather than only the final answer, can outperform outcome supervision on mathematical reasoning.

Verifiers are powerful because they separate generation from judgement. A generator can produce many candidates; a verifier can score them. The engineering challenge is building a verifier that is accurate on your domain and cheap enough to run at inference time.

RL-trained reasoning models

DeepSeek-R1 demonstrated that reinforcement learning can train a model to produce long reasoning traces and improve on tasks such as mathematics and coding. OpenAI's o1 system card described a model trained to spend more time reasoning before responding. The s1 project showed that a small amount of carefully curated reasoning data plus test-time scaling can produce strong results.

These systems blur the line between training and inference. The model learns how to think, and the serving system decides how long it is allowed to think. Both parts matter.

What test-time compute actually buys

Snell et al. studied how to allocate test-time compute optimally and found that the benefit depends on the difficulty of the problem and the strength of the base model. More compute is not uniformly better. On easy tasks it is wasted; on tasks beyond the model's capability it may not help at all.

A useful mental model is a budget curve. For each task category, measure accuracy as a function of generated tokens or samples. Find the point where additional compute stops paying for itself. That point is different for classification, multi-step maths, code generation and open-ended writing.

When reasoning hurts

Reasoning models can be slower, more expensive and more verbose. They can also overthink simple requests, produce plausible but incorrect rationales, or follow a flawed plan confidently. On tasks that require a short factual answer, a reasoning-heavy model may be worse than a direct one.

The production decision is therefore not "reasoning or no reasoning". It is "which requests deserve which budget". Route simple work to a cheaper path, and reserve long reasoning for cases where the evaluation shows a real gain.

Evaluating reasoning systems

Evaluate at the level of solved tasks, not just answer accuracy. Report:

• Accuracy at the chosen compute budget.
• Cost per solved task, including failed attempts.
• Latency distribution, including the tail.
• Consistency across repeated runs.
• Whether the reasoning trace is faithful to the answer.
• Performance on adversarial and out-of-distribution inputs.

A model that is 5% more accurate at 10x the cost may still be the right choice for a high-stakes task. That is a product decision, not a benchmark result.

References

1. Wei et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS.
2. Kojima et al. (2022). Large Language Models are Zero-Shot Reasoners. NeurIPS.
3. Wang et al. (2022). Self-Consistency Improves Chain of Thought Reasoning in Language Models. ICLR.
4. Cobbe et al. (2021). Training Verifiers to Solve Math Word Problems.
5. Lightman et al. (2023). Let's Verify Step by Step. ICLR.
6. Yao et al. (2023). Tree of Thoughts: Deliberate Problem Solving with Large Language Models. NeurIPS.
7. Snell et al. (2024). Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters.
8. DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.
9. OpenAI (2024). OpenAI o1 System Card.
10. Muennighoff et al. (2025). s1: Simple Test-Time Scaling.