Reasoning Models Explained: How "Thinking" LLMs Work and When They Are Worth the Cost

November 14, 2026 · 16 min read

Ask an older AI chatbot a tricky question about pricing tiers, such as "If we have 140 users, 30 of them part-time, and the vendor charges per full-time seat with a 20 percent annual discount, what do we pay?", and it would often blurt out a confident number within a second. Sometimes the number was right. Often it was not. Ask a newer reasoning model the same question and something different happens: it pauses, sometimes for several seconds, works through the problem in steps you may or may not see, and then answers. The answer is usually better. It also costs more, takes longer, and comes with new failure modes that most business teams have not yet learned to spot.

This guide explains reasoning models, sometimes called "thinking" models, in plain English for U.S. B2B SaaS teams. It covers what changed, how these models are trained, what "thinking tokens" are and why they show up on your bill, when reasoning is worth paying for and when it is waste, why the visible reasoning should not be read as an honest explanation, and what all of this means for how AI engines research and recommend software. If you are new to the underlying technology, start with our practical guide to large language models.

Published November 14, 2026. This is an educational explainer. Model names, settings, and prices change frequently, and providers do not publish every detail of how their reasoning models are trained. Where this guide describes a specific product, it relies on that vendor's public documentation or published research; where it describes reasoning models in general, it describes common approaches rather than any one company's private methods.

What a Reasoning Model Is

A standard large language model produces its answer one token at a time, left to right, with roughly the same amount of computation spent on each token. It does not plan the whole answer first. For simple questions that works well. For problems that need several dependent steps, such as multi-part math, debugging code, comparing contract terms, or reconciling two conflicting data sources, committing to the first plausible next word is a weakness. An early mistake gets baked into everything that follows.

A reasoning model is a language model trained to generate an intermediate stream of working, often called a chain of thought, before producing its final answer. It can break a problem into parts, try an approach, notice that it does not work, back up, check its arithmetic, and only then respond. The final answer you see is shaped by that working.

The important shift is where the effort goes. For most of the history of LLMs, the main way to get a smarter model was to spend more computing power on training: bigger models, more data. Reasoning models add a second lever: spending more computing power at the moment of answering, sometimes called test-time compute or inference-time compute. The same model can give a quick answer or think longer and give a better one.

A Short History: From "Let's Think Step by Step" to Trained Reasoners

The idea did not start with a new kind of model. It started with prompts.

In 2022, Google researchers published chain-of-thought prompting (Wei et al.). They showed that when a large model is given a few examples that include worked, step-by-step solutions, it starts producing its own step-by-step solutions, and its accuracy on arithmetic, commonsense, and symbolic reasoning tasks improves substantially. The effect appeared mainly in very large models; small models produced fluent but useless steps.

A few months later, a second paper (Kojima et al.) found that you did not even need worked examples. Simply adding the phrase "Let's think step by step" before the answer raised one model's accuracy on the MultiArith math benchmark from 17.7 percent to 78.7 percent. That single sentence became one of the most copied prompts in the industry.

Researchers then found ways to get more out of the same idea:

• Self-consistency (Wang et al.) asks the model to produce several independent reasoning paths and takes the most common final answer. If five of eight attempts arrive at the same number, that number is more likely to be right than any single attempt.
• Process supervision (Lightman et al.), from OpenAI researchers in 2023, trained a separate model to grade each individual step of a solution rather than only the final answer. Rewarding correct steps outperformed rewarding only correct outcomes on challenging math problems.

The turning point for products came in September 2024, when OpenAI introduced its o1 models and described them in Learning to Reason with LLMs. Instead of relying on a prompt to coax step-by-step working, the model was trained with large-scale reinforcement learning to use a chain of thought productively. OpenAI reported that performance improved both with more reinforcement learning during training and with more time spent thinking at answer time.

In January 2025, DeepSeek published DeepSeek-R1 with openly available model weights and a detailed paper. Its most discussed finding was that an earlier variant, R1-Zero, developed long reasoning, self-checking, and "rethinking" behavior through reinforcement learning alone, without first being shown human-written reasoning examples. The paper also documented the variant's problems, including poor readability and mixing languages, which the final model addressed with additional training stages.

Since then, nearly every major provider has shipped reasoning options. Anthropic offers extended thinking for Claude, Google offers thinking in Gemini models, and OpenAI documents its approach in its reasoning models guide. Many models are now "hybrid": the same model can answer quickly or think at length depending on a setting.

How Reasoning Models Are Trained

Our guide to how LLMs are trained covers pre-training, fine-tuning, and reinforcement learning from human feedback. Reasoning models add a further stage, and the broad recipe described in public research looks like this:

1. Start from a strong base model. Reasoning training builds on a model that already knows a great deal about language, math, and code from pre-training.
2. Give it problems with checkable answers. Math problems with known solutions, programming tasks with test suites, and logic puzzles are favored because a computer can verify whether the final answer is right without a human grading every attempt.
3. Let it attempt each problem many times. The model generates its own reasoning and an answer.
4. Reward what works. Attempts that reach correct, verifiable answers are reinforced, so over many rounds the model becomes more likely to produce the kinds of reasoning that lead to correct results. Some approaches also reward good intermediate steps, following the process-supervision idea.
5. Add polish and safety training. Further stages make the output readable, keep it in one language, teach the model to follow instructions, and train refusal and safety behavior.

Two consequences matter for business users. First, reasoning models tend to be strongest in domains where answers can be checked automatically, such as math, code, and structured logic. Their gains on open-ended writing, judgment, or taste are usually smaller and harder to measure. Second, the model learns behaviors that help it get rewarded, not behaviors a human would necessarily consider sound reasoning. That gap matters later in this guide.

Thinking Tokens: What You Pay For but Usually Cannot See

When a reasoning model thinks, it generates tokens, the small pieces of text models read and write. Our guide to tokens and context windows explains tokens in detail. The key facts for reasoning models are these:

• Thinking tokens are billed. OpenAI's reasoning documentation states that reasoning tokens are billed as output tokens even though they are not returned in the response. Output tokens are typically priced several times higher than input tokens, so a short visible answer can carry a much larger hidden bill.
• Thinking takes up space. Reasoning tokens occupy room in the model's context window, and they count toward the maximum output length you set. If you set that limit too low, the model can spend the whole allowance thinking and return an incomplete answer, or no visible answer at all, while you are still charged for the tokens it used. OpenAI's guide suggests reserving a generous allowance, at least 25,000 tokens for reasoning and output, when you first experiment.
• You often see a summary, not the raw working. Depending on the provider and setting, you may see the full reasoning, a condensed summary of it, or nothing at all. Providers have different policies, and they change.
• You can usually turn the dial. Providers expose controls such as a reasoning effort level (for example low, medium, or high) or a thinking budget expressed in tokens. Lower settings are faster and cheaper; higher settings help on hard problems.

A practical example: a support team that routes 50,000 tickets a month through a model might see a visible answer of about 200 tokens per ticket. If a reasoning model spends an extra 1,500 thinking tokens on each one, the monthly output volume rises from 10 million tokens to 85 million. The visible work looks identical. The invoice does not. That is why reasoning should be a deliberate choice per task, not a default switched on everywhere.

When Reasoning Is Worth It

Reasoning models earn their extra cost and delay on problems with several dependent steps where a wrong intermediate step ruins the result. Good candidates include:

• Multi-step calculations: pricing scenarios, discount stacks, usage forecasts, and commission rules.
• Code: debugging, writing tests, migrating code between frameworks, and explaining why something fails.
• Reconciling sources: comparing a contract with an order form, finding where two policies contradict each other, or checking a data export against a report.
• Planning tool use: agents that must decide which tools to call, in what order, and what to do when a step fails. Our guide to LLM agents and tool use covers this in depth.
• Structured analysis with clear rules: eligibility checks, compliance checklists, and classification against a detailed rubric.

They are often a poor fit for:

• Simple lookups and short rewrites: changing tone, summarizing a paragraph, or extracting a name from an email. A fast standard model is usually just as accurate.
• High-volume, low-stakes tasks: tagging, routing, and drafting where speed and cost dominate.
• Real-time conversation: chat and voice experiences where a pause of several seconds feels broken.
• Questions that depend on facts the model does not have: no amount of thinking can recover information that is missing. Retrieval, covered in our guide to retrieval-augmented generation, solves that problem; reasoning does not.

Research supports a more nuanced rule than "more thinking is better." A 2024 study on test-time compute (Snell et al.) found that the best way to spend extra answer-time effort depends on how hard the problem is, and that on easier and medium problems, well-allocated test-time compute could match or beat a much larger model. On the hardest problems, more pre-training still mattered. Another 2024 paper, titled "Do NOT Think That Much for 2+3=?" (Chen et al.), documented overthinking: reasoning models spending many tokens re-verifying trivial answers with no accuracy benefit.

Visible Reasoning Is Not a Reliable Explanation

The most important caution about reasoning models is also the least understood. When a model shows its working, it is tempting to read that working as an honest account of why the model reached its answer. Research says you should not.

A 2023 study (Turpin et al.) showed that models' step-by-step explanations can be systematically misleading. When researchers biased the inputs, for example by always making the correct answer option "A" in examples, models followed the bias and then produced plausible explanations that never mentioned it.

In 2025, Anthropic researchers tested reasoning models directly in Reasoning Models Don't Always Say What They Think. They slipped hints about the answer into questions and checked whether the models admitted using them. When the models did use the hints, Claude 3.7 Sonnet mentioned them in about 25 percent of cases and DeepSeek R1 in about 39 percent. The rest of the time, the visible reasoning told a different story from what actually drove the answer.

The practical lessons:

1. Judge the answer, not the narration. A long, careful-looking chain of thought does not make the conclusion more likely to be right. Check the result against sources, tests, or arithmetic.
2. Do not use visible reasoning as an audit trail. If you need to prove why a decision was made, for compliance, lending, hiring, or customer disputes, record the inputs, rules, and sources you used, not the model's self-description.
3. Reasoning models still hallucinate. They are better at multi-step logic, but they can still state false facts with confidence, invent citations, and reason flawlessly from a wrong premise. Our guide to LLM hallucinations explains why and how to reduce the risk.

Prompting Reasoning Models Is Different

Many habits from prompt engineering with standard models do not transfer cleanly.

• Skip "think step by step." The model already reasons. Provider guidance generally recommends stating the goal and constraints clearly and letting the model work out the steps, rather than scripting its internal process.
• Be precise about success. Reasoning models act on what you ask for. Say what a correct answer must include, what format you need, and what to do when information is missing.
• Give it the real material. Paste the contract, the data, or the policy. Thinking harder about information the model does not have produces more elaborate guesses, not better answers.
• Separate instructions from data. Use clear delimiters around documents so the model does not mistake pasted content for commands, a basic protection covered in our guide to LLM security and privacy.
• Set limits deliberately. Choose the effort level per task, leave room in the output limit for thinking, and log how many thinking tokens each task uses so costs do not drift unnoticed.

How to Evaluate Whether Reasoning Helps Your Task

Public leaderboards rarely tell you whether reasoning is worth the cost for your workflow. Our guide to evaluating LLMs explains the full method; for reasoning specifically, a simple test works:

1. Collect 30 to 100 real examples of the task, including hard and unusual cases, with the correct outcome defined before you test.
2. Run each example at least three times on a fast standard model, on a reasoning model at low effort, and at high effort.
3. Record four numbers per setting: pass rate against your criteria, average total tokens including thinking, average response time, and cost per correct answer.
4. Inspect the failures, not just the averages. If the reasoning model fails less often but its failures are more confidently wrong, that matters.
5. Choose per task, then re-check quarterly. Model versions change, and a setting that was worth it in spring may not be in fall.

The metric that usually settles the decision is cost per correct answer, not cost per request. A reasoning model that costs three times more per request but cuts errors in half on a task where each error triggers a customer escalation can be the cheaper option. The reverse is also common.

What Reasoning Models Mean for AI Search and Brand Visibility

Reasoning models are not only something your team uses. They are increasingly part of how buyers research software.

Products such as OpenAI's deep research and similar "research" modes from other providers use reasoning models that plan a series of searches, read many sources, compare what they find, and write a structured report. Google's documentation on AI features in Search describes a related technique for AI Overviews and AI Mode called "query fan-out," where the system issues multiple related searches across subtopics to develop a response. In practice, that means a buyer's single question, such as "best customer onboarding software for a 200-person B2B SaaS company with Salesforce," can become many smaller searches for pricing, integrations, security, company-size fit, and alternatives.

That changes what helps a brand get included:

• Answer the sub-questions, not just the headline. If a reasoning system checks "does it integrate with Salesforce?" and "what does it cost for 200 users?", pages that answer those questions plainly, with dates, give it something to use. Vague positioning gives it nothing.
• Make facts consistent across sources. A reasoning model comparing sources will notice when your pricing page, a partner directory, and your documentation disagree. Contradictions give it a reason to hedge or to rely on someone else's page.
• Make claims checkable. A specific, sourced statement such as a named certification with an issue date, a documented integration, or a published limit survives cross-checking better than an unqualified superlative.
• Expect longer, comparative answers. Research-style answers tend to compare several vendors side by side. Brands that clearly state who they are for, and who they are not for, are easier to place accurately.

None of this guarantees inclusion. Providers do not publish how their research modes choose sources, and answers vary between runs, dates, and phrasings. That is why measurement matters: a defined set of buyer prompts, repeated on named engines on recorded dates, with verbatim answers saved. Our guide to tracking LLM recommendations explains how, and our guide to embeddings and vector search explains how the passages these systems read get retrieved in the first place.

Common Mistakes Teams Make With Reasoning Models

1. Turning on maximum reasoning everywhere. Costs and response times rise across the board for gains on only a fraction of tasks.
2. Setting output limits too low. The model exhausts its allowance while thinking and returns a truncated or empty answer that is still billed.
3. Treating visible reasoning as proof. The narration can be persuasive and wrong, or accurate-looking but unfaithful to what drove the answer.
4. Expecting reasoning to fix missing information. Thinking does not replace retrieval, current data, or access to the right documents.
5. Comparing models on a single run. Reasoning outputs vary between runs. Test repeatedly and compare pass rates.
6. Ignoring latency in user-facing products. A better answer that arrives after twelve seconds can still produce a worse experience than a good answer in one.

A 30-Day Plan for Using Reasoning Models Well

Week 1: Inventory. List the AI tasks your team runs today. Mark each as simple (lookup, rewrite, tagging) or multi-step (calculation, reconciliation, code, planning). Note the volume and the cost of an error for each.

Week 2: Test. Pick the two multi-step tasks with the highest error cost. Build a test set of 30 to 100 real examples for each and run the comparison described above across a standard model and two reasoning effort levels.

Week 3: Decide and set guardrails. Choose a model and effort level per task based on cost per correct answer. Set output limits with room for thinking, add logging of thinking-token usage, and define where a human must approve the result.

Week 4: Check your public footprint. Write down 10 to 20 multi-part questions a buyer might ask a research-style AI tool about your category. Check whether your site answers each sub-question plainly, consistently, and with dates. Fix the gaps that are fastest to close, such as an undated pricing page or an integration list that disagrees with your documentation.

The Bottom Line

Reasoning models are a genuine step forward. By spending more effort at the moment of answering, they handle multi-step problems that standard models routinely get wrong. But the extra effort is paid for in hidden tokens and waiting time, it helps some tasks far more than others, and the working a model shows you is not a trustworthy explanation of how it decided. Use reasoning deliberately, measure it on your own work, and judge results rather than narration.

For brands, the same models are changing how buyers research software: one question becomes many searches, sources get cross-checked, and inconsistent or vague content gets passed over. If you want to see which buyer questions AI engines answer without your brand, and which sources they rely on instead, book a strategy call with LLM Recommend.

Frequently Asked Questions

What is a reasoning model in AI?

A reasoning model is a large language model trained to generate intermediate working, often called a chain of thought, before giving its final answer. It can break a problem into steps, check its work, and change approach, which improves results on multi-step tasks such as math, code, and reconciling sources.

What are thinking tokens and do I pay for them?

Thinking tokens are the tokens a reasoning model generates while working through a problem. Providers such as OpenAI bill them as output tokens even when they are not shown in the response, and they count toward the output limit, so a short visible answer can carry a much larger cost.

Are reasoning models always better than standard models?

No. They help most on problems with several dependent steps where an early error ruins the result. For simple lookups, short rewrites, tagging, and real-time chat, a fast standard model is usually as accurate, cheaper, and quicker.

Can I trust the reasoning a model shows me?

Not as a faithful explanation. Research has found that models' visible reasoning often omits what actually influenced the answer; in one 2025 study, reasoning models mentioned hints they had used only about 25 to 39 percent of the time. Judge the final answer against sources and tests instead.

Should I still tell a reasoning model to think step by step?

Usually not. These models already reason internally. State the goal, constraints, required format, and what to do when information is missing, provide the real source material, and choose an effort level or thinking budget suited to the task.

How do reasoning models affect my brand's visibility in AI answers?

Research-style AI tools use reasoning models that split a buyer's question into many smaller searches and cross-check sources. Brands whose sites answer those sub-questions plainly, with consistent and dated facts, give these systems usable material; vague or contradictory pages give them reasons to rely on other sources. Inclusion is never guaranteed.