Chain-of-Thought Prompting in 2026: What Works and When to Skip It

How we wrote this: the research and vendor guidance come from the original papers and the providers’ current documentation, checked on October 21, 2026. The four content templates are starting points to adapt; we haven’t benchmarked them across models.

What Is Chain-of-Thought Prompting?

Chain-of-thought prompting gets a language model to write out its intermediate reasoning before committing to a final answer, like a student showing their working.

The idea was formalised by Google researchers in “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” (Wei et al., 2022). With eight worked examples in the prompt, a 540-billion-parameter model reached state-of-the-art accuracy on GSM8K, a benchmark of grade-school maths word problems. The same paper found the benefit appeared only in very large models of that era; small models didn’t gain.

A few months later, “Large Language Models are Zero-Shot Reasoners” (Kojima et al., 2022) showed you don’t always need examples. Adding “Let’s think step by step” took one model from 17.7% to 78.7% on the MultiArith benchmark and from 10.4% to 40.7% on GSM8K.

Why it works: a model generates each token based on everything before it. When it writes the reasoning first, the final answer is conditioned on those steps rather than produced in one leap, which cuts down errors on problems with several dependent parts.

The Main Variants

Technique	How it works	When to use it
Zero-shot CoT	Add a trigger such as “Let’s think step by step” (Kojima et al., 2022)	Quick reasoning boost on a standard model
Few-shot CoT	Include worked examples that show the reasoning and the answer (Wei et al., 2022)	Specialised tasks and fixed output formats
Self-consistency	Sample several reasoning paths and take the most common answer (Wang et al., 2022)	When one wrong chain is costly and you can afford several calls
Tree of thoughts	Explore branching reasoning paths, evaluate them, and backtrack (Yao et al., 2023, Princeton and Google DeepMind)	Planning and search problems with no single path
Built-in reasoning	The model reasons in a separate phase before answering (OpenAI reasoning models, Claude extended thinking, Gemini thinking)	Hard multi-step tasks; replaces manual CoT on these models

Reasoning Models Changed the Advice

The biggest change since 2022: the leading providers now ship models that do chain-of-thought internally. That changes how you should prompt them.

• OpenAI. OpenAI’s reasoning best practices tell you to avoid chain-of-thought prompts with its reasoning models. They already reason internally, and asking them to think step by step “may not enhance performance (and can sometimes hinder it)”.
• Anthropic. With extended thinking, Claude’s prompting guide recommends general instructions over prescriptive steps. A prompt like “think thoroughly” often beats a hand-written step-by-step plan.
• Google. Gemini’s thinking mode reasons before responding. You control how much it thinks with a budget, not with prompt phrasing.

So: on a reasoning model, state the goal, the constraints and the output you want, and let it plan. Save explicit “step 1, step 2” instructions for standard models, or for cases where the steps are business rules the model can’t know (“check the price against our sheet before recommending”).

Four Templates for Content and SEO Work

These are written for standard models, where explicit structure helps. On a reasoning model, keep the goal, inputs and output format, and drop the numbered steps.

Template 1: Content brief

Keyword: "[target keyword]"
Work through these before writing anything:
1. The likely search intent (informational, commercial, navigational)
2. The questions a searcher has next
3. What the current top results cover, and what they leave out
4. Three angles that fill that gap
Then write a 300-word brief based on your analysis.

Template 2: Title tag comparison

Compare two title tags for the same page:
Title A: "[first title]"
Title B: "[second title]"
For each, assess: keyword match, specificity, clarity of the promise,
and how it would stand out among the current results.
Then recommend one and explain why in two sentences.

Template 3: Internal link map

Here are pages from my site: [paste URLs and titles]
1. Group them into topic clusters
2. Pick the strongest hub page in each cluster
3. Decide which pages should link to each hub
4. Flag pages with no natural link target
Output a JSON array of [source_url, target_url, anchor_text].

Template 4: Search Console error triage

Here is my Search Console page indexing report: [paste]
For each issue type: the usual causes, the likely traffic impact,
and a fix priority from 1 to 3.
Then list the three fixes to do this week.

For more structures like these, see our advanced prompting techniques guide and the prompt engineering for SEO guide.

When Chain-of-Thought Goes Wrong

• Confident wrong reasoning. A fluent chain can lead to a wrong answer, and the explanation makes the error harder to spot on review. Check the facts, not the prose.
• Reasoning without benefit. On simple lookups or rewrites, CoT adds length and cost without improving the result.
• Inconsistent examples. In few-shot CoT, examples that reason in different styles produce inconsistent output. Keep them uniform.
• Double reasoning. Asking a reasoning model to also think step by step in its answer wastes tokens and, per OpenAI, can hurt results.

The Benchmarks You’ll See Cited

Benchmark	What it tests	Origin
GSM8K	About 8,500 grade-school maths word problems needing several steps	OpenAI (Cobbe et al., 2021)
MATH	Competition-level maths problems	Hendrycks et al., 2021
MMLU	Multiple-choice questions across 57 subjects	Hendrycks et al., 2020
HumanEval	Writing Python functions that pass unit tests	OpenAI (Chen et al., 2021)

Benchmarks tell you how models compare on standard problems. They don’t tell you how a prompt will do on your content tasks, so test on a handful of your own real examples.

New to prompting? Start with our prompt engineering guide.

Frequently Asked Questions

Does chain-of-thought prompting work on all AI models?

It helps most on large standard models. In the original 2022 research, small models didn’t benefit. Reasoning models already reason internally, and OpenAI advises against chain-of-thought prompts for them.

What is the difference between zero-shot and few-shot CoT?

Zero-shot CoT adds a trigger such as “Let’s think step by step” with no examples. Few-shot CoT includes worked examples showing the reasoning and the answer. Few-shot takes more effort and helps most when the task or format is specialised.

Should I tell a reasoning model to think step by step?

Generally no. OpenAI says it may not help its reasoning models and can hinder them, and Anthropic recommends general instructions such as “think thoroughly” over prescriptive steps when Claude’s extended thinking is on. Give the goal, the constraints and the output format instead.

Can chain-of-thought reduce hallucinations?

Sometimes, on reasoning errors. It does not stop a model inventing facts, citations or statistics, and a confident chain can make an invented fact more convincing. Verify specifics against primary sources.

What is tree-of-thoughts prompting?

Tree of thoughts (Yao et al., 2023) has the model explore several reasoning branches, evaluate them and backtrack from dead ends. It suits planning and search problems; for everyday content tasks, a single structured prompt is usually enough.

Last updated: 2026-10-21. Sources: Wei et al. 2022 (arXiv:2201.11903), Kojima et al. 2022 (arXiv:2205.11916), Wang et al. 2022, Yao et al. 2023, OpenAI reasoning best practices, Anthropic prompting best practices.