Tree of thought prompting with a branching prompt
Free prompt courseLesson 14 of 229 minVideoGo to the lesson
Guide
4 min
- Plain English
The short answer
Tree of thought prompting makes an AI explore different lines of reasoning, score how promising each one is, drop the weak ones and continue with the best. It suits problems where the first idea often fails, like planning and puzzles. A single-prompt version only approximates the research method, which runs many model calls.

What is tree of thought prompting?
Tree of thought prompting is a method that makes a model explore several lines of reasoning, score each one, drop the weak ones and continue with the best. It works on 4 parts: branches (distinct options), scoring criteria, pruning and depth, which is how many levels you expand before you decide.
The method comes from the paper Tree of Thoughts: Deliberate Problem Solving with Large Language Models (Yao et al., arXiv 2305.10601, 2023). In the paper it runs as a search: one set of model calls proposes the next thoughts, another set judges them, and a search routine walks the tree and can backtrack. That takes multiple model calls per problem.
In a chat you don't get that search loop, so you imitate it with one prompt that asks for 3 moves: branch (give distinct options), score (rate each against criteria) and prune (cut the weakest, expand the rest). It's an approximation, and that's fine for most everyday decisions.
How does tree of thought differ from chain of thought prompting?
Tree of thought differs from chain of thought prompting by exploring several reasoning branches and pruning them, while chain of thought follows one path. Chain of thought writes the steps of a single line of reasoning; if the first step is wrong, the whole answer inherits the mistake.
| Point | Chain of thought (CoT) | Tree of thought (ToT) |
|---|---|---|
| Paths | 1 single path | 3 or more branches |
| Judging | None during the run | Each branch is scored against criteria |
| Recovery from a bad start | Rare, the chain keeps going | Weak branches are pruned, the best one continues |
| Token cost | Lower | Higher, every branch adds scores and reasons |
So ToT adds 2 steps on top of a chain: scoring and pruning. That's what lets it walk away from a first idea that doesn't hold up, which a single chain can't do.
How do you write a tree of thought prompt template?
To write a tree of thought prompt template, state the problem and the scoring criteria, then ask for distinct options, scores and a prune step. Vague criteria give vague scores, so run your brief through the prompt rewriting tool before you branch; that prompt optimizer will rewrite loose criteria into ones you can measure. Then follow 5 steps:
- State the problem and 3 to 4 criteria for a good answer. Don't skip the criteria: without them the model hands out a 4 out of 5 to everything.
- Ask for 3 distinct first approaches, not variations of one.
- Ask for a score per approach on each criterion, with a number and a one-line reason.
- Prune the lowest total, then expand the top 1 or 2 by one more step. Repeat to the depth you set, usually 2 or 3 levels.
- Finish with the winner, why the others lost and what new fact would change the choice.
Worked example: for "first 100 customers for a meal-prep service", the criteria are cost, speed to the first 20 customers and effort for one person. The model offers 3 strategies, scores them, drops the weakest and lists the first 3 steps for the 2 it keeps. The branch and prune prompt in the examples section does this in one go, and the second prompt runs it in rounds.
What are the limits of tree of thought prompting in a chat?
Tree of thought prompting in a chat has 3 limits: it approximates the research method, the model scores its own ideas leniently and it costs more tokens.
- It's an approximation. One reply can't run the paper's search with separate judging calls, so there's no real backtracking unless you ask for it.
- Self-scoring is the weak point. The model grades ideas it just wrote, and it's lenient with them. Concrete criteria cut that leniency: "costs under 500 dollars" is harder to fudge than "affordable". Ask for a reason per score, and treat each score as a prompt for your own judgement, not as truth.
- It costs more. Each extra branch adds its own scores and reasons, and you'll pay for all of them,, so the reply gets longer and burns more tokens.
The round-by-round version fixes the first 2 limits: the model branches and scores, then stops, and you type which branches to keep or prune. You're the judge, so leniency matters less. Reasoning models already explore on their own: OpenAI says they don't need step-by-step coaching, and Anthropic recommends general instructions like "think thoroughly" over hand-written plans (OpenAI, Anthropic). On those models a lighter prompt that asks for alternatives with trade-offs often works as well.
Tree of thought prompting before and after
The same request, asked two ways. Read the second one slowly. Each line does a job.
What's the best way to get my first 100 customers for a new meal-prep service?
Give me 3 genuinely different strategies for getting the first 100 customers for a new meal-prep service in a mid-sized city. Score each from 1 to 5 on cost, speed to first 20 customers and effort for one person, with a one-line reason for each score. Drop the weakest. Take the best two and list the first three concrete steps of each. Then recommend one and say what would change your mind.
Why it works
The after version forces distinct options, scoring against criteria and pruning, instead of one plausible-sounding answer.
Tree of thought prompting examples
Branch, score and prune in one prompt
Explicit criteria and a prune step are what make the branching useful, not decorative.
Problem: {{problem}} Criteria for a good answer: {{criteria}} Do this in order. 1. Branch: give 3 genuinely different approaches (not variations of one). Name each in a few words. 2. Score: rate each approach from 1 to 5 on every criterion, with a one-line reason per score. Show it as a table. 3. Prune: drop the lowest total and say why. 4. Expand: for the remaining {{keep}}, give the next {{depth}} concrete steps and the main risk. 5. Decide: pick one, and say what new fact would make you switch to another. Be critical of your own ideas. If two approaches score the same, say what you'd need to know to separate them. Don't invent facts about my situation; list your assumptions instead.
How to use
- Copy it with the button above.
- Fill the blanks in {{double braces}} with your own details.
- Paste it into ChatGPT, Claude or Gemini and send it.
- Then follow up: ask for one change at a time.
Before you run “Branch, score and prune in one prompt” for the first time, the lesson on how to use AI prompts step by step walks through copying, filling and sending it.
Tree of thought in rounds you can steer
Handing the keep and prune decisions to you fixes the main weakness of single-prompt ToT: the model grading itself.
We'll explore this problem as a tree, one level per round. Problem: {{problem}} Criteria: {{criteria}} Round 1 (your first reply): give 3 different first approaches, scored against the criteria in a table. Don't expand any yet. Then stop. When I type keep: A, C (for example), expand only those approaches by one level: for each, give 2 distinct next moves, score them, and show the tree so far as an indented list. When I type prune: B, cut that branch and say in one line what we learned from it. When I type back: return to the previous level and offer a new branch. When I type decide: recommend one path with its steps, its main risk and what would change your mind. After each reply, print a short tree summary, not more than 6 lines, so we don't lose the structure.
Lesson 14 of 22Techniques9 min
Lesson: Compare options (tree of thought)
Instead of taking the first idea, ask for different approaches, score them against your criteria, then keep the best and explore further. That's tree of thought in a chat, and Prompt Studio's Compare options technique is its one-level version.
Use it when
- Headlines, names, plans or structures where the first idea isn't always best.
- You can state the criteria that make one option better.
- A decision you'll want to explain later.
Skip it when
- There's one right answer to look up.
- You don't have criteria: brainstorm freely first.
We'll explore this problem as a tree, one level per round. Problem: Decide how to structure a 6,000 word guide so it's easy to scan Criteria: reader can find answers fast, avoids repetition, easy to update Round 1 (your first reply): give 3 different first approaches, scored against the criteria in a table. Don't expand any yet. Then stop. When I type keep: A, C (for example), expand only those approaches by one level: for each, give 2 distinct next moves, score them, and show the tree so far as an indented list. When I type prune: B, cut that branch and say in one line what we learned from it. When I type back: return to the previous level and offer a new branch. When I type decide: recommend one path with its steps, its main risk and what would change your mind. After each reply, print a short tree summary, not more than 6 lines, so we don't lose the structure.
**Round 1: three first approaches for structuring a 6,000-word guide**
Scores are 1 (poor) to 5 (strong).
| Approach | Description | Find answers fast | Avoids repetition | Easy to update | Total |
|---|---|---|---|---|---|
| **A. Task-based sections** | Organize by what the reader is trying to do ("How do I set up X?", "How do I fix Y?"), with question-style headings and a linked contents list at the top | 5 | 3 | 4 | 12 |
| **B. Layered (summary, then detail)** | One-page quick-start or cheat sheet up front, followed by deeper sections for each topic, with a "read more" link from each summary point | 4 | 2 | 3 | 9 |
| **C. Modular reference** | Short, self-contained topic blocks in a consistent template (what, when, steps, pitfalls), each stored as its own unit and cross-linked rather than restated | 3 | 5 | 5 | 13 |
**Quick read:** A wins on speed for readers who arrive with a goal. C wins on maintenance and avoiding duplication. B helps skimmers but tends to repeat itself, because the summary and the detail say the same thing twice.
**Tree so far**
```
Root: structure a 6,000-word guide
├─ A. Task-based sections (12)
├─ B. Layered summary + detail (9)
└─ C. Modular reference blocks (13)
```
Type `keep:`, `prune:`, `back:` or `decide:` to continue.What to notice: Three clearly different approaches, each scored on the three criteria, then it stopped and waited as round 1 asked. The scores make the trade-off visible: task-based wins on findability, modular reference on maintenance. Your criteria do the deciding, not the model's taste.
Try it in Prompt StudioOpens the studio with the Compare options technique already selected.
Quick check: 3 questions
Practice: write your own
Write a compare prompt for a choice you're facing: a name, a headline or a plan. Ask for three clearly different options, scored against criteria you name, then a pick with a reason.
Prompt Studio's rubric scores it right here. Nothing's sent anywhere.
Your progress is saved in this browser only. No sign-up.
Which problems make tree of thought prompting worth the tokens?
Tree of thought prompting is worth the tokens for 5 kinds of problems, including strategy choices, planning with trade-offs and puzzles, plus the structure of a long piece and any problem where the first idea is often wrong.
Don't use it for lookups, rewrites or tasks with one obvious path. It's overkill there, it won't make a simple answer better, and you'll wait longer for nothing. Scores get steadier when the prompt carries one scored sample, a format shown in few-shot prompting examples. You needn't write the branching prompt yourself either: a model can draft it for you, the trick behind these meta prompt examples. For consensus instead of exploration, try self-consistency, and the overview of every method is on types of prompting.
Tree of thought prompting FAQ
Still stuck? Build it in Prompt Studio. It'll score each part as you go.
Can ChatGPT do tree of thought prompting in one prompt?
Yes, ChatGPT can do tree of thought prompting in one prompt as an approximation, and the paper's version uses many model calls. One prompt can ask for distinct approaches, a scored comparison and a pick, which imitates the idea. It isn't a true search, since the model's judging its own options. Running it in rounds, with you choosing which branches to keep, is more reliable.
What is the 'three experts' tree of thought prompt?
The 'three experts' tree of thought prompt is a one-line approximation that asks three made-up experts to share steps and drop out when wrong. It's quick to paste, but it doesn't name any criteria, so nobody checks how a step gets judged. A criteria-based branch, score and prune prompt gives you scores you can actually read and argue with.
Who created the tree of thought method?
The tree of thought method was published by Yao and co-authors in the 2023 paper Tree of Thoughts. The authors are Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao and Karthik Narasimhan, and the paper was posted to arXiv in May 2023 (2305.10601) and presented at NeurIPS 2023.