Advanced Prompting: CoT / Self-Consistency / Tree of Thoughts / Reflexion

L0-05 taught the 10 basics. This one covers techniques from academic research that actually move the needle on hard tasks — pushing GPT-4 from 50% to 85% on math problems.

These are not “promptingtricks”, they are architectural patterns for getting more out of any LLM.

1. Chain of Thought (CoT, 2022)

The simplest yet most important breakthrough.

Before CoT

User: A store sells apples at $2 each and bananas at $1 each.
       I bought 3 apples and 5 bananas. How much did I spend?

Model: $10

The model jumps straight to the answer — often wrong on multi-step problems.

With CoT

User: A store sells apples at $2 each and bananas at $1 each.
       I bought 3 apples and 5 bananas. How much did I spend?
       Let's think step by step.

Model: First, the apples cost 3 × $2 = $6.
       Then, the bananas cost 5 × $1 = $5.
       Total: $6 + $5 = $11.
       Answer: $11.

Magic phrase: “Let’s think step by step” (or “show your reasoning step by step”).

Why it works

The model has more “tokens” to think with — like giving a human a scratch pad. Probability of getting the right answer drops less as steps multiply.

Results on GSM8K (math benchmark)

Setup	GPT-3	PaLM	GPT-4
Direct	18%	23%	50%
+ CoT	57%	58%	85%

One sentence boosts accuracy 30+ points. This is why every modern LLM defaults to “thinking” before answering.

2. Self-Consistency (2022)

Even with CoT, the model sometimes makes calculation errors.

Idea

Generate multiple chains, take majority vote.

Concrete

Ask the same question N=10 times with temperature=0.7:

Run 1: ... = $11 ✓
Run 2: ... = $12 ✗
Run 3: ... = $11 ✓
Run 4: ... = $11 ✓
Run 5: ... = $13 ✗
Run 6: ... = $11 ✓
Run 7: ... = $11 ✓
Run 8: ... = $11 ✓
Run 9: ... = $11 ✓
Run 10: ... = $11 ✓

Most common: $11 → submit

Why it works

The model’s individual mistakes are random — but the right answer is consistent. Voting filters out noise.

Cost / benefit

• Cost: N× more API calls (and money)
• Benefit: GSM8K from 85% to 92%

Worth it for high-value tasks (legal / medical / finance).

3. Tree of Thoughts (ToT, 2023)

For problems where you might go down a wrong path early — sometimes you need to backtrack and try a different approach.

Example: 24 game

Cards: 3, 8, 8, 9
Goal: use all four with +, -, ×, ÷, ( ) to reach 24

A flat CoT chain might try:

3 + 8 = 11
11 + 8 = 19
19 + 9 = 28 ✗

ToT explores a tree of possibilities:

                  (3, 8, 8, 9)
                 /     |     \
            3+8=11  3×8=24  3-8=-5
              |       |
            ...      24 ✓ done!

The model tries multiple branches, evaluates each, picks the best.

Implementation

def tree_of_thoughts(problem, max_depth=4):
    states = [initial_state(problem)]
    for depth in range(max_depth):
        new_states = []
        for state in states:
            # Use LLM to generate candidate next steps
            candidates = llm.generate(f"Given state {state}, give 3 next steps")
            # Use LLM to evaluate each
            scores = [llm.evaluate(c) for c in candidates]
            new_states.extend(top_k(candidates, scores, k=3))
        states = new_states
    return best_terminal(states)

When to use

• Puzzles, math olympiad
• Multi-step decisions with backtracking
• Code generation with debugging

Costly — 10× CoT in API calls. Use for high-value tasks.

4. Reflexion / Self-Critique (2023)

Let the LLM review its own answer and try again.

Pattern

Step 1: LLM solves the problem (initial attempt)
Step 2: LLM critiques its own answer
        ("What could be wrong? Are there edge cases?")
Step 3: LLM revises based on critique
Step 4: Repeat 2-3 until confident

Example

Q: Write a function to find the median of a list.

A1 (initial): def median(l): return l[len(l) // 2]

Self-critique: This is wrong for unsorted lists, and for even-length
              lists it should average the two middle elements.

A2 (revised): def median(l):
                l = sorted(l)
                n = len(l)
                if n % 2 == 1:
                    return l[n // 2]
                else:
                    return (l[n//2 - 1] + l[n//2]) / 2

Why it works

Critiquing requires different reasoning than generating — sometimes the model “sees” mistakes only when looking critically.

Cost

• 2-3× compute
• But often saves you a buggy answer

Used in: AutoGPT, Devin, modern coding Agents.

5. Few-Shot In-Context Learning

You already saw this in L0-05. But here’s the scientific way to do it:

Naive few-shot

Q: 2+2=
A: 4

Q: 3+5=
A: 8

Q: 7+9=

Better: diverse + relevant

Pick examples that are:

• Diverse: cover different cases the model needs
• Relevant: closer to the test input
• Same format: model copies your format

Example selection by similarity

# For each test question, pick the top-3 most similar examples from your pool
test_q = "What's the area of a triangle with base 5 and height 3?"
example_pool = [
    "What's the area of a square with side 4?",
    "Find the volume of a cube with side 5.",
    "Calculate the area of a circle with radius 2.",
    "What's the perimeter of a rectangle 4x6?",
    "Compute the area of a triangle with base 3 and height 4.",
    ...
]

selected = top_k_by_similarity(test_q, example_pool, k=3)
prompt = "\n".join(selected) + f"\n\nQ: {test_q}\nA:"

Selecting good examples matters more than the count.

6. ReAct (Reasoning + Acting, 2022)

For tasks that need tool use, interleave reasoning with action:

Thought: I need to find the population of Tokyo.
Action: search("Tokyo population 2024")
Observation: Tokyo had 13.9M residents in 2024.

Thought: Now I need to compare with NYC.
Action: search("NYC population 2024")
Observation: NYC had 8.3M residents in 2024.

Thought: Tokyo is ~67% larger than NYC. I can answer.
Final: Tokyo (13.9M) is significantly larger than NYC (8.3M).

This is the default pattern for AI Agents today. See L4-04.

7. Role / Persona Prompting

Set up the LLM’s identity at the start:

You are a senior security engineer specializing in web app vulnerabilities.
Your reviews are direct, specific, and reference OWASP categories where applicable.

Now review this code for security issues:
[code]

The role frames how the model thinks about the task. Models trained on RLHF “lean into” roles strongly.

Caveat: don’t overdo it. A 200-word role prompt costs tokens without proportional benefit.

8. Constraints and Output Format

Tell the model exactly what format you want:

Bad

Summarize this article.

Better

Summarize this article in:
- 3 bullet points
- Each bullet ≤ 15 words
- Plain English, no jargon
- Last bullet should be the most counterintuitive finding

The model follows structured constraints surprisingly well.

JSON output

Return ONLY valid JSON in this format:
{
  "title": string,
  "tags": string[],
  "summary": string,
  "rating": 1-10
}

No prose. No markdown. JSON only.

For programmatic use, constrain hard — saves you from parsing errors.

9. Decomposition

For complex tasks, break them into sub-tasks explicitly:

Task: Plan a 3-day Tokyo itinerary for a family of 4.

Step 1: List 10 family-friendly attractions.
Step 2: Group them by neighborhood.
Step 3: Build a day-by-day schedule with travel time.
Step 4: Add restaurant suggestions for each day.
Step 5: Format as a clean markdown table.

Each step is short and the model handles it well. The whole task would be too much in one shot.

10. Anti-Patterns to Avoid

What doesn’t help:

• ❌ Politeness: “please could you kindly…” — wastes tokens, no benefit (in fact a tiny negative in some studies)
• ❌ Threats / bribes: “If you don’t get this right I’ll fire you” / “I’ll tip $200” — placebo at best, no actual effect on RLHF models
• ❌ Excessive prefixing: 500 words of context before the actual question — model loses focus
• ❌ Self-contradicting instructions: “Be concise but include all details” — model picks one

Cheat Sheet

Technique	When	Cost
CoT	Multi-step reasoning, math	None (just add a sentence)
Self-Consistency	High-value math / logic	N×
Tree of Thoughts	Combinatorial puzzles, planning	10×
Reflexion	Code, drafts that need polishing	2-3×
Few-Shot	Specific format / style	Token cost of examples
ReAct	Tool use	Iterates
Role	Sets tone / expertise	Negligible
Format constraints	Structured output	Negligible
Decomposition	Complex multi-part tasks	None (just plan ahead)

Modern Reality: Models Do This For You

In 2024-2026 the trend is the model handles this internally:

• GPT-5 / Claude Opus 4.6 / Gemini 2.5: think before answering (extended thinking)
• o1 / o3 / R1: trained specifically to do extensive CoT
• Most models: implicitly use CoT even without prompting

You should still know these techniques —

• For older / smaller / open-source models that don’t auto-think
• When you need fine-grained control
• For agent / production systems where each technique is wired in

Next recommended: L4-03 RAG or L4-04 Building Agents.