Advanced Prompt Engineering Techniques That Hold Up

Advanced prompt engineering is a small set of techniques with published results behind them, plus a large amount of folklore. This covers the first group: what each technique does, the paper that established it, and the conditions under which it stops paying for itself.

The honest summary before the detail:

• Few-shot examples are the highest-return technique and the least discussed as "advanced"
• Chain-of-thought helps on multi-step reasoning and adds cost and latency everywhere else
• Self-consistency buys accuracy by running the same problem several times and voting — expensive, and sometimes worth it
• ReAct and tool use are what turn a prompt into an agent
• Decomposition — splitting one hard prompt into several easy ones — outperforms most single-prompt cleverness

The meta-point: newer models need less of this. Techniques that gave large gains on 2022-era models often give little on current ones, because the behaviour got trained in. Measure on the model you're actually using rather than trusting a result from three model generations ago.

Quick Comparison

Few-Shot: Still the Best Return

Putting examples of the task directly in the prompt, so the model infers the pattern rather than being described it.

This was the central finding of the GPT-3 paper — models perform tasks from examples in the prompt without any retraining. It remains the highest ratio of improvement to effort available.

What makes examples work:

• Cover the edge cases, not the easy ones. Three examples of a straightforward case teach less than one straightforward and two awkward ones
• Keep the format identical across examples. Inconsistency in your examples produces inconsistency in the output
• Include a "none of the above" example if that's a valid answer, or the model will always pick something
• Three to five is usually the plateau. More rarely helps and costs context

Where it fails: when your examples accidentally share a feature you didn't intend. If all three sample invoices are from the same vendor, the model may learn the vendor rather than the task.

Chain-of-Thought: Real, and Narrower Than Advertised

Asking the model to work through its reasoning before answering. The original result showed substantial gains on arithmetic and multi-step reasoning benchmarks.

Where it genuinely helps: multi-step arithmetic, logic puzzles, structured comparisons, anything where an intermediate mistake propagates.

Where it costs without helping: classification, extraction, formatting, summarization. You pay for the extra tokens and the latency and get the same answer.

Two practical notes that matter more than the technique itself:

The reasoning is not an explanation. A model's stated reasoning is generated text that may or may not describe how it arrived at the answer. Useful for spotting an obviously wrong step; not evidence of the actual process, and not something to hand a regulator as an audit trail.

Current reasoning models do this internally. Instructing a model that already reasons by default to "think step by step" is redundant and occasionally counterproductive. Check what your model does before adding it.

Self-Consistency: Paying for Accuracy

Run the same prompt several times, take the most common answer. The paper showed it improves on chain-of-thought by marginalising over reasoning paths.

Straightforward and expensive — five samples cost five times as much. It earns that when:

• The answer is a discrete value that can be compared across runs
• Being wrong is costly enough to justify the multiplier
• You've confirmed the model is inconsistent on this task in the first place

It's useless for open-ended generation, since you can't take a majority vote over five differently-worded paragraphs. Reserve it for classification, extraction, and numerical answers.

A cheaper approximation: run twice, and escalate to a human or a stronger model only when the two disagree. You get most of the error detection at a fraction of the cost.

ReAct and Tool Use: Where Prompting Becomes Agents

ReAct interleaves reasoning with actions — the model thinks, calls a tool, reads the result, thinks again. This is the foundation under most current agent frameworks.

What it fixes: a model reasoning alone is limited to what it was trained on and what you pasted. Give it search, a calculator, or a database and it can work from current, specific information.

What it introduces: every tool call is a place to fail. The loop can repeat, stall, or pursue an unproductive path. Practical deployments need a step limit, a per-tool permission scope, and logging of the intermediate steps — the same controls that matter for any LLM agent.

The tree-of-thoughts approach extends the idea to exploring several reasoning branches and backtracking. Genuinely powerful on search-like problems, and costly enough that it's rarely the right first move.

Decomposition: The Underrated One

Splitting one complicated prompt into a sequence of simple ones.

A prompt asked to extract data, evaluate it, and write a recommendation will do all three adequately. Three prompts each doing one job will do each well — and when the result is wrong, you know which step failed.

Why it beats single-prompt engineering: you can test each step, fix one without disturbing the others, and use a cheap model for the mechanical steps and an expensive one only for the judgment. That last point often cuts cost while improving quality.

The cost: more moving parts, more latency, and something to maintain. Worth it once a prompt is running repeatedly rather than being typed once.

What to Skip

Techniques with a lot of circulation and little evidence:

• Emotional pressure — "this is very important to my career." Reported effects are inconsistent and model-specific
• Elaborate personas — "you are a world-class expert with 30 years of experience." A brief role helps set register; the credentials do nothing
• Threats and rewards. No
• Very long instruction preambles. Past a point they dilute the actual task, and the model's attention to any one instruction drops

A useful test: if a technique has no mechanism you can articulate and no published result, treat it as folklore until you've measured it on your own task. Structured methods, by contrast, are covered in our guide to prompt engineering frameworks — those are organising devices rather than performance claims, which is a fair thing for them to be.

Measuring, Which Is the Actual Advanced Skill

Every technique above has conditions. The skill that separates people who are good at this is not knowing more techniques — it's having a way to tell whether a change helped.

The minimum viable version:

1. Twenty real cases with known-good answers. Not synthetic ones
2. A pass criterion you can apply consistently — exact match, a checklist, or a second model scoring against a rubric
3. Run the baseline three times, because the variance will surprise you
4. Change one thing, run again, compare
5. Keep the cases. They're what tells you a model update broke something

Without this you're guessing, and prompt changes that feel like improvements frequently aren't. Once you have it, most of the folklore falls away on its own.

If assembling and re-running that kind of setup is the part that stops you, Taku mirrors working AI configurations into a desktop workspace and runs them without the environment work — the free app library is the place to see what that covers. Taku is in Beta, and the Mac app is available now.

Key Points

• Few-shot examples give the best return — cover edge cases, keep the format identical, three to five is the plateau
• Chain-of-thought helps multi-step reasoning and adds cost elsewhere; current reasoning models often do it already
• Stated reasoning is not an explanation of how the model actually got there
• Self-consistency buys accuracy with money and only works for discrete answers
• ReAct turns prompting into agents and needs step limits, scoped permissions, and logged intermediate steps
• Decomposition beats single-prompt cleverness — testable steps, and cheap models for the mechanical ones
• Newer models need less technique. Measure on the model you're using, not on a three-generation-old result
• The real advanced skill is evaluation: twenty real cases and a consistent pass criterion

FAQ

What are advanced prompt engineering techniques?

Few-shot examples, chain-of-thought reasoning, self-consistency sampling, ReAct-style tool use, and task decomposition. Each has published research behind it and conditions where it stops being worth the cost — none is a universal improvement.

Does chain-of-thought prompting still work?

On multi-step reasoning, yes. On classification, extraction, and formatting it adds tokens and latency without improving the answer. Current reasoning models also do this internally, so explicitly instructing them to think step by step is often redundant — check your model's behaviour before adding it.

What AI prompting techniques are worth learning first?

Few-shot examples and decomposition. Examples cost almost nothing and fix most format and style problems; splitting a complex prompt into simple steps makes results testable and usually cheaper, since mechanical steps can run on a smaller model.

Is self-consistency worth the cost?

Only when the answer is discrete enough to vote on and being wrong is expensive. A cheaper version is running twice and escalating only on disagreement, which catches most errors without the full multiplier.

How do I know if a prompt change actually improved anything?

Build a set of twenty real cases with known-good answers, run the baseline several times to see the natural variance, then change one thing and compare. Single-run comparisons are unreliable because generation is probabilistic, and many changes that feel better aren't.