Advanced Prompt Patterns

"I asked the model to improve its own prompt. It did. Then it improved the improvement. I am now unemployed."

Prompt, Self-Improving AI Agent

1. Self-Reflection and Iterative Refinement

Reflection is one of the most powerful patterns in modern prompt engineering. The core idea is simple: after generating an initial output, ask the model to critique its own work and then produce a revised version. This generate, critique, revise loop mirrors how skilled human writers and programmers work. They draft, review, and edit rather than producing a final version in a single pass.

Andrew Ng has identified reflection as a first-class agentic design pattern, noting that it provides outsized improvement relative to its implementation complexity. A single reflection pass can often close the gap between a weaker model with reflection and a stronger model without it.

1.1 Basic Reflection Loop

The simplest reflection pattern uses two sequential calls. The first call generates an initial output. The second call receives both the original task and the initial output, then critiques the output and produces an improved version.

Code Fragment 11.3.2 illustrates a chat completion call.

# Reflection loop: generate, critique, revise in multiple rounds
# Each round improves the draft based on structured self-critique
import openai

client = openai.OpenAI()

def reflect_and_refine(task: str, max_rounds: int = 2) -> dict:
 """Generate, critique, and revise in a loop."""

 # Step 1: Initial generation
 draft = client.chat.completions.create(
 model="gpt-4o",
 messages=[{"role": "user", "content": task}],
 temperature=0.7
 ).choices[0].message.content

 history = [{"round": 0, "output": draft}]

 for i in range(max_rounds):
 # Step 2: Critique the current draft
 critique = client.chat.completions.create(
 model="gpt-4o",
 messages=[
 {"role": "system",
 "content": """You are a rigorous reviewer. Analyze the draft for:
1. Correctness: Are there factual or logical errors?
2. Completeness: Is anything important missing?
3. Clarity: Is the writing clear and well-structured?
List specific issues, then rate overall quality 1-10."""},
 {"role": "user",
 "content": f"Task: {task}\n\nDraft:\n{draft}"}
 ],
 temperature=0.3
 ).choices[0].message.content

 # Step 3: Revise based on critique
 draft = client.chat.completions.create(
 model="gpt-4o",
 messages=[
 {"role": "system",
 "content": "Revise the draft to address every issue raised."},
 {"role": "user",
 "content": f"Task: {task}\nDraft:\n{draft}\nCritique:\n{critique}"}
 ],
 temperature=0.4
 ).choices[0].message.content

 history.append({"round": i + 1, "critique": critique, "output": draft})

 return {"final": draft, "history": history}

Figure 11.3.1 shows this generate, critique, and revise loop in action.

1.2 Reflexion: Memory-Augmented Self-Improvement

Reflexion (Shinn et al., 2023) extends basic reflection by adding persistent memory across attempts. When a task involves multiple trials (such as coding challenges or multi-step reasoning), Reflexion stores natural-language "lessons learned" from each failed attempt. On subsequent attempts, the agent reads these lessons before trying again, avoiding previously encountered pitfalls. This is analogous to how a human programmer keeps a mental note of bugs they have already debugged.

The Reflexion architecture has three components:

1. Actor: Generates actions or outputs for the task.
2. Evaluator: Provides a binary or scalar signal (e.g., did the code pass all tests?).
3. Self-reflection: Given the failure signal and the trajectory, generates a natural-language reflection that is stored in a memory buffer.

Code Fragment 11.3.3 illustrates a chat completion call.

# Reflexion: solve coding tasks with persistent lesson memory
# Failed test results are reflected upon and stored as natural-language lessons
import openai

client = openai.OpenAI()

def reflexion_code_solver(
 task: str, tests: list[str], max_attempts: int = 3
) -> dict:
 """Solve a coding task with Reflexion-style memory."""
 memory = [] # Persistent lessons from past failures

 for attempt in range(max_attempts):
 # Build context with accumulated lessons
 memory_str = "\n".join(
 f"Lesson {i+1}: {m}" for i, m in enumerate(memory)
 )

 prompt = f"""Solve this coding task.
Task: {task}

{"Lessons from previous attempts:" + chr(10) + memory_str if memory else ""}

Return ONLY the Python function."""

 # Generate code
 code = client.chat.completions.create(
 model="gpt-4o",
 messages=[{"role": "user", "content": prompt}],
 temperature=0.2
 ).choices[0].message.content

 # Run tests (evaluator)
 results = run_tests(code, tests)

 if all(r["passed"] for r in results):
 return {"code": code, "attempts": attempt + 1}

 # Reflect on failures and add to memory
 failures = [r for r in results if not r["passed"]]
 reflection = client.chat.completions.create(
 model="gpt-4o",
 messages=[{"role": "user",
 "content": f"""My code failed these tests: {failures}
My code was:
{code}
In one sentence, what key insight did I miss?"""}],
 temperature=0.3
 ).choices[0].message.content

 memory.append(reflection)

 return {"code": code, "attempts": max_attempts, "failed": True}

1.3 When Reflection Helps vs. When It Wastes Compute

Reflection is not universally beneficial. It adds latency (two to three extra API calls per round) and cost. Use it when the task has objectively verifiable quality criteria (tests pass, facts are correct, format matches spec). Avoid it for subjective tasks where the model may "revise" a perfectly good answer into something worse, or for simple tasks where the first-pass accuracy is already above 95%. When prompt-level iteration plateaus, consider whether fine-tuning the model itself would yield more reliable gains than additional reflection rounds.

2. Meta-Prompting: Prompts That Generate Prompts

Meta-prompting uses one LLM call to write the prompt for a subsequent call. Instead of manually crafting a system prompt, you describe what you need the prompt to accomplish and let the model generate it. This is particularly useful when you need domain-specific prompts for areas where you lack expertise, or when you want to rapidly prototype multiple prompt variants for testing.

Code Fragment 11.3.2 illustrates a chat completion call.

# Meta-prompting: use one LLM call to generate a system prompt for another
# Useful for domains outside your expertise or rapid prototyping
import openai

# Initialize OpenAI client (reads OPENAI_API_KEY from env)
client = openai.OpenAI()

def generate_expert_prompt(task_description: str, audience: str) -> str:
 """Use an LLM to generate a specialized system prompt."""
 meta_prompt = f"""You are a prompt engineering expert. Create a detailed
system prompt for an LLM that will perform the following task:

Task: {task_description}
Target audience: {audience}

Your system prompt should include:
1. A clear role definition
2. Specific output format instructions
3. Quality criteria the LLM should follow
4. Edge cases to handle
5. Two concise examples of ideal output

Return ONLY the system prompt text, no commentary."""

 # Send chat completion request to the API
 response = client.chat.completions.create(
 model="gpt-4o",
 messages=[{"role": "user", "content": meta_prompt}],
 temperature=0.7
 )
 # Extract the generated message from the API response
 return response.choices[0].message.content

# Generate a prompt for medical triage
prompt = generate_expert_prompt(
 task_description="Classify patient symptoms into urgency levels",
 audience="Emergency department nurses"
)
print(prompt[:300])

For reusable, variable-filled prompt templates, LangChain's PromptTemplate provides a declarative alternative to f-string formatting:

# Library shortcut: declarative prompt templates with LangChain
from langchain_core.prompts import ChatPromptTemplate

triage_prompt = ChatPromptTemplate.from_messages([
 ("system", "You are a {role}. Classify inputs into: {categories}."),
 ("human", "Patient symptoms: {symptoms}"),
])

# Render with variables (no manual f-string assembly)
messages = triage_prompt.invoke({
 "role": "clinical triage assistant",
 "categories": "Critical, Emergent, Urgent, Less Urgent, Non-Urgent",
 "symptoms": "chest pain, shortness of breath, onset 20 minutes ago",
})
print(messages) # ready to pass to any LangChain LLM

3. Prompt Chaining and Decomposition

Prompt chaining breaks complex tasks into a pipeline of smaller, focused LLM calls. Each call in the chain handles one well-defined subtask, and the output of one call becomes the input for the next. This is the LLM equivalent of Unix pipes or functional composition: each stage does one thing well.

Decomposition offers several advantages over monolithic prompts:

• Reliability: Each stage is simpler, so each individual call is more likely to succeed.
• Debuggability: When something fails, you can inspect intermediate outputs to identify exactly which stage broke.
• Modularity: Individual stages can be swapped, tuned, or replaced independently.
• Cost optimization: Simple stages can use cheaper, faster models while only complex stages use expensive ones.

Figure 11.3.3 depicts a four-stage prompt chain that routes each stage to an appropriately sized model.

Why prompt chaining matters for production systems. A single monolithic prompt that tries to handle classification, extraction, formatting, and error handling simultaneously is fragile and hard to debug. Prompt chaining decomposes this into a pipeline of smaller, focused prompts, each with a clear input and output contract. This mirrors software engineering best practices: small functions with single responsibilities are easier to test, debug, and optimize individually. In agent architectures, prompt chaining evolves into multi-step reasoning loops where each step may involve tool calls, conditional branching, or human-in-the-loop checkpoints.

4. DSPy: Programmatic Prompt Optimization

DSPy (Declarative Self-improving Language Programs, Khattab et al., 2023) is a framework that replaces hand-written prompts with compiled, optimizable programs. Instead of manually tweaking prompt text, you declare what each chapter should do using signatures, compose chapters into a pipeline, and let an optimizer automatically discover the best prompts, few-shot examples, and configurations.

DSPy treats prompt engineering the way PyTorch treats neural network training: you define the architecture (chapters and signatures), provide training data, and let the optimizer handle the rest.

4.1 Core Concepts

DSPy is built around three abstractions:

• Signatures: Typed input/output specifications like "question -> answer" or "context, question -> reasoning, answer". These declare what a module does without specifying how.
• Chapters: Building blocks that implement signatures. dspy.Predict does a single call. dspy.ChainOfThought adds reasoning. dspy.ReAct adds tool use. You compose chapters into programs.
• Optimizers (Teleprompters): Algorithms that tune the program. BootstrapFewShot selects optimal few-shot examples. MIPRO optimizes instructions and examples jointly. BootstrapFinetune generates training data for model finetuning.

Code Fragment 11.3.5 shows the RLHF loop.

# DSPy: declarative prompting with typed signatures and automatic optimization
# Signatures declare WHAT the module does; DSPy handles HOW
import dspy

# Configure the language model
lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

# Define a signature: what the module should do
class FactCheck(dspy.Signature):
 """Verify whether a claim is supported by the given context."""
 context: str = dspy.InputField(desc="Reference text with known facts")
 claim: str = dspy.InputField(desc="The claim to verify")
 reasoning: str = dspy.OutputField(desc="Step-by-step verification")
 verdict: str = dspy.OutputField(desc="SUPPORTED, REFUTED, or NOT ENOUGH INFO")

# Create a module that implements the signature with CoT
fact_checker = dspy.ChainOfThought(FactCheck)

# Use it (DSPy generates the prompt automatically)
result = fact_checker(
 context="The Eiffel Tower is 330 meters tall and was built in 1889.",
 claim="The Eiffel Tower is the tallest structure in Paris at over 400m."
)
print(f"Verdict: {result.verdict}")
print(f"Reasoning: {result.reasoning}")

4.2 Optimizing with DSPy

The real power of DSPy is optimization. Given a training set and a metric function, the optimizer automatically discovers the best instructions and few-shot examples for your pipeline.

Code Fragment 11.3.6 defines a reusable class.

# DSPy multi-hop QA with automatic prompt optimization via MIPROv2
# The optimizer discovers optimal instructions and few-shot examples
import dspy
from dspy.evaluate import Evaluate

# Define your program (multi-hop question answering)
class MultiHopQA(dspy.Module):
 def __init__(self):
 self.find_evidence = dspy.ChainOfThought(
 "question -> search_queries: list[str]"
 )
 self.answer = dspy.ChainOfThought(
 "question, evidence -> answer"
 )

 def forward(self, question):
 queries = self.find_evidence(question=question)
 evidence = [search(q) for q in queries.search_queries]
 return self.answer(
 question=question,
 evidence="\n".join(evidence)
 )

# Prepare training data
trainset = [
 dspy.Example(
 question="Who directed the highest-grossing film of 2023?",
 answer="Christopher Nolan"
 ).with_inputs("question"),
 # ... more examples
]

# Define a metric
def answer_match(example, prediction, trace=None):
 return example.answer.lower() in prediction.answer.lower()

# Optimize: automatically find best prompts and examples
optimizer = dspy.MIPROv2(metric=answer_match, auto="medium")
optimized_qa = optimizer.compile(MultiHopQA(), trainset=trainset)

# The optimized program has better prompts baked in
result = optimized_qa(question="What country hosted the 2024 Olympics?")
print(result.answer)

5. Automatic Prompt Engineering (APE and OPRO)

While DSPy optimizes entire programs, other approaches focus specifically on optimizing the prompt text itself. Two landmark methods illustrate this direction.

5.1 APE: Automatic Prompt Engineer

APE (Zhou et al., 2022) uses one LLM to generate candidate instructions for another LLM, then evaluates each candidate on a validation set and selects the winner. The process is straightforward: given a few input-output examples of the desired behavior, ask a model to "generate an instruction that would produce these outputs from these inputs." Generate many candidates, score them, and keep the best.

5.2 OPRO: Optimization by Prompting

OPRO (Yang et al., 2023) takes an iterative approach. It maintains a running log of previously tried prompts along with their scores. At each iteration, the optimizer LLM sees this history and generates new prompt candidates that attempt to improve on past results. This turns prompt optimization into an LLM-driven search process, as illustrated in Figure 11.3.4.

6. When Programmatic Optimization Beats Manual Engineering

The techniques above (DSPy, OPRO, APE, meta-prompting) represent a fundamental shift from artisanal prompt crafting to systematic optimization. But programmatic optimization is not always the right choice. Understanding when to invest in automated approaches versus manual tuning is itself an engineering decision with real cost implications.

Use programmatic optimization when:

• Your pipeline has multiple stages. Manual tuning of a three-stage pipeline requires optimizing each stage independently, missing interactions between stages. DSPy optimizes the full pipeline jointly, discovering configurations that no stage-by-stage approach would find.
• You have a clear evaluation metric. Programmatic optimizers need a scoring function. If you can define correctness (exact match, F1, pass/fail tests), the optimizer can exploit it. If quality is purely subjective, manual tuning with human judgment is more appropriate.
• You expect to serve high volume. Spending $10 on optimization to improve accuracy by 3% is worthwhile if the prompt serves 100,000 queries per month. For a one-time analysis, manual prompt tuning is more practical.
• You need reproducibility. Programmatic optimization produces a versioned, auditable configuration. Manual prompt tuning produces undocumented tribal knowledge that walks out the door when the prompt engineer changes teams.

Stick with manual tuning when:

• The task is a one-off or low-volume workflow.
• Quality is subjective (creative writing, marketing copy) with no ground-truth labels.
• The prompt is simple enough that a single well-crafted instruction already achieves 95%+ accuracy.
• You are in the exploratory phase and the task definition itself is still evolving.

7. Comparison of Optimization Approaches