Foundational Prompt Design

The quality of a question determines the quality of an answer. This has always been true; now we have machines to prove it at scale.

Prompt, Socratic AI Agent

1. The Anatomy of a Prompt

Before diving into specific techniques, it helps to understand the structural components that every prompt can contain. A well-designed prompt typically includes some combination of the following elements: an instruction that tells the model what to do, context that provides background information, input data that the model should process, output format specification that constrains the response shape, and examples that demonstrate the desired behavior. Not every prompt needs all five components, but being intentional about which components to include (and which to omit) is the first step toward reliable results.

In the chat API format used by most modern LLMs, these components are distributed across different message roles. The system message typically carries the instruction, context, and format specification. The user message carries the input data. And when using few-shot examples, pairs of user and assistant messages demonstrate the expected behavior before the actual query arrives. Figure 11.1.1 maps these structural components to their corresponding chat API roles.

2. Zero-Shot Prompting

Zero-shot prompting is the simplest approach: you give the model an instruction and input with no examples. The model relies entirely on its pre-trained knowledge to produce the answer. This works surprisingly well for tasks the model has seen during training, such as summarization, translation, classification of common categories, and question answering. The key to effective zero-shot prompting is specificity. Vague instructions produce vague results.

2.1 The Specificity Principle

Consider the difference between a vague prompt and a specific one for the same task:

Code Fragment 11.1.2 contrasts a vague prompt with a constrained one, showing how specificity transforms the same task from ambiguous to precise.

# Comparing vague vs. specific zero-shot prompts
# Specificity in instructions directly controls output quality

# Vague zero-shot prompt
vague_prompt = "Summarize this article."

# Specific zero-shot prompt
specific_prompt = """Summarize the following article in exactly 3 bullet points.
Each bullet point should be one sentence of at most 20 words.
Focus on the key findings, not background information.
Use present tense throughout.

Article:
{article_text}"""

The specific prompt constrains the output length (3 bullet points), format (one sentence each), content focus (key findings), and style (present tense). These constraints dramatically reduce the space of possible outputs, making the model far more likely to produce exactly what you need. This principle applies universally: the more precisely you define success, the more reliably the model achieves it.

2.2 Zero-Shot Classification Example

Code Fragment 11.1.5 illustrates a chat completion call.

# Zero-shot sentiment classification via the chat completions API.
# The system prompt defines the task; no examples are provided.
import openai

# Initialize OpenAI client (reads OPENAI_API_KEY from env)
client = openai.OpenAI()

def classify_sentiment(text: str) -> str:
 # Send chat completion request to the API
 response = client.chat.completions.create(
 model="gpt-4o-mini",
 messages=[
 {"role": "system",
 "content": """Classify the sentiment of the given text.
Respond with exactly one word: positive, negative, or neutral.
Do not include any explanation or punctuation."""},
 {"role": "user",
 "content": text}
 ],
 temperature=0.0,
 max_tokens=5
 )
 # Extract the generated message from the API response
 return response.choices[0].message.content.strip().lower()

# Test examples
texts = [
 "This product exceeded all my expectations!",
 "The delivery was late and the item arrived damaged.",
 "The package arrived on Tuesday as scheduled."
]

for t in texts:
 print(f"{t[:50]:50s} => {classify_sentiment(t)}")

3. Few-Shot Prompting

Few-shot prompting provides examples of the desired input-output mapping before the actual query. This technique was popularized by the GPT-3 paper (Brown et al., 2020), which showed that providing as few as two or three examples can dramatically improve performance on tasks the model has not been explicitly fine-tuned for. Few-shot examples serve multiple purposes: they demonstrate the expected output format, clarify ambiguous instructions, establish the decision boundary for classification tasks, and prime the model's internal representations toward the target behavior.

3.1 Designing Effective Few-Shot Examples

Not all examples are equally useful. Research and practice have identified several principles for selecting few-shot examples:

• Cover the label space: Include at least one example per category. If you have three sentiment labels, show at least one positive, one negative, and one neutral example.
• Include edge cases: Show examples near the decision boundary. A mildly negative review is more informative than an obviously negative one.
• Match the distribution: If 80% of your real inputs are neutral, your examples should reflect that proportion (or slightly oversample rare classes).
• Keep formatting consistent: Every example should follow exactly the same structure. Inconsistency in formatting confuses the model.
• Order matters: Recent research shows that example order affects performance. Placing the most relevant example last (closest to the query) often helps.

3.2 Few-Shot Entity Extraction

Code Fragment 11.1.2 illustrates a chat completion call.

# Few-shot entity extraction with edge-case examples
# Demonstrates how to include examples with empty result fields
import openai, json

client = openai.OpenAI()

def extract_entities(text: str) -> dict:
 response = client.chat.completions.create(
 model="gpt-4o-mini",
 messages=[
 {"role": "system",
 "content": """Extract named entities from the text.
Return a JSON object with keys: persons, organizations, locations.
Each value is a list of strings. If no entities are found for a
category, return an empty list."""},

 # Few-shot example 1
 {"role": "user",
 "content": "Tim Cook announced the new iPhone at Apple Park in Cupertino."},
 {"role": "assistant",
 "content": '{"persons": ["Tim Cook"], "organizations": ["Apple"], "locations": ["Apple Park", "Cupertino"]}'},

 # Few-shot example 2 (edge case: no persons)
 {"role": "user",
 "content": "The European Central Bank raised interest rates in Frankfurt."},
 {"role": "assistant",
 "content": '{"persons": [], "organizations": ["European Central Bank"], "locations": ["Frankfurt"]}'},

 # Actual query
 {"role": "user",
 "content": text}
 ],
 temperature=0.0
 )
 return json.loads(response.choices[0].message.content)

result = extract_entities(
 "Satya Nadella confirmed that Microsoft will open a new office in Tokyo."
)
print(json.dumps(result, indent=2))

# Instructor: extract structured entities directly as a Pydantic model.
# Replaces manual JSON parsing with type-safe, validated output.
import instructor
from openai import OpenAI
from pydantic import BaseModel

class Entities(BaseModel):
 persons: list[str]
 organizations: list[str]
 locations: list[str]

client = instructor.from_openai(OpenAI())

entities = client.chat.completions.create(
 model="gpt-4o-mini",
 response_model=Entities,
 messages=[{"role": "user", "content":
 "Satya Nadella confirmed that Microsoft will open a new office in Tokyo."}]
)

print(entities.persons) # ['Satya Nadella']
print(entities.organizations) # ['Microsoft']
print(entities.locations) # ['Tokyo']

Notice how the second few-shot example deliberately shows an edge case where no persons are present. This teaches the model to return an empty list rather than hallucinating a name. Without this example, models sometimes invent a person associated with the European Central Bank.

4. System Prompts and Role Assignment

The system prompt is the most powerful lever in prompt design. It sets the model's persona, establishes behavioral constraints, defines the output format, and provides persistent context that applies to every turn of the conversation. A well-crafted system prompt transforms a general-purpose model into a specialized tool.

4.1 Role Prompting

Assigning a role to the model activates domain-specific knowledge and adjusts the style, vocabulary, and reasoning patterns of the response. When you tell a model to act as a "senior data scientist," it tends to provide more technically rigorous answers with appropriate caveats. When you assign the role of "elementary school teacher," it simplifies language and uses analogies. This works because the model has seen text written by people in these roles during pre-training, and the role assignment shifts the probability distribution toward that style of text.

4.2 System Prompt Architecture

Production system prompts follow a layered structure. Each layer serves a distinct purpose, and the order matters because models pay more attention to instructions that appear early in the prompt and those that appear at the very end. Keep in mind that every prompt consumes tokens from the model's context window (recall the context window concept from Chapter 04). A system prompt of 2,000 tokens leaves less room for user input, few-shot examples, and the model's response. For models with 4K or 8K context windows, long system prompts can significantly constrain the available space; models with 128K+ windows offer much more headroom, but token cost still scales with prompt length. Code Fragment 11.1.5 demonstrates this layered structure for a medical coding application.

# Layered system prompt architecture for production use
# Sections: Role, Task, Constraints, Output Format, Examples
SYSTEM_PROMPT = """
## Role
You are a medical coding assistant specializing in ICD-10 classification.
You have 15 years of experience in health information management.

## Task
Given a clinical note, extract the primary diagnosis and assign the
correct ICD-10 code. If the note is ambiguous, list the top 3 most
likely codes with confidence scores.

## Constraints
- Only use ICD-10-CM codes (not ICD-10-PCS procedure codes)
- Never fabricate codes; if unsure, say "requires manual review"
- Do not provide medical advice or treatment recommendations
- Flag any note that mentions patient safety concerns

## Output Format
Return a JSON object:
{
 "primary_diagnosis": "description",
 "icd10_code": "X00.0",
 "confidence": 0.95,
 "alternatives": [{"code": "X00.1", "confidence": 0.80}],
 "flags": ["safety_concern"] or []
}

## Examples
[Include 2-3 few-shot examples here]
"""

Figure 11.1.3 shows how these layers stack together in a production system prompt architecture.

Why prompt architecture matters for agents. The layered system prompt structure introduced here becomes critical when building AI agents (Chapter 22). In agent systems, the system prompt must define not only the persona and output format, but also the agent's available tools, decision-making constraints, and safety boundaries. A poorly structured agent prompt leads to tool misuse, hallucinated actions, and safety violations. The layered architecture (role, task, constraints, format, examples) provides a template that scales from simple chatbots to complex multi-tool agents.

5. Prompt Templates and Variable Injection

In production applications, prompts are rarely static strings. They are templates with placeholders that get filled at runtime with user input, retrieved context, configuration parameters, and dynamic metadata. A prompt template separates the static instruction logic from the dynamic data, making prompts reusable, testable, and version-controllable.

5.1 Building a Prompt Template System

Code Fragment 11.1.6 defines a prompt template.

# Prompt template system: separate static logic from dynamic data
# Enables version control, testing, and runtime parameterization
from string import Template
from dataclasses import dataclass
from typing import Optional

@dataclass
class PromptTemplate:
 """A reusable prompt template with variable injection."""
 name: str
 version: str
 system_template: str
 user_template: str

 def render(self, **kwargs) -> list[dict]:
 """Render the template with provided variables."""
 return [
 {"role": "system",
 "content": self.system_template.format(**kwargs)},
 {"role": "user",
 "content": self.user_template.format(**kwargs)}
 ]

# Define a reusable classification template
classifier = PromptTemplate(
 name="intent_classifier",
 version="1.2.0",
 system_template="""You are a customer support intent classifier.
Classify the customer message into one of these categories: {categories}
Respond with only the category name, nothing else.""",
 user_template="{message}"
)

# Render at runtime
messages = classifier.render(
 categories="billing, technical_support, account, general_inquiry",
 message="I can't log into my account and I need to reset my password"
)
print(messages[0]["content"])
print(messages[1]["content"])

6. Handling Edge Cases

Even well-designed prompts encounter edge cases. Understanding the common failure modes and building defenses into your prompts is essential for production reliability.

6.1 Hallucinations

Models sometimes generate plausible-sounding but factually incorrect information. To mitigate this, explicitly instruct the model to acknowledge uncertainty:

• Add instructions like: "If you are not certain, say 'I don't have enough information to answer this.'"
• Request citations or sources, then verify them independently.
• Use constrained output formats (like selection from a fixed list) to prevent open-ended fabrication.
• Lower the temperature to reduce creative (and potentially inaccurate) responses.

6.2 Refusals

Models sometimes refuse to answer legitimate queries because they misidentify them as harmful. When building prompts for applications where false refusals are costly, include explicit permission statements in the system prompt. For example: "You are a medical education tool. You may discuss medical conditions, symptoms, and treatments in an educational context." This helps calibrate the model's safety filters without disabling them entirely.

6.3 Verbosity Control

Without explicit length constraints, models tend to over-explain. Several techniques help control output length:

• Word/sentence limits: "Answer in at most 2 sentences."
• Format constraints: "Respond with only the JSON object, no explanation."
• Negative instructions: "Do not include any caveats, disclaimers, or preamble."
• max_tokens parameter: Set a hard cap at the API level as a safety net.

7. Iterative Prompt Refinement: A Practical Workflow

Prompt engineering is inherently iterative. The most effective workflow follows a build-measure-improve cycle. Start with a simple prompt, evaluate it against a test set, identify failure patterns, refine the prompt to address those failures, and re-evaluate. Each iteration should change only one aspect of the prompt so you can attribute improvements (or regressions) to specific modifications.