The Observe-Steer Development Loop

"You don't spec the whole building before picking up the hammer anymore. You describe the room, watch it appear, fix what's wrong, and describe the next room."

Deploy, Relentlessly Iterative AI Agent

1. What Is Vibe Coding?

"Vibe coding" refers to a style of software development where the programmer describes what they want in natural language (or even by pointing at a screenshot) and an AI assistant generates the code. The programmer's role shifts from writing syntax to reviewing, steering, and constraining output. Think of it as pair programming where one partner is an LLM with encyclopedic knowledge of libraries and patterns but no understanding of your business context or quality requirements.

The term first gained traction in early 2025 when Andrej Karpathy described his workflow of "just vibing" with an AI, accepting whatever code it produced, and iterating on the results. While his description was deliberately provocative, the underlying shift is real: AI coding assistants (GitHub Copilot, Cursor, Claude Code, Windsurf, and others) have made it possible to generate substantial codebases from natural language descriptions in minutes rather than days.

The critical distinction lies in how you handle the generated output. Two approaches exist along a spectrum:

2. The Observe-Steer Loop

Traditional software development follows a plan-build-test cycle: write a specification, implement it, then verify. AI-assisted development replaces the heavy upfront specification with a tighter feedback loop that we call the observe-steer loop:

1. Describe. State your intent in natural language. Be specific about constraints and non-negotiables, but do not over-specify implementation details.
2. Generate. Let the AI produce code, configuration, or content.
3. Observe. Run the output. Read it carefully. Check it against your evaluation criteria. Use the observability tools from Chapter 30 to inspect traces, latency, and costs.
4. Evaluate. Does the output meet your quality gates? Run your evaluation suite (Chapter 29). Not just "does it look right?" but "does it pass the rubric?"
5. Steer. Based on what you observed, refine your prompt, add constraints, request changes, or accept the result. Feed the observation back into the next iteration.

This loop runs in minutes, not days. A single coding session might execute 10 to 50 iterations. The key difference from ad-hoc prompting is that each iteration is grounded in observable evidence (test results, traces, metrics) rather than subjective impressions.

3. Documentation as a Control Surface

In an observe-steer workflow, documentation is not an afterthought that you write once the code is "done." It is the primary control surface for the AI assistant. The documents you maintain directly shape what the AI generates, what constraints it respects, and how you verify its work.

Four types of documentation serve as active control surfaces:

• Intent documents. What the system must do, what it must never do, and who approved those decisions. These function like a requirements specification but are written in plain language that both humans and AI assistants can consume. When you paste your intent document into a coding session, the AI inherits your constraints.
• Prompt templates. Versioned prompts with placeholders, stored alongside the code they support. As discussed in Chapter 11, prompts are a form of source code and deserve the same version control discipline.
• Evaluation artifacts. Golden test sets, scoring rubrics, and regression baselines. These define "good enough" in machine-readable form, enabling automated quality gates.
• Risk and cost records. Threat models, token budgets, and routing strategies. These constrain architectural decisions and prevent cost surprises.

4. Lock-in Dynamics and Switching Costs

AI-assisted development changes the economics of switching between tools, providers, and architectures. Some switching costs decrease; others increase.

Costs that decrease:

• Refactoring cost. AI assistants can refactor entire codebases between frameworks, languages, or API providers in hours rather than weeks. Migrating from one LLM provider's SDK to another is now a task you can describe in a prompt, not a multi-sprint project.
• Boilerplate rewriting. Switching from REST to GraphQL, or from one database to another, becomes cheaper when an AI can generate the adapter layer.

Costs that increase (data gravity):

• Evaluation data. Your golden test sets, scoring rubrics, and regression baselines are tightly coupled to your current model and prompt design. Switching models means re-validating your entire evaluation suite.
• Fine-tuning investments. If you have fine-tuned a model on proprietary data, that investment is locked to one provider.
• Prompt libraries. Hundreds of carefully tuned prompts represent significant intellectual capital. While prompts are portable in theory, they often exploit model-specific behaviors that do not transfer cleanly.
• Trace and feedback data. Months of production traces, user feedback, and quality metrics are your most valuable asset for improvement. This data has gravity; it pulls your decisions toward the system that generated it.

5. The Intent + Evidence Bundle (IEB)

The deliverable for this section is the Intent + Evidence Bundle (IEB): a version-controlled folder that lives alongside your code and captures everything needed to understand, reproduce, and audit your AI product's behavior. The IEB is the professional answer to the question "How do I keep vibe coding under control?"

The IEB contains five components:

The following script initializes an IEB folder structure with template files.

# Initialize an Intent + Evidence Bundle (IEB) folder structure
# Run once at project start to scaffold the IEB directory
import os
from pathlib import Path
from datetime import date

def init_ieb(project_root: str, project_name: str = "My AI Project") -> Path:
 """Create the IEB folder structure with starter templates.

 Args:
 project_root: Path to the project's root directory.
 project_name: Human-readable project name for template headers.

 Returns:
 Path to the created ieb/ directory.
 """
 ieb_root = Path(project_root) / "ieb"
 today = date.today().isoformat()

 # Define folder structure
 folders = ["eval", "prompts"]
 for folder in folders:
 (ieb_root / folder).mkdir(parents=True, exist_ok=True)

 # --- intent.md ---
 intent_content = f"""# Intent Document: {project_name}
**Created:** {today}
**Owner:** [Your Name]
**Status:** Draft

## What This System Must Do
- [ ] [Describe the core capability in one sentence]

## What This System Must Never Do
- [ ] [List safety and compliance non-negotiables]

## Approved Scope
- [ ] [Define boundaries: what is in scope, what is out]

## Stakeholder Approvals
| Name | Role | Date | Notes |
|------|------|------|-------|
| | | | |
"""
 (ieb_root / "intent.md").write_text(intent_content, encoding="utf-8")

 # --- risk.md ---
 risk_content = f"""# Risk Register: {project_name}
**Last reviewed:** {today}

## Threat Model
| Threat | Likelihood | Impact | Mitigation |
|--------|-----------|--------|------------|
| Adversarial prompt injection | Medium | High | Input validation, guardrails |
| Data leakage in responses | Low | Critical | Output filtering, PII detection |
| Model hallucination | High | Medium | Eval suite, confidence thresholds |

## Open Risks
- [ ] [List unmitigated risks here]
"""
 (ieb_root / "risk.md").write_text(risk_content, encoding="utf-8")

 # --- cost.md ---
 cost_content = f"""# Cost and Routing Strategy: {project_name}
**Last reviewed:** {today}

## Token Budget
| Scenario | Input tokens | Output tokens | Model | Cost/call |
|----------|-------------|---------------|-------|-----------|
| Typical query | ~500 | ~200 | gpt-4o-mini | ~$0.0003 |
| Complex query | ~2,000 | ~800 | gpt-4o | ~$0.01 |

## Routing Strategy
- Route simple queries to the smaller, cheaper model.
- Escalate to the larger model when confidence is below threshold.
- See Chapter 10 for API routing patterns.

## Monthly Budget Target
- Target: $[amount]/month at [volume] queries/day
"""
 (ieb_root / "cost.md").write_text(cost_content, encoding="utf-8")

 # --- eval/golden.jsonl (starter with one example) ---
 golden_example = (
 '{"input": "What is your refund policy?", '
 '"expected_intent": "refund_inquiry", '
 '"expected_contains": ["refund", "policy"], '
 '"notes": "Baseline test case"}\n'
 )
 (ieb_root / "eval" / "golden.jsonl").write_text(
 golden_example, encoding="utf-8"
 )

 # --- prompts/v001_system_prompt.txt ---
 prompt_template = f"""# System Prompt v001
# Created: {today}
# Changelog: Initial version

You are a helpful assistant for {project_name}.

## Instructions
- Answer user questions accurately and concisely.
- If you are unsure, say so explicitly.
- Never fabricate information.

## Constraints
- Do not discuss competitors.
- Do not reveal internal system details.
- Keep responses under 300 words unless the user requests more detail.
"""
 (ieb_root / "prompts" / "v001_system_prompt.txt").write_text(
 prompt_template, encoding="utf-8"
 )

 print(f"IEB initialized at: {ieb_root}")
 print(f" intent.md - Define your non-negotiables")
 print(f" risk.md - Document threats and mitigations")
 print(f" cost.md - Set token budgets and routing rules")
 print(f" eval/ - Add golden test cases to golden.jsonl")
 print(f" prompts/ - Version your prompt templates here")
 return ieb_root


# Example usage
if __name__ == "__main__":
 init_ieb(".", project_name="Customer Support Bot")

6. Putting It Together: The Responsible Vibe Coding Workflow

The following workflow combines the observe-steer loop with the IEB into a repeatable development process:

1. Initialize. Run init_ieb(). Fill in intent.md with non-negotiables. Seed eval/golden.jsonl with at least 10 test cases.
2. Describe. Open your AI coding assistant. Paste intent.md as context. Describe the feature you want to build.
3. Generate. Let the assistant produce code. Accept the output into your working directory.
4. Observe. Run the code. Examine the output. Check traces using your observability stack (Chapter 30).
5. Evaluate. Run the evaluation suite (Chapter 29) against your golden set. Record pass/fail rates.
6. Steer. If evaluation fails, refine the prompt or constraints. If a new failure mode appears, add it to the golden set. If a risk becomes apparent, update risk.md.
7. Commit. Commit all changes: code, prompts, eval results, and any IEB updates. Use meaningful commit messages that reference the steer that motivated the change.
8. Repeat. Return to step 2 for the next feature or refinement.