Part VI: Agentic AI
Chapter 25: Specialized Agents

Domain-Specific Agent Design Patterns

"Every domain thinks it is special. They are all correct, and that is precisely the design challenge."

Agent X Agent X, Domain-Adapted AI Agent
Big Picture

Every domain imposes unique constraints that reshape how agents are designed. Healthcare agents operate under regulatory constraints (HIPAA, GDPR) and patient safety requirements. Legal agents must handle confidentiality, citation accuracy, and jurisdiction-specific rules. Finance agents face compliance requirements and real-time risk controls. Customer service agents need persona consistency and escalation paths. This section presents battle-tested design patterns for four high-impact domains, showing how the general agent architecture from Chapter 22 adapts to domain-specific constraints, tool sets, and success criteria.

Prerequisites

This section builds on agent foundations from Chapter 22, tool use from Chapter 23, and multi-agent patterns from Chapter 24.

1. Healthcare Agent Architectures

Healthcare agents operate under the strictest constraints of any domain. Regulatory compliance (HIPAA in the US, GDPR in Europe), patient safety requirements, liability considerations, and the need for clinical accuracy create a unique design space. Healthcare agents are never fully autonomous for clinical decisions; they always operate as decision support tools with mandatory human oversight by licensed clinicians.

The most successful healthcare agents focus on administrative and information retrieval tasks: appointment scheduling, insurance pre-authorization, clinical documentation (generating notes from doctor-patient conversations), literature search for treatment options, and drug interaction checking. These tasks benefit from automation while keeping clinical decision-making firmly in human hands. The agent architecture typically includes a clinical knowledge graph as a tool, with verified medical ontologies (SNOMED CT, ICD-10, RxNorm) providing structured medical knowledge that the agent can query.

Patient data handling requires special attention. Healthcare agents must implement role-based access control (only access the data they need for the current task), audit logging (every data access is recorded), data minimization (process only necessary patient information), and secure communication channels. The agent's context window must be managed carefully to prevent patient data from leaking across sessions. Memory systems described in Section 25.2 must be adapted with explicit data retention and deletion policies.

Key Insight

In healthcare, the most valuable agents are those that save clinician time on non-clinical tasks. Physicians spend approximately 50% of their time on documentation and administrative work. An agent that handles clinical note generation, insurance coding, or referral letter drafting frees clinician time for patient care. This "assistant" framing avoids the regulatory and liability challenges of clinical decision-making while delivering measurable ROI.

2. Legal Agent Design Patterns

Legal agents assist with contract review, legal research, compliance checking, and document drafting. The domain requires precise language, accurate citation of legal authorities, and awareness of jurisdictional variations. A contract review agent must not just find problematic clauses but explain why they are problematic, cite relevant case law or regulations, and suggest alternative language, all while accounting for the specific jurisdiction and deal type.

The most effective legal agents use a combination of RAG over legal databases (case law, statutes, regulations) and structured rule engines for compliance checking. The rule engine encodes known legal requirements as executable rules ("California consumer contracts must include a 72-hour cancellation clause"), while the RAG system provides flexibility for novel questions that rules do not cover. This hybrid approach provides both reliability for known patterns and adaptability for new situations.

Real-World Scenario: Contract Review Agent Architecture

Who: A legal operations manager at a 300-person technology company that reviewed 60 vendor contracts per month.

Situation: Two in-house attorneys spent the majority of their time on routine contract review (NDAs, SaaS agreements, consulting MSAs), leaving little capacity for higher-value work like M&A support and IP strategy.

Problem: Each contract review took approximately 4 hours, most of which was spent on standard clauses that rarely contained issues. The bottleneck was not the complex legal analysis but the mechanical process of reading every clause to find the 5 to 10% that required attention.

Decision: The team deployed a five-agent pipeline: (1) a parsing agent extracted individual clauses, (2) a classification agent categorized each clause by type (indemnification, liability, IP, termination), (3) a risk assessment agent compared each clause against company standard positions, (4) a research agent searched case law for precedents on flagged clauses, and (5) a drafting agent suggested alternative language. Every suggestion included a confidence score and citation, with low-confidence items flagged for attorney review as a guardrail.

Result: Review time dropped from 4 hours to 45 minutes per contract. Attorneys now focused exclusively on flagged clauses and high-risk provisions, freeing 60% of their time for strategic legal work.

Lesson: In expert domains, the agent's primary value is not replacing expert judgment but filtering the 90% of routine content so experts can concentrate on the 10% that genuinely requires their expertise.

3. Finance Agent Architectures

Financial agents handle tasks ranging from portfolio analysis to regulatory reporting. The domain demands numerical precision, auditability, and real-time data access. A financial agent that rounds a number incorrectly, uses stale data, or misinterprets a regulation can cause significant financial and legal consequences. These constraints shape the architecture toward verified computation (the agent writes code that is executed rather than doing math in its head), real-time data feeds (not relying on training data for current prices), and comprehensive audit trails.

Compliance is a cross-cutting concern. Financial agents must implement transaction monitoring (flag suspicious patterns), regulatory reporting (generate reports in the exact formats required by regulators), and access control (ensure traders can only access data for their authorized instruments and markets). The agent's reasoning must be fully traceable: for any output, an auditor should be able to reconstruct the data sources, the reasoning steps, and the calculations that produced it.

Warning

Never let an LLM perform financial calculations through text generation. LLMs are unreliable at arithmetic, especially with large numbers, currency conversions, and compound interest. Always have the agent write code (Python, SQL) to perform calculations, execute the code in a sandbox, and report the results. Verify critical calculations with a second, independent computation path.

Key Insight

The common pattern across all domain-specific agents is: use the LLM for reasoning and natural language, but delegate precision tasks to specialized tools. Healthcare agents use verified medical ontologies (not the LLM's training data) for drug interactions. Legal agents use structured rule engines (not prompt engineering) for compliance checks. Financial agents use executed code (not text generation) for calculations. The LLM orchestrates the workflow, interprets results, and communicates with users, but the domain-critical computations are handled by deterministic, auditable tools. This hybrid pattern (see Chapter 12 on Hybrid ML/LLM) is the key to building reliable agents in high-stakes domains. The application patterns in Section 28.2 (Finance) and Section 28.3 (Healthcare) provide additional domain-specific implementation guidance.

Exercises

Exercise 25.5.1: Domain-Specific Safety Requirements Conceptual

For each domain (healthcare, legal, finance), identify one safety requirement that is unique to that domain and would not apply to a general-purpose agent.

Answer Sketch

Healthcare: HIPAA compliance and clinical validation before any patient-facing recommendation. Legal: jurisdiction awareness (advice valid in California may be invalid in Texas). Finance: regulatory compliance (SEC rules for investment advice, anti-money-laundering checks for transactions). Each adds domain-specific constraints that shape the agent's architecture.

Exercise 25.5.2: Healthcare Agent Architecture Conceptual

Design the high-level architecture for a clinical decision support agent. What tools does it need, what guardrails must be in place, and how should it handle uncertainty?

Answer Sketch

Tools: medical knowledge base search, drug interaction checker, clinical guidelines lookup, patient record reader. Guardrails: never provide a definitive diagnosis (always 'suggest consulting a physician'), flag drug interactions as high-priority alerts, require physician approval for treatment suggestions. Uncertainty handling: express confidence levels, present differential diagnoses rather than single answers, and escalate low-confidence cases.

Exercise 25.5.3: Legal Agent Citation Requirements Coding

Write a validation function that checks whether a legal agent's response includes proper citations. Every legal claim must reference a statute, case, or regulation. Flag uncited claims.

Answer Sketch

Parse the response into individual claims (sentences). For each claim, check for citation patterns: case names (e.g., 'Smith v. Jones'), statute references (e.g., '26 U.S.C. section 501'), or regulation codes. Return a list of uncited claims with their positions. A production system would also verify that cited cases and statutes actually exist using a legal database lookup.

Exercise 25.5.4: Finance Agent Compliance Conceptual

A finance agent provides investment recommendations. Describe the regulatory and ethical guardrails it must have, and explain the consequences of failing to implement them.

Answer Sketch

Required guardrails: disclaimer that output is not financial advice, suitability checks (recommendations must match the user's risk profile), conflict of interest disclosure, audit trail of all recommendations, and compliance with SEC/FINRA rules. Failure consequences: regulatory fines, legal liability for unsuitable recommendations, loss of customer trust, and potential ban from providing advisory services.

Exercise 25.5.5: Cross-Domain Patterns Conceptual

Identify three design patterns that are common across healthcare, legal, and finance domain agents. Why do these patterns recur in high-stakes domains?

Answer Sketch

(1) Human-in-the-loop for critical decisions (because errors have severe consequences). (2) Audit logging of all agent actions (because regulators require traceability). (3) Citation and evidence requirements (because claims must be verifiable). These patterns recur because high-stakes domains share the need for accountability, traceability, and human oversight.

Key Takeaways
Self-Check
Q1: What unique challenges do healthcare agent architectures face compared to general-purpose agents?
Show Answer

Healthcare agents must handle strict regulatory requirements (HIPAA, clinical validation), high-stakes decisions where errors can harm patients, the need for explainability and auditability, integration with electronic health records (EHR), and the requirement for physician oversight.

Q2: Why is human-in-the-loop especially critical for healthcare agents?
Show Answer

In healthcare, incorrect agent outputs can lead to misdiagnosis, inappropriate treatment recommendations, or missed critical conditions. Human oversight ensures that a qualified clinician reviews agent outputs before they affect patient care, satisfying both safety and regulatory requirements.

What Comes Next

In the next section, SWE-bench and Agentic Software Engineering Evaluation, we take a deep look at how the SWE-bench benchmark evaluates agent performance on real-world software engineering tasks.

References and Further Reading

Domain-Specific Agent Applications

Thirunavukarasu, A.J., Ting, D.S.J., Elangovan, K., et al. (2023). "Large Language Models in Medicine." Nature Medicine.

Comprehensive review of LLM applications in healthcare covering clinical decision support, documentation, and patient communication, providing the foundation for healthcare agent design.

Paper

Guha, N., Nyarko, J., Ho, D.E., et al. (2024). "LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models." NeurIPS 2023.

Provides a comprehensive legal reasoning benchmark that informs the design of legal agent systems, covering tasks from contract analysis to statutory interpretation.

Paper

Wu, S., Irsoy, O., Lu, S., et al. (2023). "BloombergGPT: A Large Language Model for Finance." arXiv preprint.

Describes a domain-specific LLM trained on financial data, demonstrating the value of specialized pretraining for building effective financial analysis agents.

Paper

Safety in High-Stakes Domains

Xi, Z., Chen, W., Guo, X., et al. (2023). "The Rise and Potential of Large Language Model Based Agents: A Survey." arXiv preprint.

Surveys agent applications across domains including healthcare, law, and finance, covering domain-specific safety requirements and design considerations.

Paper

Kapoor, S., Stroebl, B., Siber, Z.S., et al. (2024). "AI Agents That Matter." arXiv preprint.

Discusses evaluation methodology for agent reliability in production settings, particularly relevant for high-stakes domains requiring consistent performance guarantees.

Paper