Automation first, AI second: use LLMs only where judgment is needed
TL;DR
Steps with clear rules (filtering, format conversion, API calls, math) → plain logic: cheap, fast, stable, testable. Steps needing language understanding, fuzzy classification, summarizing, drafting → an LLM. Result: lower cost and fewer unpredictable failures.
Classifying workflow steps
| Step | Use | Why |
|---|---|---|
| Filter email by sender / subject | Rule | Clear rule, 100% accurate |
| Read content, detect customer intent | LLM | Natural language, ambiguous |
| Look up an order in the DB | Code / API | Structured data |
| Draft a polite reply | LLM | Context-aware writing |
| Compute refunds, delivery dates | Code | “Roughly right” isn't allowed |
| Approve refunds above a threshold | Human | Financial risk |
Questions to ask per step
- Can you write an if/else or regex for it? → If yes, don't use AI.
- What does one mistake cost? → If high, add human approval.
- Does the output need a fixed structure? → Enforce a JSON schema + validate.
References: Building effective agents — Anthropic