ReAct and Toolformer: the origins of every agent's think–act loop
TL;DR
ReAct gave agents the think–act–observe loop; Toolformer showed models can learn when to call tools. Every modern agent inherits both strengths and weaknesses: cap steps, describe tools clearly, treat tool results as untrusted data.
Every agent today, from Claude Code to research assistants, runs the same loop: think → call a tool → read the result → think again. That loop has clear roots in two papers from 2022–2023: ReAct (Princeton and Google) and Toolformer (Meta AI). Understanding them explains why agents are powerful, and why they often "get lost".
ReAct: interleaving reasoning and acting
Yao et al. (2022) observed that reasoning alone (chain-of-thought) makes models prone to inventing facts since they can't check against the world, while acting alone leaves them without a plan. ReAct combines the two into an interleaved trace:
Thought: I need to know when company X was founded.
Action: search["Company X founded"]
Observation: Company X was founded in 1998 in...
Thought: I have the founding year; now find the current CEO.
Action: search["Company X CEO"]
...- For question answering (HotpotQA) and fact verification (FEVER), looking things up on Wikipedia reduced hallucination compared with pure CoT.
- In the interactive environments ALFWorld and WebShop, ReAct beat imitation and reinforcement learning methods by absolute success rate gains of 34% and 10% respectively, with only one or two examples in the prompt.
- Visible reasoning traces let humans read and correct agent behaviour.
Toolformer: models that teach themselves when to call tools
Schick et al. (2023) asked a different question: instead of teaching through prompts, can a model teach itself to use APIs? The approach:
- The model inserts candidate API calls into text (calculator, Q&A, search, translation, calendar).
- Those calls are executed.
- Only calls that help predict the following text are kept, and the model is fine-tuned on that data.
The result: a 6.7-billion-parameter model using tools beat much larger models on some tasks such as arithmetic, without losing general language ability.
From research to today
| Original idea | Today's form |
|---|---|
| Thought / Action / Observation as plain text | Thinking blocks, structured tool_use and tool_result in the API |
| A few fixed tools (search, lookup) | Hundreds of tools via MCP, with tool search to load them on demand |
| Models learning when to call tools | Models trained to use tools, choosing them from their descriptions |
Risks inherited from the ReAct loop
- Infinite loops: agents call the same tool repeatedly when results disappoint. Cap the number of steps.
- Compounding errors: one wrong observation early on skews every later step.
- Tool results are untrusted data: returned text can contain malicious instructions (prompt injection).
- Bad tool descriptions = wrong calls: the model picks tools by name and description; vague descriptions cause mistakes.
How to apply it
- Dev: When an agent fails, read the thought/action/observation trace to find the first step that went wrong, not just the final output.
- Tech Lead: Set step limits, timeouts and stop conditions for every agent loop; log every tool call in full.
- PM: For a new task, start with 2–3 clear tools; add more when real needs appear, rather than a dozen from day one.
Sources & references
- ReAct: Synergizing Reasoning and Acting in Language Models — Yao et al. (arXiv)
- Toolformer: Language Models Can Teach Themselves to Use Tools — Schick et al. (arXiv)