The Limits of the Chatbot Mental Model
The Default Mental Model
When most people think of AI agents, they think of chatbots. A conversation thread. A user types a message. The model responds. Maybe it calls a tool. The conversation continues. History accumulates.
This mental model is seductive because it is simple. But simplicity is not the same as sufficiency. The chatbot model breaks in fundamental ways when applied to real-world agentic work.
Where It Breaks
Linear History
Chatbots treat interaction as a flat sequence of messages. There is no branching, no parallel threads, no hierarchy of tasks and subtasks. Every exchange is appended to one growing conversation. This makes it impossible to represent the kind of structured, concurrent work that real problem-solving requires.
Poor Isolation
In a chatbot, everything happens in one shared context. A tool call to read a database, a reasoning step about strategy, and a response to the user all occur in the same space. There is no boundary between concerns. A failure in one area can corrupt the others.
No Explicit Permissions
Chatbots do not have a permission model. The model can call any tool it has access to, with any arguments, at any time. There is no concept of “this agent may read but not write,” or “this action requires approval.” The security model is all-or-nothing.
No Real Process Model
There is no concept of spawning a worker, scoping its task, limiting its context, and collecting its result. Every step happens in the main thread with the full context. This is like running an operating system with one process and no memory protection.
Weak Memory Discipline
Context windows fill up. Older messages get truncated or lost. There is no tiered memory. No compression. No selective retrieval. No distinction between working memory, episodic memory, and long-term knowledge. The system forgets randomly rather than strategically.
Low Auditability
When something goes wrong in a chatbot, the debugging experience is reading the transcript. There are no structured logs, no execution traces, no decision records, no policy evaluations. You cannot replay, inspect, or verify what happened and why.
The Cost of the Wrong Model
These are not minor inconveniences. They are architectural limitations that prevent agentic systems from being reliable, safe, and composable at scale. Building production agentic software on the chatbot model is like building a web application with global variables and no error handling. It works in demos. It fails in reality.
We Need a Better Abstraction
The chatbot model is a UI pattern, not an architectural pattern. It describes how humans interact with models, not how systems should be structured. To build agentic systems that work, we need to separate the interaction model from the operational model — and design the operational model with the rigor it deserves.