AI Agent vs Plain Chatbot — A Field Perspective

“Do we need a chatbot or an agent?” I get this question every week in client meetings. Sounds like word choice — it’s actually the call that drives architecture, cost, and operational risk. Here’s how I draw the line in practice.

Definitions first

Mixing the two words wrecks the design. Here’s how I split them.

A chatbot takes user input, calls the LLM once, returns the answer. Almost no state, linear flow, no writes to external systems. FAQ bots, first-line support, simple Q&A — that’s chatbot territory.

An AI agent receives a goal and runs multiple steps autonomously. It calls tools, observes results, decides what to do next. It keeps state and actually writes to external systems. Contract pipelines, code review bots, some internal assistants — those are real agents.

Two key differences: autonomy, and the existence of a loop.

What that looks like on real projects

A case where a chatbot was enough

A European SaaS client wanted to “automate support with an AI agent.” I pulled three months of tickets and read through them: 80% were straightforward FAQ. Conclusion: one RAG chatbot, done.

• Input: customer question
• Processing: retrieve relevant chunks from internal docs → Claude generates the answer
• Output: answer + sources

No tools, no loop. And yet — 45% of tickets resolved automatically, ROI within three months. Here, the chatbot was the right answer. Pushing for an agent would have doubled or tripled the cost for nothing.

A case where an agent was the only option

Another client wanted “to receive a contract PDF, extract mandatory clauses, and push them into Salesforce.” Clearly multiple distinct steps:

1. Extract text from the PDF
2. Classify clauses (payment, termination, liability, etc.)
3. Attach metadata to each clause
4. Check consistency against existing contracts
5. Call the Salesforce API to create the record
6. Escalate to a human on failure

Trying to do this with a chatbot turns the code into a patchwork. The LLM has to call tools, observe results, branch. I built it with the Claude Agent SDK, and it now processes hundreds of contracts a day.

The implementation difference

In code, the difference is obvious.

Chatbot: linear function

async function handleMessage(query: string, userId: string) {
  const chunks = await retrieve(query, userId);
  const answer = await claude.messages.create({
    model: "claude-sonnet-4-5",
    max_tokens: 1024,
    messages: [
      { role: "user", content: buildPrompt(query, chunks) },
    ],
  });
  return answer;
}

One LLM call, one DB lookup. Done. Debugging is trivial.

Agent: loop + tools + state

async function runAgent(goal: string, userId: string) {
  const state = { messages: [{ role: "user", content: goal }], done: false };

  while (!state.done && state.messages.length < MAX_STEPS) {
    const response = await claude.messages.create({
      model: "claude-sonnet-4-5",
      max_tokens: 2048,
      tools: AVAILABLE_TOOLS,
      messages: state.messages,
    });

    state.messages.push({ role: "assistant", content: response.content });

    if (response.stop_reason === "tool_use") {
      const toolResults = await executeTools(response.content, userId);
      state.messages.push({ role: "user", content: toolResults });
    } else {
      state.done = true;
    }
  }

  return state;
}

Loop, tool execution, max steps, failure handling, audit logs. Complexity jumps an order of magnitude. So does the operational cost.

When to skip the agent

Three traps I see often.

When a decision tree is enough. An agent costs 5–10× more and is non-deterministic. Rule-based logic or a simple chatbot does a better job.

When a single call gives you the answer. “Summarize this document” doesn’t need an agent. One LLM call is enough.

Real-time transactions where failure is expensive. Payments, anything with legal weight — don’t ship a fully autonomous agent. Design a workflow where a human signs off at the end. No compromise on this.

When the agent earns its keep

Conversely, when these line up, the agent clearly wins:

• The task has multiple steps, and the steps depend on the input.
• You need to orchestrate several external systems.
• You can afford the run to take minutes.
• Failures are recoverable — retry, or hand off to a human.

Contract processing, research tasks, code review automation, complex data migrations. That’s the profile.

Practical advice

Decide early. Scaling a chatbot up to an agent is easy. Walking back a badly designed agent into a chatbot is a full rewrite.

And even when you do build an agent, don’t hand everything to the LLM. A good agent system clearly separates deterministic code (the flows you control) from LLM calls (the spots that need judgment). At each step I ask myself: “Would an if/else do?” If the answer is yes, it’s an if/else. That’s the core of an agent you can actually run in production.

If you’re weighing whether to go with an agent, I offer a free 30-minute call to walk through your use case together.