Most "AI Agents" in Production Are Just LLMs With Tool Calls

Most systems marketed as AI agents are language models with tool calls and nothing more. They can query an API, pull data and trigger an action, but they don't maintain memory across sessions, they don't decompose goals into sub-tasks on their own and they don't self-correct when something breaks. That distinction changes your build decisions, your purchasing and your expectations going forward.

The Spectrum From LLM to True AI Agent

There's a spectrum between a plain language model and a system that deserves the word "agent," and it helps to think of it in four levels.

Level 1: LLM alone. It generates text from a prompt. No external actions, no data lookups, no integrations. You type, it responds.

Level 2: LLM with tool calls. The model can request that an external function be run (query a database, call an API, send an email) and receive the result. This is where the vast majority of production "agents" sit today.

Level 3: Orchestrated multi-step pipeline. Multiple LLM calls chained together with conditional logic. If step one returns X, route to step two; if it returns Y, route to step three. A human designs the flow in advance.

Level 4: Autonomous agent. The system maintains memory across sessions, decomposes goals into sub-tasks independently and self-corrects when it hits an error.

Lilian Weng's widely cited taxonomy from "LLM Powered Autonomous Agents" (June 2023) defines three pillars of genuine agency: Memory, Planning and Tool Use. Most systems sold as agents have Tool Use and nothing else, one pillar out of three that Weng's taxonomy requires.

Tool calling, as documented by both Anthropic and OpenAI, is a structured way for an LLM to request that an external function be run and receive the result. It's a feature of the model interface, and it tells you nothing about whether the system reasons on its own.

Frameworks like LangChain, CrewAI and AutoGPT can build toward level-four systems, but they're frequently used to build level-two or level-three tool-call chains. The framework name on the architecture diagram does not determine where the system sits on this spectrum. 

Three Questions That Reveal What You Have

You don't need a technical background to figure out whether what you've been sold is a pipeline or an agent. Three questions will get you there, and you can ask them during a vendor demo.

1. Does it maintain memory across sessions without being re-prompted? Ask the vendor to close the session, reopen it and reference something from the prior conversation. A system with genuine agency retains that context. Most don't.

2. Does it self-correct when it hits an error, or does it stop and wait? Ask the vendor what happens when the system gets bad data from an API call. An answer like "it logs the error" tells you it's a pipeline.

3. Does it decompose a goal into sub-tasks on its own, or does a human define each step? Planning means the system receives an objective and generates its own task sequence. Prompt-chained systems require a human to pre-define every step in advance. Give the system a goal it hasn't been explicitly configured for and watch what happens.

If the answer to all three is no, the system is not an agent. That's not necessarily a problem. But you should know what you have, because it changes what you can expect from it and what it costs to maintain.

Why Most Mid-Market Workflows Don't Need True Agents

This is where the argument gets counterintuitive: for most operations workflows at mid-market companies, a well-engineered pipeline with tool calls is the better architecture. Cheaper, more predictable, easier to maintain.

Autonomous agents introduce non-determinism by design. The system makes its own decisions about how to reach a goal. For workflows where consistency and auditability matter (order processing, invoice matching, compliance reporting), that unpredictability is a liability, not a feature. When the system's reasoning path isn't pre-defined, the failure modes are harder to diagnose. You can't trace why it did what it did because it decided on its own.

The workflows common across Michigan manufacturing, distribution and professional services illustrate this well: inventory lookups, quote generation, document routing, customer status updates. These are high-volume, well-defined processes where the value comes from reliability and speed, not from the system thinking for itself.

The cost difference is worth naming. Building and maintaining a true agentic system requires more engineering, more testing and more ongoing monitoring than a well-designed pipeline. For most of these use cases, the added complexity doesn't return proportional value.

Most operations leaders we work with don't need their system to think for itself. They need it to do the right thing, every time, without someone babysitting it.

Where True Agents Are Worth the Complexity

There are genuine use cases where autonomous, self-correcting agents justify the added engineering. They share a common profile: open-ended goals, variable environments and tolerance for occasional wrong turns.

True agents earn their complexity when the task space is too large or too variable to pre-define every step. Research synthesis across changing data sources. Dynamic scheduling with many interacting constraints. Multi-party negotiation workflows where the inputs shift mid-process.

The tolerance-for-error threshold matters. If a wrong decision costs the business money or damages a customer relationship, a human-in-the-loop pipeline is safer. If the cost of a wrong turn is low and the cost of a human doing it manually is high, the math starts to shift.

Consider a procurement team that needs to monitor dozens of supplier feeds, flag anomalies in pricing or delivery timelines, cross-reference those anomalies against contract terms and draft preliminary response emails. That workflow has enough variability and enough sub-task complexity that agentic behavior starts to earn its cost. A pipeline would need dozens of pre-defined branches to cover the same ground, and it would still miss edge cases.

Contrast that with a manufacturer routing inbound orders to the right fulfillment path. The inputs are known, the outputs are known and the whole thing runs on speed and accuracy. That's a decision tree, not an agent problem.

What to Ask Before Signing an AI Automation Contract

Before committing to any AI automation build, four questions will tell you whether the proposal matches your workflow or whether you're paying for architecture you don't need.

1. What is the system's failure mode? A good answer names a specific behavior: it stops and alerts a human, it retries with a fallback, it logs the error and skips. A vague answer ("it handles errors gracefully") is a red flag.

2. How is success measured? If the vendor can't name a specific workflow metric that will change (time per transaction, error rate, volume handled without human intervention), there's no way to know if the system is working. Measurement has to be defined before building, not after.

3. Is this architecture the right fit for this workflow, or is it what your team knows how to build? Vendors default to the frameworks they know. That's not always the wrong answer, but the buyer should ask the question. If you're not sure whether a workflow is worth automating in the first place, start there before evaluating architecture.

4. What does ongoing maintenance look like? Agentic systems require more monitoring than pipelines. "It runs itself" is not an accurate answer for any production system, and it's especially not accurate for one that makes its own decisions.

The Label Doesn't Matter but the Outcome Does

The label matters because it shapes expectations, pricing and maintenance planning. A system sold as an agent that is actually a pipeline will be evaluated against agent-level expectations it cannot meet. That mismatch is where disappointment lives.

The ROI failure pattern we see usually comes down to architecture mismatch rather than the technology itself: the system was built at the wrong level of complexity for the workflow, or success was never defined before anyone started building. Both of those are solvable problems, but only if someone asks the right questions before writing the first line of code.

The better question isn't "did I buy an agent?" It's "did this change how the business runs?"

Our approach starts with the workflow, not the technology. The architecture follows the problem. If you need to know where to start with AI automation, that's a better starting point than picking a framework.