Why AI Agents Built on the Same Models Get Wildly Different Results

G2 read through more than 2,950 verified reviews from January to mid-July and asked what people complain about most. The biggest gripe was accuracy, not price. Hallucinations and wrong answers made up 14% of everything reviewers said they disliked, which put it ahead of cost in a year when cost is all anyone wants to argue about.

It's worth knowing who those reviewers are, because it changes what the number means. Most of them are talking about the general-purpose assistants everybody has open in a browser tab. They aren't CX teams describing a support agent in production. They're people who ask a model something, get a confident answer back, and then have to work out for themselves whether it's true.

Then G2 went to four vendors who build production systems on those same models and asked what their failure rates actually look like. As low as 1%.

So what's really being compared isn't one model against another. It's a model you prompt yourself against a system somebody built around a model, same technology underneath, wildly different results. Maven was one of the four vendors G2 asked.

What's behind the gap

None of what those four described had anything to do with which model they'd picked. They all said more or less the same thing. Their systems work in steps instead of one shot. Answers get pulled from live data rather than whatever the model happens to remember from training. They score how often a task actually gets finished, and they rewrite prompts and logic at least once a week. One of them does it daily. That's all work done around the model, not to it.

The more useful thing they agreed on is where the effort goes. All four said the hard part isn't the everyday question, it's the edge case and the multi-step task, so they build systems that notice when they're out of their depth and hand the conversation to a person instead of guessing. Maven's line in the survey was that “autonomy without trustworthy fallback logic isn't a feature; it's a liability.”

That's most of the gap right there. When you prompt a model yourself, you're the one checking its work. A production system is built so that job never lands on the customer. G2 said it plainly: these vendors aren't seeing fewer failures, they're catching them before anyone notices.

The agent-versus-chatbot debate is mostly about words

G2 asked its four vendors straight out whether AI agents are more reliable than what the category still calls chatbots. They all said roughly the same thing, and it isn't great news for anyone selling on the word. In production the two aren't nearly as different as the marketing suggests, and swapping one label for the other doesn't fix anything by itself.

That matters more for buyers than it does for vendors. If you're shopping for the word “agent,” you're buying a label rather than a capability. One vendor calling its product an agent and another calling it something else tells you nothing about whether either one looks things up in your current knowledge, notices when it isn't sure, or gets checked against real conversations after launch. Those are properties of agentic AI in customer service, and not one of them arrives with the label.

The questions that do tell you something are about how the thing is built, and you can ask all five on a first call.

• Where do answers come from? Does it look things up in your live knowledge, or go off what the model absorbed in training?
• What happens when it isn't sure? Does it hand off, or guess in the same confident voice it uses when it's right?
• How often does someone check its answers against real conversations, and what are they measuring?
• Can your team change how the agent behaves, or does every tweak go back through the vendor?
• Can it actually do something in your systems, or can it only tell the customer what to do?

If a vendor can answer all five, somebody did the work. If they nail the first one and get fuzzy after that, they bought a model and wrapped a widget around it. There's a longer version of this list in our guide to evaluating AI agents for enterprise customer service.

The parts buyers skip

What those four vendors described is the same stack Maven's five-layer architecture is built on. The model's one layer. Everything that makes it reliable sits around it: what the agent knows, what it's allowed to do, how you check its work once it's live, and who holds the guardrails. Buyers usually shop the model, because that's the layer with the famous names attached. Everything G2's data points at is in the other four, and it's the reason autonomous resolution is a different thing from a good answer.

It's also why fast to deploy and reliable aren't opposites, whatever the category assumes. Building those layers is real engineering, but it happens once, inside the platform, instead of turning into a six-month project inside every customer's stack.

What it looks like when the work's been done

Mastermind got to 93 percent autonomous resolution in six weeks. K1x hit 80 percent in its first week. Rho held 95 percent CSAT while taking on 12 percent more contacts a month without hiring anyone. Quest built AskQ on Maven, plugged it into a knowledge base and case system that were already there with no data migration, and judges it on customer satisfaction rather than deflection.

Maven's own reviews point the same way. 4.8 out of 5 across 20 verified reviews, and it's worth reading what people actually bring up: one engine handling chat, email and voice instead of separate bots to keep in sync; knowledge retrieval that flags stale, duplicate and conflicting articles rather than answering from whichever one it found first; setup measured in days on top of Salesforce or Zendesk. All of that is the stuff around the model.

In G2's Fall 2026 reports, out August 25, Maven landed in the Mid-Market Grid for Customer Service Automation, the Mid-Market Results Index for AI Customer Support Agents, and the Implementation Index for AI Customer Support Agents. Index reports come from what customers say about the results they got and how the rollout went, not from how big or well known a company is. They measure the exact thing G2's wider data says is going wrong everywhere else.

The gap is the story

The number to pay attention to is the distance between 14% and 1%, because both sit on top of the same handful of models. Nothing about the technology explains a gap that big. What explains it is whether anyone bothered to build the parts that catch a bad answer before a customer sees it, which is why a resolution rate only means something once you know how it was measured.

Most buyers are still comparing model names. The four vendors G2 talked to gave that up a while ago.

If you want to see the layers rather than read about them, book a demo.